A user account is required in order to edit this wiki, but we've had to disable public user registrations due to spam.

To request an account, ask an autoconfirmed user on Chat (such as one of these permanent autoconfirmed members).

Encoding: Difference between revisions

From WHATWG Wiki
Jump to navigation Jump to search
(store some notes on encodings)
 
No edit summary
 
(14 intermediate revisions by 2 users not shown)
Line 1: Line 1:
This page tracks notes related to the [http://dvcs.w3.org/hg/encoding/raw-file/tip/Overview.html Encoding Standard]. See [[Web Encodings]] for some historical data with respect to encodings and their labels.  
This page tracks notes related to the [http://encoding.spec.whatwg.org/ Encoding Standard]. See [[Web Encodings]] for some historical data with respect to encodings and their labels.


==Implementations==
==Implementations==


http://code.google.com/p/stringencoding/ implements the standard in JavaScript
* http://code.google.com/p/stringencoding/ implements the standard in JavaScript
* http://coq.no/darcsweb/?r=flex-encoding is an experimental (f)lex implementation of the decoding algorithms


==Legacy implementations==
==Legacy implementations==
Line 24: Line 25:


* http://www-archive.mozilla.org/projects/intl/UniversalCharsetDetection.html
* http://www-archive.mozilla.org/projects/intl/UniversalCharsetDetection.html
* http://mxr.mozilla.org/mozilla-central/source/extensions/universalchardet/src/base/ (Gecko; tests are a few directories up)
* http://code.google.com/p/juniversalchardet/ (Gecko's detector in Java)
* http://code.google.com/p/juniversalchardet/ (Gecko's detector in Java)
* https://bugzilla.mozilla.org/show_bug.cgi?id=631751 (UTF-16 sniffing)
* https://bugzilla.mozilla.org/show_bug.cgi?id=631751 (UTF-16 sniffing in Gecko)
* http://trac.webkit.org/browser/trunk/Source/WebCore/loader/TextResourceDecoder.cpp (WebKit; is this all?)
 
===Gecko notes===
 
* Thai detection was not enabled [https://bugzilla.mozilla.org/show_bug.cgi?id=488426 before June 2009]
* Encodings not in the spec: iso-2022-cn, euc-tw
* GB2312 is supported, but superset gbk/gb18030 is not
* UTF-16 sniffing is HTML specific


==Misc==
==Misc==


* [http://zaynar.co.uk/docs/charset-encoding-xss.html XSS vulnerabilities with unusual character encodings]
* Test multiple BOMs
* Test multiple BOMs
==Labels==
Labels in Opera that are not in the spec: 'ansi_x3.4-1986', 'cn-gb', 'cp367', 'cp50220', 'cp51932', 'cp932', 'cp936', 'csascii', 'cscp50220', 'cscp51932', 'csinvariant', 'csiso646basic1983', 'csunicode', 'csunicode11', 'csunicode11utf7', 'csunicodeascii', 'csunicodejapanese', 'csunicodelatin1', 'csviscii', 'cswindows31j', 'euc-cn', 'euc-tw', 'extended_unix_code_packed_format_for_japanese', 'ibm367', 'invariant', 'iso-10646', 'iso-10646-j-1', 'iso-10646-ucs-2', 'iso-10646-ucs-basic', 'iso-10646-unicode-latin1', 'iso-2022-cn', 'iso-2022-jp-1', 'iso-celtic', 'iso-ir-199', 'iso-ir-226', 'iso-ir-6', 'iso646-us', 'iso8859-16', 'iso885916', 'iso_646.basic:1983', 'iso_646.irv:1991', 'iso_8859-10:1992', 'iso_8859-14', 'iso_8859-14:1998', 'iso_8859-16', 'iso_8859-16:2001', 'iso_8859-6-e', 'iso_8859-6-i', 'iso_8859-8-e', 'iso_8859-8-i', 'l10', 'l8', 'latin-9', 'latin10', 'latin8', 'microsoft-cp1250', 'microsoft-cp1251', 'microsoft-cp1252', 'microsoft-cp1253', 'microsoft-cp1254', 'microsoft-cp1255', 'microsoft-cp1256', 'microsoft-cp1257', 'microsoft-cp1258', 'ms932', 'ms936', 'ref', 'tis-620-2533', 'unicode-1-1', 'unicode-1-1-utf-7', 'us', 'utf-7', 'viscii', 'windows-936', 'x-mac-ce', 'x-mac-greek', 'x-mac-turkish', 'x-user-defined'


[[Category:Spec_coordination]]
[[Category:Spec_coordination]]

Latest revision as of 10:31, 7 August 2013

This page tracks notes related to the Encoding Standard. See Web Encodings for some historical data with respect to encodings and their labels.

Implementations

Legacy implementations

Gecko
http://mxr.mozilla.org/mozilla-central/source/intl/uconv/
Chromium
http://src.chromium.org/svn/trunk/deps/third_party/icu46/README.chromium
http://src.chromium.org/svn/trunk/deps/third_party/icu46/source/data/mappings/convrtrs.txt
http://src.chromium.org/svn/trunk/deps/third_party/icu46/source/data/mappings/ucmlocal.mk

Japanese encodings

Got these links after the standard was written:

Sniffing

Gecko notes

  • Thai detection was not enabled before June 2009
  • Encodings not in the spec: iso-2022-cn, euc-tw
  • GB2312 is supported, but superset gbk/gb18030 is not
  • UTF-16 sniffing is HTML specific

Misc

Labels

Labels in Opera that are not in the spec: 'ansi_x3.4-1986', 'cn-gb', 'cp367', 'cp50220', 'cp51932', 'cp932', 'cp936', 'csascii', 'cscp50220', 'cscp51932', 'csinvariant', 'csiso646basic1983', 'csunicode', 'csunicode11', 'csunicode11utf7', 'csunicodeascii', 'csunicodejapanese', 'csunicodelatin1', 'csviscii', 'cswindows31j', 'euc-cn', 'euc-tw', 'extended_unix_code_packed_format_for_japanese', 'ibm367', 'invariant', 'iso-10646', 'iso-10646-j-1', 'iso-10646-ucs-2', 'iso-10646-ucs-basic', 'iso-10646-unicode-latin1', 'iso-2022-cn', 'iso-2022-jp-1', 'iso-celtic', 'iso-ir-199', 'iso-ir-226', 'iso-ir-6', 'iso646-us', 'iso8859-16', 'iso885916', 'iso_646.basic:1983', 'iso_646.irv:1991', 'iso_8859-10:1992', 'iso_8859-14', 'iso_8859-14:1998', 'iso_8859-16', 'iso_8859-16:2001', 'iso_8859-6-e', 'iso_8859-6-i', 'iso_8859-8-e', 'iso_8859-8-i', 'l10', 'l8', 'latin-9', 'latin10', 'latin8', 'microsoft-cp1250', 'microsoft-cp1251', 'microsoft-cp1252', 'microsoft-cp1253', 'microsoft-cp1254', 'microsoft-cp1255', 'microsoft-cp1256', 'microsoft-cp1257', 'microsoft-cp1258', 'ms932', 'ms936', 'ref', 'tis-620-2533', 'unicode-1-1', 'unicode-1-1-utf-7', 'us', 'utf-7', 'viscii', 'windows-936', 'x-mac-ce', 'x-mac-greek', 'x-mac-turkish', 'x-user-defined'