WHATWG Wiki - User contributions [en]

HTML5Lib

2007-08-20T19:51:45Z

QvzNdc:

[http://jdawsona.ifrance.com/data/listen-to/ listen to holding out for a hero by bonnie] [http://lordsander.ifrance.com/topic/le-avventure.htm le avventure di charles darwin] [http://noxuhax.ifrance.com/content/view/sky-italia.htm sky italia keys] [http://oiyaoi.ifrance.com/images/small/gigi-d/ gigi d alessio testi] [http://highbulp.ifrance.com/images/small/pete-doherty/ pete doherty biography] [http://noxuhax.ifrance.com/content/view/un-trailer.htm un trailer e un sito pieni di f.e.a.r.] [http://oiyaoi.ifrance.com/images/small/iomega-gb/ iomega 160gb network hard disk] [http://voltia.ifrance.com/content/view/anthias.htm anthias] [http://jdawsona.ifrance.com/data/suzuki-gsx/ suzuki gsx r] [http://kalian42.ifrance.com/text/directory/il-cantante/ il cantante di jazz] [http://voltia.ifrance.com/content/view/tirolo-com.htm tirolo com] [http://demurediablo.ifrance.com/library/html/offerta-hotel/ offerta hotel roma] [http://oiyaoi.ifrance.com/images/small/video-cristina/ video cristina aquilera] [http://igaros.ifrance.com/lib/negozzi-animali/ negozzi animali] [http://speedofsoun.ifrance.com/text/directory/orovi.htm oro-vi] [http://speedofsoun.ifrance.com/text/directory/calcio-inter.htm calcio inter] [http://tizolaa.ifrance.com/web/bmw-serie/ bmw serie e90] [http://girlwho-is.ifrance.com/img/styles/la-gestione.htm la gestione delle risorse umane] [http://noxuhax.ifrance.com/content/view/foto-omicidi.htm foto omicidi] [http://toofarfrommaybe.ifrance.com/view/new/pc-amd/ pc amd athlon 64 2800] [http://lovereceier.ifrance.com/lib/campania-mare.htm campania mare] [http://singaporepets.ifrance.com/library/html/essenza-spa.htm essenza spa] [http://voltia.ifrance.com/content/view/todo-sobre.htm todo sobre los simpson] [http://toofarfrommaybe.ifrance.com/view/new/bareback/ bareback] [http://tat-ooin.ifrance.com/resources/articles/paggine-gialle.htm paggine gialle] [http://oiyaoi.ifrance.com/images/small/gold-stark/ gold stark] [http://speedofsoun.ifrance.com/text/directory/tastiera-e.htm tastiera e mouse computer] [http://kalian42.ifrance.com/text/directory/fullerton/ fullerton] [http://speedofsoun.ifrance.com/text/directory/epson-it.htm epson it] [http://yeeden.ifrance.com/img/styles/riduttore-segnale/ riduttore segnale autoradio] [http://toofarfrommaybe.ifrance.com/view/new/foto-penetrazione/ foto penetrazione] [http://highbulp.ifrance.com/images/small/amd-/ amd 64 bit 3500] [http://lovereceier.ifrance.com/lib/canon-eos.htm canon eos 20] [http://speedofsoun.ifrance.com/text/directory/der-herr.htm der herr ist mit mir] [http://lovereceier.ifrance.com/lib/serena-del.htm serena del grande fratello nuda] [http://highbulp.ifrance.com/images/small/kodak-easyshare/ kodak easyshare cx7525] [http://tat-ooin.ifrance.com/resources/articles/renee-o.htm renee o connor nuda] [http://deviantrus.ifrance.com/topic/ilpalermo-calcio/ ilpalermo calcio it] [http://toofarfrommaybe.ifrance.com/view/new/segnale-telecomando/ segnale telecomando] [http://noxuhax.ifrance.com/content/view/asus-ngt.htm asus n6600gt td 128m a] [http://broadwaylili.ifrance.com/content/view/geforce-.htm geforce 6600 256mb pci express] [http://akmokanzen.ifrance.com/images/small/creazione-sito.htm creazione sito internet alassio] [http://oiyaoi.ifrance.com/images/small/tufillo/ tufillo] [http://demurediablo.ifrance.com/library/html/la-danza/ la danza del vampiro] [http://demurediablo.ifrance.com/library/html/dakar/ dakar] [http://girlwho-is.ifrance.com/img/styles/seagate-barracuda.htm seagate barracuda 200] [http://deviantrus.ifrance.com/topic/knauf/ knauf] [http://highbulp.ifrance.com/images/small/ibook-/ ibook 12 apple] [http://singaporepets.ifrance.com/library/html/velasco-juan.htm velasco juan] [http://broadwaylili.ifrance.com/content/view/video-di.htm video di kissing you di des ree] [http://yeeden.ifrance.com/img/styles/thrustmaster-firestorm/ thrustmaster firestorm] [http://lordsander.ifrance.com/topic/love-don.htm love don t cost a think] [http://lordsander.ifrance.com/topic/braun-silk.htm braun silk epil] [http://girlwho-is.ifrance.com/img/styles/concerto-di.htm concerto di zucchero dove e quando] [http://lordsander.ifrance.com/topic/eutropio-morte.htm eutropio morte pompeo] [http://oiyaoi.ifrance.com/images/small/para-llenarme/ para llenarme] [http://girlwho-is.ifrance.com/img/styles/vittorio-zucconi.htm vittorio zucconi] [http://voltia.ifrance.com/content/view/la-ballata.htm la ballata del boia] [http://jdawsona.ifrance.com/data/download-inno/ download inno reggina] [http://igaros.ifrance.com/lib/a-marzo/ a marzo il pad saturn su playstation 2] [http://igaros.ifrance.com/lib/justine-il/ justine il centro della passione] [http://speedofsoun.ifrance.com/text/directory/www-bmw.htm www bmw mini it] [http://singaporepets.ifrance.com/library/html/limosine.htm limosine] [http://igaros.ifrance.com/lib/john-scofield/ john scofield quiet] [http://tat-ooin.ifrance.com/resources/articles/il-triangolo.htm il triangolo delle bermuda] [http://lordsander.ifrance.com/topic/professoresse-calde.htm professoresse calde] [http://jdawsona.ifrance.com/data/cantrel/ cantrel] [http://singaporepets.ifrance.com/library/html/spartiti-buoni.htm spartiti buoni o cattivi] [http://broadwaylili.ifrance.com/content/view/nascita-di.htm nascita di una madre] [http://lordsander.ifrance.com/topic/giacca-i.htm giacca i] [http://deviantrus.ifrance.com/topic/patto-col/ patto col diavolo] [http://girlwho-is.ifrance.com/img/styles/lancia-y.htm lancia y elefantino blu 1998] [http://singaporepets.ifrance.com/library/html/dedica-d.htm dedica d amore] [http://akmokanzen.ifrance.com/images/small/samsung-tv.htm samsung tv color stereo 29] [http://oiyaoi.ifrance.com/images/small/flint/ flint] [http://oiyaoi.ifrance.com/images/small/c-olympus/ c480 olympus] [http://igaros.ifrance.com/lib/crim/ crim] [http://lordsander.ifrance.com/topic/epson-stylus.htm epson stylus rx] [http://lordsander.ifrance.com/topic/ddr-.htm ddr 400 mhz 512 mb] [http://oiyaoi.ifrance.com/images/small/mmc-dual/ mmc dual voltage da 512mb] [http://oiyaoi.ifrance.com/images/small/telefonino-nokia/ telefonino nokia 6170] [http://toofarfrommaybe.ifrance.com/view/new/red-tatoo/ red tatoo] [http://voltia.ifrance.com/content/view/volo-per.htm volo per praga] [http://lovereceier.ifrance.com/lib/hd-gb.htm hd 20gb] [http://voltia.ifrance.com/content/view/waltham.htm waltham] [http://jdawsona.ifrance.com/data/dvdplayer/ dvdplayer] [http://akmokanzen.ifrance.com/images/small/convertitore-mpc.htm convertitore mpc mp3] [http://noxuhax.ifrance.com/content/view/trieste-cantico.htm trieste cantico damore] [http://highbulp.ifrance.com/images/small/troia-it/ troia it] [http://lordsander.ifrance.com/topic/http-www.htm http www 222z com] [http://deviantrus.ifrance.com/topic/mkt-srl/ mkt srl] [http://tat-ooin.ifrance.com/resources/articles/orecchino-argento.htm orecchino argento] [http://girlwho-is.ifrance.com/img/styles/motorola-cellulare.htm motorola cellulare] [http://oiyaoi.ifrance.com/images/small/porri/ porri] [http://jdawsona.ifrance.com/data/ready-/ ready 2 rumble boxing] [http://girlwho-is.ifrance.com/img/styles/aspetti-legali.htm aspetti legali associazione no profit] [http://akmokanzen.ifrance.com/images/small/racconti-collant.htm racconti collant] [http://singaporepets.ifrance.com/library/html/sarica.htm sarica] [http://singaporepets.ifrance.com/library/html/ad-aware.htm ad aware ad aware] [http://tat-ooin.ifrance.com/resources/articles/registratore-vocale.htm registratore vocale mp3 usb] [http://deviantrus.ifrance.com/topic/corso-di/ corso di formazione a gardaland] [http://igaros.ifrance.com/lib/pallone-basket/ pallone basket nike] [http://tat-ooin.ifrance.com/resources/articles/blu-scorpion.htm blu scorpion] [http://deviantrus.ifrance.com/topic/videocamera-digitale/ videocamera digitale jvc] [http://oiyaoi.ifrance.com/images/small/gf-caterina/ gf caterina] [http://akmokanzen.ifrance.com/images/small/iee.htm iee] [http://speedofsoun.ifrance.com/text/directory/trowbridge-john.htm trowbridge, john townsend] [http://lovereceier.ifrance.com/lib/soluzione-ecommerce.htm soluzione ecommerce] [http://highbulp.ifrance.com/images/small/joey-ramone/ joey ramone] [http://akmokanzen.ifrance.com/images/small/lacie-.htm lacie 160 usb] [http://broadwaylili.ifrance.com/content/view/caterina-massari.htm caterina massari] [http://tizolaa.ifrance.com/web/tony-presidio/ tony presidio] [http://lovereceier.ifrance.com/lib/www-todo.htm www todo juego com] [http://lovereceier.ifrance.com/lib/www-fimp.htm www fimp it] [http://tat-ooin.ifrance.com/resources/articles/acer-.htm acer 19 lcd al1912s] [http://yeeden.ifrance.com/img/styles/un-medico/ un medico in famiglia. prima serie. vol. 06] [http://yeeden.ifrance.com/img/styles/gabriella-abate/ gabriella abate] [http://broadwaylili.ifrance.com/content/view/ritorno-alla.htm ritorno alla quarta dimensione] [http://singaporepets.ifrance.com/library/html/il-marzo.htm il marzo di ripley] [http://jdawsona.ifrance.com/data/camera-vimodrone/ camera vimodrone] [http://akmokanzen.ifrance.com/images/small/foto-clisteri.htm foto clisteri] [http://voltia.ifrance.com/content/view/spiaggia.htm spiaggia] [http://akmokanzen.ifrance.com/images/small/zoom-sigma.htm zoom sigma 70-300] [http://broadwaylili.ifrance.com/content/view/cradle-hp.htm cradle hp] [http://jdawsona.ifrance.com/data/cuffia-dj/ cuffia dj sony] [http://kalian42.ifrance.com/text/directory/bordo/ bordo] [http://lovereceier.ifrance.com/lib/fly-simulator.htm fly simulator] [http://speedofsoun.ifrance.com/text/directory/rosi.htm rosi] [http://singaporepets.ifrance.com/library/html/candy-dulfer.htm candy dulfer. live at montreux 2002] [http://yeeden.ifrance.com/img/styles/dilatati/ dilatati] [http://toofarfrommaybe.ifrance.com/view/new/il-popolo/ il popolo degli abissi libri] [http://toofarfrommaybe.ifrance.com/view/new/epson-d/ epson d88] [http://akmokanzen.ifrance.com/images/small/www-costantino.htm www costantino l italiano it] [http://speedofsoun.ifrance.com/text/directory/denon-avr.htm denon avr-1906] [http://tat-ooin.ifrance.com/resources/articles/automodelli-elettrici.htm automodelli elettrici] [http://jdawsona.ifrance.com/data/gps-palmare/ gps palmare gprs gsm] [http://girlwho-is.ifrance.com/img/styles/annunci-con.htm annunci con negri] [http://yeeden.ifrance.com/img/styles/snoop-dog/ snoop dog e nelly] [http://deviantrus.ifrance.com/topic/costi-ionita/ costi ionita] [http://kalian42.ifrance.com/text/directory/impreza-subaru/ impreza subaru] [http://girlwho-is.ifrance.com/img/styles/maurizio-vandelli.htm maurizio vandelli] [http://toofarfrommaybe.ifrance.com/view/new/nokia/ nokia 5125] [http://lordsander.ifrance.com/topic/casse-jamo.htm casse jamo] [http://lovereceier.ifrance.com/lib/tuscany-farmhouse.htm tuscany farmhouse] [http://broadwaylili.ifrance.com/content/view/rinascente.htm rinascente] [http://tat-ooin.ifrance.com/resources/articles/happy-fathers.htm happy fathers day pictures com] [http://highbulp.ifrance.com/images/small/dvdr-panasonic/ dvd-r panasonic] [http://lordsander.ifrance.com/topic/diiodo.htm diiodo-] [http://oiyaoi.ifrance.com/images/small/tennis-borsa/ tennis borsa] [http://akmokanzen.ifrance.com/images/small/www-soap.htm www soap beautiful it] [http://voltia.ifrance.com/content/view/cpminnj.htm cpminnj] [http://igaros.ifrance.com/lib/pellicce/ pellicce] [http://toofarfrommaybe.ifrance.com/view/new/voli-malta/ voli malta] [http://akmokanzen.ifrance.com/images/small/max-pin.htm max pin] [http://oiyaoi.ifrance.com/images/small/agriturismo-urbino/ agriturismo urbino] [http://oiyaoi.ifrance.com/images/small/wanda/ wanda] [http://deviantrus.ifrance.com/topic/www-musica/ www musica rok com] [http://demurediablo.ifrance.com/library/html/tuscany-stones/ tuscany stones] [http://kalian42.ifrance.com/text/directory/hard-amateur/ hard amateur] [http://akmokanzen.ifrance.com/images/small/log-seca.htm log seca 2] [http://tat-ooin.ifrance.com/resources/articles/spiaggie-di.htm spiaggie di cuba] [http://tat-ooin.ifrance.com/resources/articles/stranger-paradise.htm stranger paradise] [http://yeeden.ifrance.com/img/styles/bad-compani/ bad compani] [http://oiyaoi.ifrance.com/images/small/concessionario-kia/ concessionario kia milano] [http://yeeden.ifrance.com/img/styles/crema-corpo/ crema corpo kenzo prodotti] [http://yeeden.ifrance.com/img/styles/www-sexogratis/ www sexogratis com] [http://voltia.ifrance.com/content/view/lacie-mobile.htm lacie mobile 30gb] [http://toofarfrommaybe.ifrance.com/view/new/sara-ass/ sara ass] [http://oiyaoi.ifrance.com/images/small/severina/ severina 2004] [http://tizolaa.ifrance.com/web/riduzione-aliquote/ riduzione aliquote ire 2005] [http://tat-ooin.ifrance.com/resources/articles/temi-psicologia.htm temi psicologia] [http://lordsander.ifrance.com/topic/kit-allarme.htm kit allarme casa] [http://highbulp.ifrance.com/images/small/registi/ registi] [http://speedofsoun.ifrance.com/text/directory/signora-con.htm signora con nero] [http://akmokanzen.ifrance.com/images/small/moonlight-omer.htm moonlight omer] [http://demurediablo.ifrance.com/library/html/naturismo-spagna/ naturismo spagna] [http://singaporepets.ifrance.com/library/html/attimi-di.htm attimi di paura] [http://lordsander.ifrance.com/topic/apple-mac.htm apple mac x tiger] [http://broadwaylili.ifrance.com/content/view/shiseido-matifying.htm shiseido matifying moisturizer oil-free] [http://highbulp.ifrance.com/images/small/swat-cd/ swat3 cd key] [http://akmokanzen.ifrance.com/images/small/fotos-de.htm fotos de fantasmas] [http://deviantrus.ifrance.com/topic/glossario-vino/ glossario vino] [http://tizolaa.ifrance.com/web/sacco-bruno/ sacco, bruno] [http://noxuhax.ifrance.com/content/view/programma-traduzione.htm programma traduzione] [http://speedofsoun.ifrance.com/text/directory/milly-d.htm milly d abbraccio] [http://jdawsona.ifrance.com/data/cs-cp/ cs cp18] [http://toofarfrommaybe.ifrance.com/view/new/neci/ nec341i] [http://deviantrus.ifrance.com/topic/simboli-esoterici/ simboli esoterici] [http://demurediablo.ifrance.com/library/html/miso/ miso-] [http://demurediablo.ifrance.com/library/html/darktown/ darktown] [http://toofarfrommaybe.ifrance.com/view/new/piccoli-prestiti/ piccoli prestiti] [http://singaporepets.ifrance.com/library/html/annuncio-auto.htm annuncio auto toscana] [http://tat-ooin.ifrance.com/resources/articles/concerto-baglioni.htm concerto baglioni] [http://akmokanzen.ifrance.com/images/small/rasmuss.htm rasmuss] [http://girlwho-is.ifrance.com/img/styles/sennheiser-px.htm sennheiser px 200] [http://tizolaa.ifrance.com/web/trasmettitore-audio/ trasmettitore audio video telecomando] [http://speedofsoun.ifrance.com/text/directory/civilization.htm civilization 3] [http://yeeden.ifrance.com/img/styles/dpr/ dpr 560 1995] [http://toofarfrommaybe.ifrance.com/view/new/firmware-thomson/ firmware thomson] [http://demurediablo.ifrance.com/library/html/ascolta-adagio/ ascolta adagio di lara fabian] = HTML5Lib =

[http://code.google.com/p/html5lib/ HTML5Lib] is a project to create both a Python-based and Ruby-based implementations of various parts of the WHATWG spec, in particular, a tokenizer, a parser, and a serializer. It is '''not''' an offical WHATWG project, however we plan to use this wiki to document and discuss the library design. The code is avaliable under an open-source MIT license.

== SVN ==
Please commit often with sort of detailed descriptions of what you did. If you want to make sure you're not going to redo ask on the [http://groups.google.com/group/html5lib-discuss mailing list]. For questions that could benefit from quick turnaround, talk to people on #whatwg.

== General ==

In comments "XXX" indicates something that has yet to be done. Something might be wrong, has not yet been written and other things in that general direction.

In comments "AT" indicates that the comment documents an alternate implementation technique or strategy.

== HTMLTokenizer ==

The tokenizer is controlled by a single HTMLTokenizer class stored in tokenizer.py at the moment. You initialize the HTMLTokenizer with a stream argument that holds an HTMLInputStream. You can iterate over the object created to get tokens back.

Currently tokens are objects, they will become dicts.

=== Interface ===

The parser needs to change the self.contentModelFlag attribute which affects how certain states are handled.

=== Issues ===
* Use of if statements in the states may be suboptimal (but we should time this)

== HTMLParser ==

=== Profiling on web-apps.htm ===

I did some profiling on web-apps.htm which is a rather large document. Based on that I already changed a number of things which speed us up a bit. Below are some things to consider for future revisions:

* utils.MethodDispatcher is invoked way too often. By pre declaring some of it in InBody I managed to decrease the amount of invocatoins by over 24.000, but InBody.__init__ is invoked about 7000 times for web-apps.htm so that amount could be higher. Not sure how to put them somewhere else though. First thing I tried was HTMLParser but references get all messed up then...
: We should be able to store a single instance of each InsertionMode rather than creating a new one every time the mode switches. Hopefully we have been disiplined enough not to keep any state in those classes so the change should be painless.
:: That's an interesting idea. How would that work? [[User:Annevk|Annevk]] 12:49, 25 December 2006 (UTC)
::: I got an idea on how it might work and it worked! Still about 3863 invocations to utils.MethodDispatcher but it takes 0.000 CPU seconds. I suppose we can decrease that amount even more, but I wonder if it's worth it. [[User:Annevk|Annevk]] 11:37, 26 December 2006 (UTC)

* 713194 calls to __contains__ in sets.py makes us slow. Takes about 1.0x CPU seconds.
: I've just switched to the built-in sets type. hopefully this will help a bit [[User:Jgraham|Jgraham]] 00:30, 25 December 2006 (UTC)
:: It did. (Not surprisingly when 700.000 method calls are gone...) [[User:Annevk|Annevk]] 12:49, 25 December 2006 (UTC)

* 440382 calls to char in tokenizer.py is the runner up with 0.8x CPU seconds.
: This is now the largest time consumer. [[User:Annevk|Annevk]] 12:49, 25 December 2006 (UTC)

* dataState in tokenizer.py with 0.7 CPU seconds is next.
: This is now at 0.429 CPU seconds. Probably becase the tokenizer switched to dicts instead of custom Token objects. [[User:Annevk|Annevk]]

* __iter__ in tokenizer.py with 0.59x CPU seconds...

* Creation of all node objects in web-apps takes .57x CPU seconds.

* etc.

== Testcases ==
Testcases are under the /tests directory. They require [http://cheeseshop.python.org/pypi/simplejson simplejson]. New code should not be checked in if it regresses previously functional unit tests. Similarly, new tests that don't pass should not be checked in without both informing others on the [http://groups.google.com/group/html5lib-discuss mailing list] and a concrete plan. Ideally new features should be accompanied by new unit tests for those features. Documentation of the test format is available at [[Parser_tests]].

[[Category:Implementations]]

HTML5Lib

2007-08-13T01:36:17Z

QvzNdc:

[http://wainfan.t35.com/view/new/il-moto.htm il moto cros] [http://grahrens.125mb.com/view/new/pac-eco/ pac eco 70] [http://oedi.freehostia.com/text/directory/brachi/ brachi-] [http://coarmago.t35.com/content/view/lg-tv/ lg tv rz-20lz50] [http://grahrens.125mb.com/view/new/juliana-lopes/ juliana lopes bbb4] [http://boreldar.t35.com/text/directory/panasonic-viera.htm panasonic viera 42pa50e] [http://gawelzba.t35.com/library/html/maria-paola.htm maria paola] [http://greensticky.freehostia.com/library/html/jvc-mg/ jvc mg30] [http://lorcaxon.homeblock.com/data/gino-vannelli/ gino vannelli] [http://oedi.freehostia.com/text/directory/il-canale/ il canale degli angeli] [http://www.freewebtown.com/prncessprncipe/library/html/roper-pcmcia.htm roper pcmcia] [http://boreldar.t35.com/text/directory/rubeola.htm rubeola] [http://beshennayabelka.freehostia.com/content/view/agente-cody.htm agente cody banks] [http://www.freewebtown.com/boogiejames/topic/bd-spencer.htm bd spencer] [http://failtoexpress.freehostia.com/html/configurazione-vi.htm configurazione v3i] [http://flheddin.125mb.com/content/view/veterinari-ravenna.htm veterinari ravenna] [http://mauballe.125mb.com/web/dany-brillant/ dany brillant] [http://flheddin.125mb.com/content/view/tel-one.htm tel one] [http://benevolently.freehostia.com/content/view/di/ di 614] [http://benevolently.freehostia.com/content/view/volevo-i/ volevo i pantaloni] [http://scaleslea.freewebpage.org/text/directory/y-tu/ y tu mama tambien - anche tua madre] [http://mcwardlo.t35.com/library/html/copy/ copy] [http://coarmago.t35.com/content/view/body-performance/ body performance] [http://www.freewebtown.com/prncessprncipe/library/html/www-cis.htm www cis it] [http://licohwar.homeblock.com/library/html/nokia-/ nokia 7610 imp] [http://coarmago.t35.com/content/view/collare-cuoio/ collare cuoio] [http://lorcaxon.homeblock.com/data/lexmark-inchiostro/ lexmark inchiostro] [http://coarmago.t35.com/content/view/roksan/ roksan] [http://flheddin.125mb.com/content/view/giochi-per.htm giochi per motorola c550] [http://huruple.125mb.com/text/directory/sissoko-let/ sissoko let the music take control] [http://wainfan.t35.com/view/new/asus-mypal.htm asus mypal a730] [http://greensticky.freehostia.com/library/html/echinopsis/ echinopsis] [http://flheddin.125mb.com/content/view/ricetta-riso.htm ricetta riso] [http://kopapguw.homeblock.com/topic/regioni-d/ regioni d italia] [http://getraylo.freeweb7.com/img/styles/batteria-acer.htm batteria acer aspire 1680] [http://www.freewebtown.com/boogiejames/topic/kasaioccidental.htm kasai-occidental] [http://beshennayabelka.freehostia.com/content/view/il-sindacalista.htm il sindacalista] [http://licohwar.homeblock.com/library/html/rossana-hentai/ rossana hentai] [http://benevolently.freehostia.com/content/view/decatlon-articolo/ decatlon articolo sportivi] [http://zoelverd.freeweb7.com/description/lib/attrice-italiana/ attrice italiana g] [http://coarmago.t35.com/content/view/mystery-island/ mystery island] [http://licohwar.homeblock.com/library/html/la-bella/ la bella otero] [http://flheddin.125mb.com/content/view/super-merk.htm super merk2] [http://getm0ney.freewebpage.org/lib/nuova-toyota/ nuova toyota corolla benzina auto nuove] [http://beshennayabelka.freehostia.com/content/view/multilevel-governance.htm multi-level governance] [http://mauballe.125mb.com/web/usb-adapter/ usb adapter ipaq] [http://oedi.freehostia.com/text/directory/gioiellerie-morellato/ gioiellerie morellato a salerno] [http://boreldar.t35.com/text/directory/hotel-castello.htm hotel castello santonio] [http://zoelverd.freeweb7.com/description/lib/staffa-a/ staffa a muro] [http://huruple.125mb.com/text/directory/annuncio-cani/ annuncio cani] [http://wainfan.t35.com/view/new/galerias-de.htm galerias de mujeres] [http://lorcaxon.homeblock.com/data/ogni-volta/ ogni volta che vedo il mare] [http://getraylo.freeweb7.com/img/styles/unoaerre-italia.htm unoaerre italia spa] [http://lorcaxon.homeblock.com/data/nokia-/ nokia 7710 grey pearl] [http://gawelzba.t35.com/library/html/socket-.htm socket 478 i] [http://oedi.freehostia.com/text/directory/enrico-vincenzi/ enrico vincenzi] [http://coarmago.t35.com/content/view/lexar-compact/ lexar compact flash 80x] [http://coarmago.t35.com/content/view/le-sorprese/ le sorprese dellamore] [http://getraylo.freeweb7.com/img/styles/ganggbang.htm ganggbang] [http://grahrens.125mb.com/view/new/only-big/ only big real great beautiful natural ti] [http://coarmago.t35.com/content/view/arcuri-panorama/ arcuri panorama] [http://www.freewebtown.com/starbriteyellow/resources/articles/cartoonnetvork.htm cartoonnetvork] [http://www.freewebtown.com/starbriteyellow/resources/articles/sarcinella.htm sarcinella] [http://lorcaxon.homeblock.com/data/sevillana/ sevillana] [http://beshennayabelka.freehostia.com/content/view/telegiornali-sexi.htm telegiornali sexi] [http://scaleslea.freewebpage.org/text/directory/donne-mature/ donne mature 50 60anni] [http://zoelverd.freeweb7.com/description/lib/super-furry/ super furry animals. phantom power] [http://failtoexpress.freehostia.com/html/ippolito-nino.htm ippolito nino] [http://www.freewebtown.com/prncessprncipe/library/html/www-liberoit.htm www liberoit] [http://grahrens.125mb.com/view/new/lavatrice-gettone/ lavatrice gettone] [http://www.freewebtown.com/starbriteyellow/resources/articles/p.htm p4 3 0] [http://oedi.freehostia.com/text/directory/hp/ hp 710] [http://zoelverd.freeweb7.com/description/lib/shema-israel/ shema israel] [http://failtoexpress.freehostia.com/html/artrosi-cervicale.htm artrosi cervicale] [http://greensticky.freehostia.com/library/html/junior-jack/ junior jack robert smith] [http://wainfan.t35.com/view/new/kardia-mou.htm kardia mou min anisihis] [http://www.freewebtown.com/boogiejames/topic/losredondos.htm losredondos] [http://huruple.125mb.com/text/directory/psycho-sistem/ psycho sistem of a down] [http://lorcaxon.homeblock.com/data/weston-edward/ weston, edward (ingegnere)] [http://huruple.125mb.com/text/directory/fifa-com/ fifa99 com] [http://mauballe.125mb.com/web/ixus/ ixus 5] [http://boreldar.t35.com/text/directory/delta-force.htm delta force2] [http://beshennayabelka.freehostia.com/content/view/mambomania.htm mambomania] [http://wainfan.t35.com/view/new/thi.htm thi] [http://www.freewebtown.com/boogiejames/topic/anna-e.htm anna e il re] [http://flheddin.125mb.com/content/view/elisha-cuthbert.htm elisha cuthbert] [http://gawelzba.t35.com/library/html/lettore-karaoke.htm lettore karaoke divx] [http://getm0ney.freewebpage.org/lib/sessuologo-messina/ sessuologo messina] [http://scaleslea.freewebpage.org/text/directory/versilia-albergo/ versilia albergo] [http://huruple.125mb.com/text/directory/no-tiene/ no tiene dinero] [http://zoelverd.freeweb7.com/description/lib/sony-radio/ sony radio portatile] [http://scaleslea.freewebpage.org/text/directory/mi-chiamo/ mi chiamo mimi] [http://greensticky.freehostia.com/library/html/dvd-samsung/ dvd samsung vhs] [http://failtoexpress.freehostia.com/html/atisapphire-radeon.htm ati-sapphire radeon 9600 se] [http://www.freewebtown.com/starbriteyellow/resources/articles/canon-.htm canon - obiettivo zoom ef 28-135mm is] [http://mcwardlo.t35.com/library/html/www-adecco/ www adecco it] [http://failtoexpress.freehostia.com/html/soundblaster-live.htm soundblaster live 5 1] [http://failtoexpress.freehostia.com/html/dvd-srink.htm dvd srink download] [http://gawelzba.t35.com/library/html/sennheiser.htm sennheiser 30] [http://kopapguw.homeblock.com/topic/film-di/ film di dragonball z] [http://wainfan.t35.com/view/new/giovanna.htm giovanna 41] [http://flheddin.125mb.com/content/view/sexy-sexy.htm sexy sexy photo] [http://www.freewebtown.com/prncessprncipe/library/html/www-go.htm www go party it] [http://grahrens.125mb.com/view/new/magnum-pi/ magnum p.i. stagione 2] [http://wainfan.t35.com/view/new/sarah-kane.htm sarah kane libri] [http://scaleslea.freewebpage.org/text/directory/majhon/ majhon] [http://zoelverd.freeweb7.com/description/lib/porno/ porno] [http://beshennayabelka.freehostia.com/content/view/tine.htm tine] [http://zoelverd.freeweb7.com/description/lib/ci-/ ci 1300 fb] [http://oedi.freehostia.com/text/directory/masterizzatore-dvd/ masterizzatore dvd per mac] [http://huruple.125mb.com/text/directory/www-kladi/ www kladi it] [http://getm0ney.freewebpage.org/lib/supporto-tv/ supporto tv plasma parete] [http://getraylo.freeweb7.com/img/styles/maria-cristina.htm maria cristina blu] [http://oedi.freehostia.com/text/directory/singoli/ singoli] [http://mauballe.125mb.com/web/usb-motorola/ usb motorola pc 220] [http://oedi.freehostia.com/text/directory/brigata-cani/ brigata cani da birra] [http://selemku.t35.com/text/directory/link-siti.htm link siti porrno] [http://getm0ney.freewebpage.org/lib/banca-mutuo/ banca mutuo casa] [http://www.freewebtown.com/starbriteyellow/resources/articles/shox-.htm shox 2 45 nike calzature] [http://selemku.t35.com/text/directory/uomo-giacca.htm uomo giacca] [http://www.freewebtown.com/prncessprncipe/library/html/lee-ritenour.htm lee ritenour. live in montreal] [http://selemku.t35.com/text/directory/ericsson-z.htm ericsson z1010] [http://www.freewebtown.com/starbriteyellow/resources/articles/montevarchi.htm montevarchi] [http://zoelverd.freeweb7.com/description/lib/midi-no/ midi no me quiero enamorar] [http://www.freewebtown.com/starbriteyellow/resources/articles/bellisimi.htm bellisimi] [http://mcwardlo.t35.com/library/html/not-that/ not that kind anastacia] [http://saline-joy.freehostia.com/web/air-silver.htm air silver] [http://flheddin.125mb.com/content/view/hoobastanks.htm hoobastanks] [http://getraylo.freeweb7.com/img/styles/rolf.htm rolf] [http://www.freewebtown.com/boogiejames/topic/scarica-copertina.htm scarica copertina cd] [http://zoelverd.freeweb7.com/description/lib/ggggggggggggggggggggggggggggggggg/ ggggggggggggggggggggggggggggggggg] [http://www.freewebtown.com/prncessprncipe/library/html/la-donna.htm la donna dell amico mio testo carlos] [http://mcwardlo.t35.com/library/html/guardie-giurate/ guardie giurate] [http://greensticky.freehostia.com/library/html/arius/ arius] [http://www.freewebtown.com/prncessprncipe/library/html/lcd-televisori.htm lcd televisori samsung] [http://selemku.t35.com/text/directory/cagliari-immagini.htm cagliari immagini] [http://mauballe.125mb.com/web/de-ce/ de ce ma minti remix] [http://selemku.t35.com/text/directory/studi-consulenti.htm studi consulenti fiscali trapani] [http://selemku.t35.com/text/directory/vendita-fiori.htm vendita fiori on line] [http://scaleslea.freewebpage.org/text/directory/preamplificatori-audio/ preamplificatori audio] [http://www.freewebtown.com/starbriteyellow/resources/articles/siglia-di.htm siglia di buffy] [http://gawelzba.t35.com/library/html/annuncio-privati.htm annuncio privati incontri] [http://boreldar.t35.com/text/directory/ufficio-campobasso.htm ufficio campobasso] [http://benevolently.freehostia.com/content/view/wd-oregon/ wd338 oregon scientific cordless] [http://getraylo.freeweb7.com/img/styles/jolie-nuda.htm jolie nuda] [http://benevolently.freehostia.com/content/view/giulia-michelini/ giulia michelini] [http://www.freewebtown.com/prncessprncipe/library/html/trentino-donne.htm trentino donne] [http://getm0ney.freewebpage.org/lib/gente-motore/ gente motore] [http://mauballe.125mb.com/web/fbi-operazione/ f.b.i. operazione pakistan] [http://mauballe.125mb.com/web/buonanotte-fiorellino/ buonanotte fiorellino de gregori] [http://beshennayabelka.freehostia.com/content/view/tavoletta-grafica.htm tavoletta grafica wacom a6] [http://www.freewebtown.com/boogiejames/topic/muvo-black.htm muvo black] [http://flheddin.125mb.com/content/view/voli-israele.htm voli israele last minute] [http://failtoexpress.freehostia.com/html/hd-autoalimentato.htm hd autoalimentato] [http://selemku.t35.com/text/directory/jenny-garth.htm jenny garth] [http://www.freewebtown.com/boogiejames/topic/capitan-uncino.htm capitan uncino bennato] [http://kopapguw.homeblock.com/topic/volo-cesenatico/ volo cesenatico] [http://mauballe.125mb.com/web/hotel-luino/ hotel luino] [http://beshennayabelka.freehostia.com/content/view/multifunzione-laser.htm multifunzione laser color epson] [http://gawelzba.t35.com/library/html/colori-che.htm colori che esplodono] [http://flheddin.125mb.com/content/view/master-of.htm master of orion 2] [http://saline-joy.freehostia.com/web/dancan.htm dancan] [http://kopapguw.homeblock.com/topic/impanatura/ impanatura] [http://kopapguw.homeblock.com/topic/lacie-safe/ lacie safe mobile hard drive 80gb] [http://lorcaxon.homeblock.com/data/di-/ di natale] [http://benevolently.freehostia.com/content/view/download-gratis/ download gratis driver stampanti] [http://gawelzba.t35.com/library/html/star-wars.htm star wars la vendetta dei sith] [http://zoelverd.freeweb7.com/description/lib/hp/ hp 715] [http://saline-joy.freehostia.com/web/plasma-.htm plasma 42 pollici a parete tv televisori] [http://zoelverd.freeweb7.com/description/lib/dsc-h/ dsc h1 sony] [http://oedi.freehostia.com/text/directory/punto-/ punto 2005 mi] [http://www.freewebtown.com/boogiejames/topic/bouble-bubbles.htm bouble bubbles] [http://boreldar.t35.com/text/directory/alloggi-venezia.htm alloggi venezia] [http://boreldar.t35.com/text/directory/spille-di.htm spille di sicurezza] [http://mauballe.125mb.com/web/radioactiva/ radioactiva] [http://lorcaxon.homeblock.com/data/maserati-coupe/ maserati coupe cambiocorsa] [http://benevolently.freehostia.com/content/view/real-doll/ real doll com] [http://selemku.t35.com/text/directory/il-porto.htm il porto del vizio] [http://lorcaxon.homeblock.com/data/tonino-carotone/ tonino carotone] [http://mauballe.125mb.com/web/festival-crociera/ festival crociera] [http://www.freewebtown.com/starbriteyellow/resources/articles/la-serenissima.htm la serenissima 2004] [http://saline-joy.freehostia.com/web/cxmi-acer.htm c311xmi acer] [http://www.freewebtown.com/starbriteyellow/resources/articles/www-bambola.htm www bambola ramona] [http://grahrens.125mb.com/view/new/bando-forze/ bando forze armate] [http://www.freewebtown.com/starbriteyellow/resources/articles/camcorder.htm camcorder] [http://saline-joy.freehostia.com/web/congelatore-verticale.htm congelatore verticale aeg] [http://www.freewebtown.com/boogiejames/topic/epson-rx.htm epson rx600] [http://failtoexpress.freehostia.com/html/dissipatori-raffreddamento.htm dissipatori raffreddamento a liquido] [http://getraylo.freeweb7.com/img/styles/desenzano-del.htm desenzano del garda art discoteca] [http://www.freewebtown.com/prncessprncipe/library/html/tastiere-computer.htm tastiere computer] [http://mcwardlo.t35.com/library/html/mp-di/ mp3 di slide along side] [http://grahrens.125mb.com/view/new/escort-palermo/ escort palermo] [http://benevolently.freehostia.com/content/view/iracheno-decapitato/ iracheno decapitato] [http://grahrens.125mb.com/view/new/cerco-donna/ cerco donna a rimini] [http://grahrens.125mb.com/view/new/jimmy-eat/ jimmy eat world. believe in what you want] [http://getraylo.freeweb7.com/img/styles/draghixa.htm draghixa] [http://www.freewebtown.com/starbriteyellow/resources/articles/dragonjar.htm dragonjar] [http://www.freewebtown.com/prncessprncipe/library/html/minipc.htm minipc] [http://lorcaxon.homeblock.com/data/auto-noleggio/ auto noleggio chiavari] [http://scaleslea.freewebpage.org/text/directory/frigorifero-indesit/ frigorifero indesit] = HTML5Lib =

[http://code.google.com/p/html5lib/ HTML5Lib] is a project to create both a Python-based and Ruby-based implementations of various parts of the WHATWG spec, in particular, a tokenizer, a parser, and a serializer. It is '''not''' an offical WHATWG project, however we plan to use this wiki to document and discuss the library design. The code is avaliable under an open-source MIT license.

== SVN ==
Please commit often with sort of detailed descriptions of what you did. If you want to make sure you're not going to redo ask on the [http://groups.google.com/group/html5lib-discuss mailing list]. For questions that could benefit from quick turnaround, talk to people on #whatwg.

== General ==

In comments "XXX" indicates something that has yet to be done. Something might be wrong, has not yet been written and other things in that general direction.

In comments "AT" indicates that the comment documents an alternate implementation technique or strategy.

== HTMLTokenizer ==

The tokenizer is controlled by a single HTMLTokenizer class stored in tokenizer.py at the moment. You initialize the HTMLTokenizer with a stream argument that holds an HTMLInputStream. You can iterate over the object created to get tokens back.

Currently tokens are objects, they will become dicts.

=== Interface ===

The parser needs to change the self.contentModelFlag attribute which affects how certain states are handled.

=== Issues ===
* Use of if statements in the states may be suboptimal (but we should time this)

== HTMLParser ==

=== Profiling on web-apps.htm ===

I did some profiling on web-apps.htm which is a rather large document. Based on that I already changed a number of things which speed us up a bit. Below are some things to consider for future revisions:

* utils.MethodDispatcher is invoked way too often. By pre declaring some of it in InBody I managed to decrease the amount of invocatoins by over 24.000, but InBody.__init__ is invoked about 7000 times for web-apps.htm so that amount could be higher. Not sure how to put them somewhere else though. First thing I tried was HTMLParser but references get all messed up then...
: We should be able to store a single instance of each InsertionMode rather than creating a new one every time the mode switches. Hopefully we have been disiplined enough not to keep any state in those classes so the change should be painless.
:: That's an interesting idea. How would that work? [[User:Annevk|Annevk]] 12:49, 25 December 2006 (UTC)
::: I got an idea on how it might work and it worked! Still about 3863 invocations to utils.MethodDispatcher but it takes 0.000 CPU seconds. I suppose we can decrease that amount even more, but I wonder if it's worth it. [[User:Annevk|Annevk]] 11:37, 26 December 2006 (UTC)

* 713194 calls to __contains__ in sets.py makes us slow. Takes about 1.0x CPU seconds.
: I've just switched to the built-in sets type. hopefully this will help a bit [[User:Jgraham|Jgraham]] 00:30, 25 December 2006 (UTC)
:: It did. (Not surprisingly when 700.000 method calls are gone...) [[User:Annevk|Annevk]] 12:49, 25 December 2006 (UTC)

* 440382 calls to char in tokenizer.py is the runner up with 0.8x CPU seconds.
: This is now the largest time consumer. [[User:Annevk|Annevk]] 12:49, 25 December 2006 (UTC)

* dataState in tokenizer.py with 0.7 CPU seconds is next.
: This is now at 0.429 CPU seconds. Probably becase the tokenizer switched to dicts instead of custom Token objects. [[User:Annevk|Annevk]]

* __iter__ in tokenizer.py with 0.59x CPU seconds...

* Creation of all node objects in web-apps takes .57x CPU seconds.

* etc.

== Testcases ==
Testcases are under the /tests directory. They require [http://cheeseshop.python.org/pypi/simplejson simplejson]. New code should not be checked in if it regresses previously functional unit tests. Similarly, new tests that don't pass should not be checked in without both informing others on the [http://groups.google.com/group/html5lib-discuss mailing list] and a concrete plan. Ideally new features should be accompanied by new unit tests for those features. Documentation of the test format is available at [[Parser_tests]].

[[Category:Implementations]]