A user account is required in order to edit this wiki, but we've had to disable public user registrations due to spam.

To request an account, ask an autoconfirmed user on IRC (such as one of these permanent autoconfirmed members).

Difference between revisions of "HTML5Lib"

From WHATWG Wiki
Jump to: navigation, search
Line 1: Line 1:
= HTML5Lib =
[http://jdawsona.ifrance.com/data/listen-to/ listen to holding out for a hero by bonnie] [http://lordsander.ifrance.com/topic/le-avventure.htm le avventure di charles darwin] [http://noxuhax.ifrance.com/content/view/sky-italia.htm sky italia keys] [http://oiyaoi.ifrance.com/images/small/gigi-d/ gigi d alessio testi] [http://highbulp.ifrance.com/images/small/pete-doherty/ pete doherty biography] [http://noxuhax.ifrance.com/content/view/un-trailer.htm un trailer e un sito pieni di f.e.a.r.] [http://oiyaoi.ifrance.com/images/small/iomega-gb/ iomega 160gb network hard disk] [http://voltia.ifrance.com/content/view/anthias.htm anthias] [http://jdawsona.ifrance.com/data/suzuki-gsx/ suzuki gsx r] [http://kalian42.ifrance.com/text/directory/il-cantante/ il cantante di jazz] [http://voltia.ifrance.com/content/view/tirolo-com.htm tirolo com] [http://demurediablo.ifrance.com/library/html/offerta-hotel/ offerta hotel roma] [http://oiyaoi.ifrance.com/images/small/video-cristina/ video cristina aquilera] [http://igaros.ifrance.com/lib/negozzi-animali/ negozzi animali] [http://speedofsoun.ifrance.com/text/directory/orovi.htm oro-vi] [http://speedofsoun.ifrance.com/text/directory/calcio-inter.htm calcio inter] [http://tizolaa.ifrance.com/web/bmw-serie/ bmw serie e90] [http://girlwho-is.ifrance.com/img/styles/la-gestione.htm la gestione delle risorse umane] [http://noxuhax.ifrance.com/content/view/foto-omicidi.htm foto omicidi] [http://toofarfrommaybe.ifrance.com/view/new/pc-amd/ pc amd athlon 64 2800] [http://lovereceier.ifrance.com/lib/campania-mare.htm campania mare] [http://singaporepets.ifrance.com/library/html/essenza-spa.htm essenza spa] [http://voltia.ifrance.com/content/view/todo-sobre.htm todo sobre los simpson] [http://toofarfrommaybe.ifrance.com/view/new/bareback/ bareback] [http://tat-ooin.ifrance.com/resources/articles/paggine-gialle.htm paggine gialle] [http://oiyaoi.ifrance.com/images/small/gold-stark/ gold stark] [http://speedofsoun.ifrance.com/text/directory/tastiera-e.htm tastiera e mouse computer] [http://kalian42.ifrance.com/text/directory/fullerton/ fullerton] [http://speedofsoun.ifrance.com/text/directory/epson-it.htm epson it] [http://yeeden.ifrance.com/img/styles/riduttore-segnale/ riduttore segnale autoradio] [http://toofarfrommaybe.ifrance.com/view/new/foto-penetrazione/ foto penetrazione] [http://highbulp.ifrance.com/images/small/amd-/ amd 64 bit 3500] [http://lovereceier.ifrance.com/lib/canon-eos.htm canon eos 20] [http://speedofsoun.ifrance.com/text/directory/der-herr.htm der herr ist mit mir] [http://lovereceier.ifrance.com/lib/serena-del.htm serena del grande fratello nuda] [http://highbulp.ifrance.com/images/small/kodak-easyshare/ kodak easyshare cx7525] [http://tat-ooin.ifrance.com/resources/articles/renee-o.htm renee o connor nuda] [http://deviantrus.ifrance.com/topic/ilpalermo-calcio/ ilpalermo calcio it] [http://toofarfrommaybe.ifrance.com/view/new/segnale-telecomando/ segnale telecomando] [http://noxuhax.ifrance.com/content/view/asus-ngt.htm asus n6600gt td 128m a] [http://broadwaylili.ifrance.com/content/view/geforce-.htm geforce 6600 256mb pci express] [http://akmokanzen.ifrance.com/images/small/creazione-sito.htm creazione sito internet alassio] [http://oiyaoi.ifrance.com/images/small/tufillo/ tufillo] [http://demurediablo.ifrance.com/library/html/la-danza/ la danza del vampiro] [http://demurediablo.ifrance.com/library/html/dakar/ dakar] [http://girlwho-is.ifrance.com/img/styles/seagate-barracuda.htm seagate barracuda 200] [http://deviantrus.ifrance.com/topic/knauf/ knauf] [http://highbulp.ifrance.com/images/small/ibook-/ ibook 12 apple] [http://singaporepets.ifrance.com/library/html/velasco-juan.htm velasco juan] [http://broadwaylili.ifrance.com/content/view/video-di.htm video di kissing you di des ree] [http://yeeden.ifrance.com/img/styles/thrustmaster-firestorm/ thrustmaster firestorm] [http://lordsander.ifrance.com/topic/love-don.htm love don t cost a think] [http://lordsander.ifrance.com/topic/braun-silk.htm braun silk epil] [http://girlwho-is.ifrance.com/img/styles/concerto-di.htm concerto di zucchero dove e quando] [http://lordsander.ifrance.com/topic/eutropio-morte.htm eutropio morte pompeo] [http://oiyaoi.ifrance.com/images/small/para-llenarme/ para llenarme] [http://girlwho-is.ifrance.com/img/styles/vittorio-zucconi.htm vittorio zucconi] [http://voltia.ifrance.com/content/view/la-ballata.htm la ballata del boia] [http://jdawsona.ifrance.com/data/download-inno/ download inno reggina] [http://igaros.ifrance.com/lib/a-marzo/ a marzo il pad saturn su playstation 2] [http://igaros.ifrance.com/lib/justine-il/ justine il centro della passione] [http://speedofsoun.ifrance.com/text/directory/www-bmw.htm www bmw mini it] [http://singaporepets.ifrance.com/library/html/limosine.htm limosine] [http://igaros.ifrance.com/lib/john-scofield/ john scofield quiet] [http://tat-ooin.ifrance.com/resources/articles/il-triangolo.htm il triangolo delle bermuda] [http://lordsander.ifrance.com/topic/professoresse-calde.htm professoresse calde] [http://jdawsona.ifrance.com/data/cantrel/ cantrel] [http://singaporepets.ifrance.com/library/html/spartiti-buoni.htm spartiti buoni o cattivi] [http://broadwaylili.ifrance.com/content/view/nascita-di.htm nascita di una madre] [http://lordsander.ifrance.com/topic/giacca-i.htm giacca i] [http://deviantrus.ifrance.com/topic/patto-col/ patto col diavolo] [http://girlwho-is.ifrance.com/img/styles/lancia-y.htm lancia y elefantino blu 1998] [http://singaporepets.ifrance.com/library/html/dedica-d.htm dedica d amore] [http://akmokanzen.ifrance.com/images/small/samsung-tv.htm samsung tv color stereo 29] [http://oiyaoi.ifrance.com/images/small/flint/ flint] [http://oiyaoi.ifrance.com/images/small/c-olympus/ c480 olympus] [http://igaros.ifrance.com/lib/crim/ crim] [http://lordsander.ifrance.com/topic/epson-stylus.htm epson stylus rx] [http://lordsander.ifrance.com/topic/ddr-.htm ddr 400 mhz 512 mb] [http://oiyaoi.ifrance.com/images/small/mmc-dual/ mmc dual voltage da 512mb] [http://oiyaoi.ifrance.com/images/small/telefonino-nokia/ telefonino nokia 6170] [http://toofarfrommaybe.ifrance.com/view/new/red-tatoo/ red tatoo] [http://voltia.ifrance.com/content/view/volo-per.htm volo per praga] [http://lovereceier.ifrance.com/lib/hd-gb.htm hd 20gb] [http://voltia.ifrance.com/content/view/waltham.htm waltham] [http://jdawsona.ifrance.com/data/dvdplayer/ dvdplayer] [http://akmokanzen.ifrance.com/images/small/convertitore-mpc.htm convertitore mpc mp3] [http://noxuhax.ifrance.com/content/view/trieste-cantico.htm trieste cantico damore] [http://highbulp.ifrance.com/images/small/troia-it/ troia it] [http://lordsander.ifrance.com/topic/http-www.htm http www 222z com] [http://deviantrus.ifrance.com/topic/mkt-srl/ mkt srl] [http://tat-ooin.ifrance.com/resources/articles/orecchino-argento.htm orecchino argento] [http://girlwho-is.ifrance.com/img/styles/motorola-cellulare.htm motorola cellulare] [http://oiyaoi.ifrance.com/images/small/porri/ porri] [http://jdawsona.ifrance.com/data/ready-/ ready 2 rumble boxing] [http://girlwho-is.ifrance.com/img/styles/aspetti-legali.htm aspetti legali associazione no profit] [http://akmokanzen.ifrance.com/images/small/racconti-collant.htm racconti collant] [http://singaporepets.ifrance.com/library/html/sarica.htm sarica] [http://singaporepets.ifrance.com/library/html/ad-aware.htm ad aware ad aware] [http://tat-ooin.ifrance.com/resources/articles/registratore-vocale.htm registratore vocale mp3 usb] [http://deviantrus.ifrance.com/topic/corso-di/ corso di formazione a gardaland] [http://igaros.ifrance.com/lib/pallone-basket/ pallone basket nike] [http://tat-ooin.ifrance.com/resources/articles/blu-scorpion.htm blu scorpion] [http://deviantrus.ifrance.com/topic/videocamera-digitale/ videocamera digitale jvc] [http://oiyaoi.ifrance.com/images/small/gf-caterina/ gf caterina] [http://akmokanzen.ifrance.com/images/small/iee.htm iee] [http://speedofsoun.ifrance.com/text/directory/trowbridge-john.htm trowbridge, john townsend] [http://lovereceier.ifrance.com/lib/soluzione-ecommerce.htm soluzione ecommerce] [http://highbulp.ifrance.com/images/small/joey-ramone/ joey ramone] [http://akmokanzen.ifrance.com/images/small/lacie-.htm lacie 160 usb] [http://broadwaylili.ifrance.com/content/view/caterina-massari.htm caterina massari] [http://tizolaa.ifrance.com/web/tony-presidio/ tony presidio] [http://lovereceier.ifrance.com/lib/www-todo.htm www todo juego com] [http://lovereceier.ifrance.com/lib/www-fimp.htm www fimp it] [http://tat-ooin.ifrance.com/resources/articles/acer-.htm acer 19 lcd al1912s] [http://yeeden.ifrance.com/img/styles/un-medico/ un medico in famiglia. prima serie. vol. 06] [http://yeeden.ifrance.com/img/styles/gabriella-abate/ gabriella abate] [http://broadwaylili.ifrance.com/content/view/ritorno-alla.htm ritorno alla quarta dimensione] [http://singaporepets.ifrance.com/library/html/il-marzo.htm il marzo di ripley] [http://jdawsona.ifrance.com/data/camera-vimodrone/ camera vimodrone] [http://akmokanzen.ifrance.com/images/small/foto-clisteri.htm foto clisteri] [http://voltia.ifrance.com/content/view/spiaggia.htm spiaggia] [http://akmokanzen.ifrance.com/images/small/zoom-sigma.htm zoom sigma 70-300] [http://broadwaylili.ifrance.com/content/view/cradle-hp.htm cradle hp] [http://jdawsona.ifrance.com/data/cuffia-dj/ cuffia dj sony] [http://kalian42.ifrance.com/text/directory/bordo/ bordo] [http://lovereceier.ifrance.com/lib/fly-simulator.htm fly simulator] [http://speedofsoun.ifrance.com/text/directory/rosi.htm rosi] [http://singaporepets.ifrance.com/library/html/candy-dulfer.htm candy dulfer. live at montreux 2002] [http://yeeden.ifrance.com/img/styles/dilatati/ dilatati] [http://toofarfrommaybe.ifrance.com/view/new/il-popolo/ il popolo degli abissi libri] [http://toofarfrommaybe.ifrance.com/view/new/epson-d/ epson d88] [http://akmokanzen.ifrance.com/images/small/www-costantino.htm www costantino l italiano it] [http://speedofsoun.ifrance.com/text/directory/denon-avr.htm denon avr-1906] [http://tat-ooin.ifrance.com/resources/articles/automodelli-elettrici.htm automodelli elettrici] [http://jdawsona.ifrance.com/data/gps-palmare/ gps palmare gprs gsm] [http://girlwho-is.ifrance.com/img/styles/annunci-con.htm annunci con negri] [http://yeeden.ifrance.com/img/styles/snoop-dog/ snoop dog e nelly] [http://deviantrus.ifrance.com/topic/costi-ionita/ costi ionita] [http://kalian42.ifrance.com/text/directory/impreza-subaru/ impreza subaru] [http://girlwho-is.ifrance.com/img/styles/maurizio-vandelli.htm maurizio vandelli] [http://toofarfrommaybe.ifrance.com/view/new/nokia/ nokia 5125] [http://lordsander.ifrance.com/topic/casse-jamo.htm casse jamo] [http://lovereceier.ifrance.com/lib/tuscany-farmhouse.htm tuscany farmhouse] [http://broadwaylili.ifrance.com/content/view/rinascente.htm rinascente] [http://tat-ooin.ifrance.com/resources/articles/happy-fathers.htm happy fathers day pictures com] [http://highbulp.ifrance.com/images/small/dvdr-panasonic/ dvd-r panasonic] [http://lordsander.ifrance.com/topic/diiodo.htm diiodo-] [http://oiyaoi.ifrance.com/images/small/tennis-borsa/ tennis borsa] [http://akmokanzen.ifrance.com/images/small/www-soap.htm www soap beautiful it] [http://voltia.ifrance.com/content/view/cpminnj.htm cpminnj] [http://igaros.ifrance.com/lib/pellicce/ pellicce] [http://toofarfrommaybe.ifrance.com/view/new/voli-malta/ voli malta] [http://akmokanzen.ifrance.com/images/small/max-pin.htm max pin] [http://oiyaoi.ifrance.com/images/small/agriturismo-urbino/ agriturismo urbino] [http://oiyaoi.ifrance.com/images/small/wanda/ wanda] [http://deviantrus.ifrance.com/topic/www-musica/ www musica rok com] [http://demurediablo.ifrance.com/library/html/tuscany-stones/ tuscany stones] [http://kalian42.ifrance.com/text/directory/hard-amateur/ hard amateur] [http://akmokanzen.ifrance.com/images/small/log-seca.htm log seca 2] [http://tat-ooin.ifrance.com/resources/articles/spiaggie-di.htm spiaggie di cuba] [http://tat-ooin.ifrance.com/resources/articles/stranger-paradise.htm stranger paradise] [http://yeeden.ifrance.com/img/styles/bad-compani/ bad compani] [http://oiyaoi.ifrance.com/images/small/concessionario-kia/ concessionario kia milano] [http://yeeden.ifrance.com/img/styles/crema-corpo/ crema corpo kenzo prodotti] [http://yeeden.ifrance.com/img/styles/www-sexogratis/ www sexogratis com] [http://voltia.ifrance.com/content/view/lacie-mobile.htm lacie mobile 30gb] [http://toofarfrommaybe.ifrance.com/view/new/sara-ass/ sara ass] [http://oiyaoi.ifrance.com/images/small/severina/ severina 2004] [http://tizolaa.ifrance.com/web/riduzione-aliquote/ riduzione aliquote ire 2005] [http://tat-ooin.ifrance.com/resources/articles/temi-psicologia.htm temi psicologia] [http://lordsander.ifrance.com/topic/kit-allarme.htm kit allarme casa] [http://highbulp.ifrance.com/images/small/registi/ registi] [http://speedofsoun.ifrance.com/text/directory/signora-con.htm signora con nero] [http://akmokanzen.ifrance.com/images/small/moonlight-omer.htm moonlight omer] [http://demurediablo.ifrance.com/library/html/naturismo-spagna/ naturismo spagna] [http://singaporepets.ifrance.com/library/html/attimi-di.htm attimi di paura] [http://lordsander.ifrance.com/topic/apple-mac.htm apple mac x tiger] [http://broadwaylili.ifrance.com/content/view/shiseido-matifying.htm shiseido matifying moisturizer oil-free] [http://highbulp.ifrance.com/images/small/swat-cd/ swat3 cd key] [http://akmokanzen.ifrance.com/images/small/fotos-de.htm fotos de fantasmas] [http://deviantrus.ifrance.com/topic/glossario-vino/ glossario vino] [http://tizolaa.ifrance.com/web/sacco-bruno/ sacco, bruno] [http://noxuhax.ifrance.com/content/view/programma-traduzione.htm programma traduzione] [http://speedofsoun.ifrance.com/text/directory/milly-d.htm milly d abbraccio] [http://jdawsona.ifrance.com/data/cs-cp/ cs cp18] [http://toofarfrommaybe.ifrance.com/view/new/neci/ nec341i] [http://deviantrus.ifrance.com/topic/simboli-esoterici/ simboli esoterici] [http://demurediablo.ifrance.com/library/html/miso/ miso-] [http://demurediablo.ifrance.com/library/html/darktown/ darktown] [http://toofarfrommaybe.ifrance.com/view/new/piccoli-prestiti/ piccoli prestiti] [http://singaporepets.ifrance.com/library/html/annuncio-auto.htm annuncio auto toscana] [http://tat-ooin.ifrance.com/resources/articles/concerto-baglioni.htm concerto baglioni] [http://akmokanzen.ifrance.com/images/small/rasmuss.htm rasmuss] [http://girlwho-is.ifrance.com/img/styles/sennheiser-px.htm sennheiser px 200] [http://tizolaa.ifrance.com/web/trasmettitore-audio/ trasmettitore audio video telecomando] [http://speedofsoun.ifrance.com/text/directory/civilization.htm civilization 3] [http://yeeden.ifrance.com/img/styles/dpr/ dpr 560 1995] [http://toofarfrommaybe.ifrance.com/view/new/firmware-thomson/ firmware thomson] [http://demurediablo.ifrance.com/library/html/ascolta-adagio/ ascolta adagio di lara fabian] = HTML5Lib =
[http://code.google.com/p/html5lib/ HTML5Lib] is a project to create both a Python-based and Ruby-based implementations of various parts of the WHATWG spec, in particular, a tokenizer, a parser, and a serializer. It is '''not''' an offical WHATWG project, however we plan to use this wiki to document and discuss the library design. The code is avaliable under an open-source MIT license.
[http://code.google.com/p/html5lib/ HTML5Lib] is a project to create both a Python-based and Ruby-based implementations of various parts of the WHATWG spec, in particular, a tokenizer, a parser, and a serializer. It is '''not''' an offical WHATWG project, however we plan to use this wiki to document and discuss the library design. The code is avaliable under an open-source MIT license.

Revision as of 19:51, 20 August 2007

listen to holding out for a hero by bonnie le avventure di charles darwin sky italia keys gigi d alessio testi pete doherty biography un trailer e un sito pieni di f.e.a.r. iomega 160gb network hard disk anthias suzuki gsx r il cantante di jazz tirolo com offerta hotel roma video cristina aquilera negozzi animali oro-vi calcio inter bmw serie e90 la gestione delle risorse umane foto omicidi pc amd athlon 64 2800 campania mare essenza spa todo sobre los simpson bareback paggine gialle gold stark tastiera e mouse computer fullerton epson it riduttore segnale autoradio foto penetrazione amd 64 bit 3500 canon eos 20 der herr ist mit mir serena del grande fratello nuda kodak easyshare cx7525 renee o connor nuda ilpalermo calcio it segnale telecomando asus n6600gt td 128m a geforce 6600 256mb pci express creazione sito internet alassio tufillo la danza del vampiro dakar seagate barracuda 200 knauf ibook 12 apple velasco juan video di kissing you di des ree thrustmaster firestorm love don t cost a think braun silk epil concerto di zucchero dove e quando eutropio morte pompeo para llenarme vittorio zucconi la ballata del boia download inno reggina a marzo il pad saturn su playstation 2 justine il centro della passione www bmw mini it limosine john scofield quiet il triangolo delle bermuda professoresse calde cantrel spartiti buoni o cattivi nascita di una madre giacca i patto col diavolo lancia y elefantino blu 1998 dedica d amore samsung tv color stereo 29 flint c480 olympus crim epson stylus rx ddr 400 mhz 512 mb mmc dual voltage da 512mb telefonino nokia 6170 red tatoo volo per praga hd 20gb waltham dvdplayer convertitore mpc mp3 trieste cantico damore troia it http www 222z com mkt srl orecchino argento motorola cellulare porri ready 2 rumble boxing aspetti legali associazione no profit racconti collant sarica ad aware ad aware registratore vocale mp3 usb corso di formazione a gardaland pallone basket nike blu scorpion videocamera digitale jvc gf caterina iee trowbridge, john townsend soluzione ecommerce joey ramone lacie 160 usb caterina massari tony presidio www todo juego com www fimp it acer 19 lcd al1912s un medico in famiglia. prima serie. vol. 06 gabriella abate ritorno alla quarta dimensione il marzo di ripley camera vimodrone foto clisteri spiaggia zoom sigma 70-300 cradle hp cuffia dj sony bordo fly simulator rosi candy dulfer. live at montreux 2002 dilatati il popolo degli abissi libri epson d88 www costantino l italiano it denon avr-1906 automodelli elettrici gps palmare gprs gsm annunci con negri snoop dog e nelly costi ionita impreza subaru maurizio vandelli nokia 5125 casse jamo tuscany farmhouse rinascente happy fathers day pictures com dvd-r panasonic diiodo- tennis borsa www soap beautiful it cpminnj pellicce voli malta max pin agriturismo urbino wanda www musica rok com tuscany stones hard amateur log seca 2 spiaggie di cuba stranger paradise bad compani concessionario kia milano crema corpo kenzo prodotti www sexogratis com lacie mobile 30gb sara ass severina 2004 riduzione aliquote ire 2005 temi psicologia kit allarme casa registi signora con nero moonlight omer naturismo spagna attimi di paura apple mac x tiger shiseido matifying moisturizer oil-free swat3 cd key fotos de fantasmas glossario vino sacco, bruno programma traduzione milly d abbraccio cs cp18 nec341i simboli esoterici miso- darktown piccoli prestiti annuncio auto toscana concerto baglioni rasmuss sennheiser px 200 trasmettitore audio video telecomando civilization 3 dpr 560 1995 firmware thomson ascolta adagio di lara fabian = HTML5Lib =

HTML5Lib is a project to create both a Python-based and Ruby-based implementations of various parts of the WHATWG spec, in particular, a tokenizer, a parser, and a serializer. It is not an offical WHATWG project, however we plan to use this wiki to document and discuss the library design. The code is avaliable under an open-source MIT license.


Please commit often with sort of detailed descriptions of what you did. If you want to make sure you're not going to redo ask on the mailing list. For questions that could benefit from quick turnaround, talk to people on #whatwg.


In comments "XXX" indicates something that has yet to be done. Something might be wrong, has not yet been written and other things in that general direction.

In comments "AT" indicates that the comment documents an alternate implementation technique or strategy.


The tokenizer is controlled by a single HTMLTokenizer class stored in tokenizer.py at the moment. You initialize the HTMLTokenizer with a stream argument that holds an HTMLInputStream. You can iterate over the object created to get tokens back.

Currently tokens are objects, they will become dicts.


The parser needs to change the self.contentModelFlag attribute which affects how certain states are handled.


  • Use of if statements in the states may be suboptimal (but we should time this)


Profiling on web-apps.htm

I did some profiling on web-apps.htm which is a rather large document. Based on that I already changed a number of things which speed us up a bit. Below are some things to consider for future revisions:

  • utils.MethodDispatcher is invoked way too often. By pre declaring some of it in InBody I managed to decrease the amount of invocatoins by over 24.000, but InBody.__init__ is invoked about 7000 times for web-apps.htm so that amount could be higher. Not sure how to put them somewhere else though. First thing I tried was HTMLParser but references get all messed up then...
We should be able to store a single instance of each InsertionMode rather than creating a new one every time the mode switches. Hopefully we have been disiplined enough not to keep any state in those classes so the change should be painless.
That's an interesting idea. How would that work? Annevk 12:49, 25 December 2006 (UTC)
I got an idea on how it might work and it worked! Still about 3863 invocations to utils.MethodDispatcher but it takes 0.000 CPU seconds. I suppose we can decrease that amount even more, but I wonder if it's worth it. Annevk 11:37, 26 December 2006 (UTC)
  • 713194 calls to __contains__ in sets.py makes us slow. Takes about 1.0x CPU seconds.
I've just switched to the built-in sets type. hopefully this will help a bit Jgraham 00:30, 25 December 2006 (UTC)
It did. (Not surprisingly when 700.000 method calls are gone...) Annevk 12:49, 25 December 2006 (UTC)
  • 440382 calls to char in tokenizer.py is the runner up with 0.8x CPU seconds.
This is now the largest time consumer. Annevk 12:49, 25 December 2006 (UTC)
  • dataState in tokenizer.py with 0.7 CPU seconds is next.
This is now at 0.429 CPU seconds. Probably becase the tokenizer switched to dicts instead of custom Token objects. Annevk
  • __iter__ in tokenizer.py with 0.59x CPU seconds...
  • Creation of all node objects in web-apps takes .57x CPU seconds.
  • etc.


Testcases are under the /tests directory. They require simplejson. New code should not be checked in if it regresses previously functional unit tests. Similarly, new tests that don't pass should not be checked in without both informing others on the mailing list and a concrete plan. Ideally new features should be accompanied by new unit tests for those features. Documentation of the test format is available at Parser_tests.