Testsuite

Requirements

• Each test needs a "reviewed" marker of some sort
• It must be easy to find tests where the spec has changed under them
• The barrier to contribution must be as low as possible
• Testcases should have somewhat stable URIs
• If test can be done using JavaScript preferably require it to be in JavaScript so engines can be more efficiently tested (i.e. automated).
• It must be easy to review tests
• Standardize a test format?

Non-requirements

• There does not need to be a single consistent test harness for the whole of HTML5. (When sections can be tested in isolation, each section should use a test harness that is suited to that section's testing requirements. E.g. there is little value in fitting canvas tests and parser tests into the same framework, and it may add a lot of complexity.)

Format proposal

See http://omocha.w3.org/wiki/newformat for a format proposal that should meet most of these requirements. That format is based on the Mozilla Mochitest format for running JavaScript based client-side tests which can be run automatically.

Existing tests

• Test cases
• The specification has links to test in the status boxes.
• IE's tests
• Opera's tests
• (Some of?) WebKit's tests
• Some of Mozilla's mochitests
  • Testsuite/Mozilla
• Some of Mozilla's reftests
• Philip's canvas tests
• Hixie's ad-hoc tests
• Lachlan's tests
• gsnedders' number parsing tests
• zcorpan's tests
• Mathias’s tests
• html5lib tests
• hsivonen's video tests
• Document Object Model (DOM) Conformance Test Suites
• Window test suite

W3C

• Geolocation
• WebApps WG
• HTML WG
• CSS WG