Validator.nu Full-Stack Tests - Revision history

MikeSmith at 14:07, 17 March 2015

2015-03-17T14:07:53Z

← Older revision		Revision as of 14:07, 17 March 2015
Line 1:		Line 1:
			{{Obsolete}}
	Validator.nu has a framework for doing full-stack HTML5 validator testing in an implementation-independent manner. Currently, the framework is lacking tests.		Validator.nu has a framework for doing full-stack HTML5 validator testing in an implementation-independent manner. Currently, the framework is lacking tests.

Hsivonen: s/http/https/; s/hsivonen.iki.fi/hsivonen.com/

2014-11-21T08:37:02Z

s/http/https/; s/hsivonen.iki.fi/hsivonen.com/

← Older revision		Revision as of 08:37, 21 November 2014
Line 5:		Line 5:
	==The Front End==		==The Front End==

	The front end for the system is the script named <code>validator-tester.py</code> in the <code>test-harness/</code> directory of the <code>validator</code> SVN module. The script requires a recent version of [~~http~~://pypi.python.org/pypi/simplejson simplejson].		The front end for the system is the script named <code>validator-tester.py</code> in the <code>test-harness/</code> directory of the <code>validator</code> SVN module. The script requires a recent version of [https://pypi.python.org/pypi/simplejson simplejson].

	The script is documented on [[Validator.nu validator-tester.py\|a separate page]].		The script is documented on [[Validator.nu validator-tester.py\|a separate page]].
Line 52:		Line 52:
	Next, the test file needs to be put on an HTTP server. If you are not testing the internal encoding declaration, the absence of an encoding declaration or a particular encoding, you should serve the file with the header <code>Content-Type: text/html; charset=utf-8</code> (or <code>Content-Type: application/xhtml+xml</code> for XHTML5 tests).		Next, the test file needs to be put on an HTTP server. If you are not testing the internal encoding declaration, the absence of an encoding declaration or a particular encoding, you should serve the file with the header <code>Content-Type: text/html; charset=utf-8</code> (or <code>Content-Type: application/xhtml+xml</code> for XHTML5 tests).

	In this case, the sample test has the URI [http://hsivonen.~~iki.fi~~/test/moz/unescaped-ampersand.html <code><nowiki>http://hsivonen.~~iki.fi~~/test/moz/unescaped-ampersand.html</nowiki></code>].		In this case, the sample test has the URI [http://hsivonen.com/test/moz/unescaped-ampersand.html <code><nowiki>http://hsivonen.com/test/moz/unescaped-ampersand.html</nowiki></code>].

	===Generating a Reference Result===		===Generating a Reference Result===
Line 58:		Line 58:
	If this is a regression test (i.e. a check for the error has already been implemented in Validator.nu), the reference result can be generated using Validator.nu.		If this is a regression test (i.e. a check for the error has already been implemented in Validator.nu), the reference result can be generated using Validator.nu.

	First, use the [~~http~~://html5.validator.nu/?doc=http%3A%2F%2Fhsivonen.~~iki.fi~~%2Ftest%2Fmoz%2Funescaped-ampersand.html HTML UI to check] that the right error is given and that the right piece of source is highlighted.		First, use the [https://html5.validator.nu/?doc=http%3A%2F%2Fhsivonen.com%2Ftest%2Fmoz%2Funescaped-ampersand.html HTML UI to check] that the right error is given and that the right piece of source is highlighted.

	Then, use <code>validator-tester.py</code> to dump the result in a JSON file:		Then, use <code>validator-tester.py</code> to dump the result in a JSON file:
	<pre>python validator-tester.py dumpuri http://hsivonen.~~iki.fi~~/test/moz/unescaped-ampersand.html temp.json</pre>		<pre>python validator-tester.py dumpuri http://hsivonen.com/test/moz/unescaped-ampersand.html temp.json</pre>

	You can then open the temporary file in your favorite text editor:		You can then open the temporary file in your favorite text editor:
Line 68:		Line 68:
	You should see contents like this:		You should see contents like this:
	<pre>{		<pre>{
	"http://hsivonen.~~iki.fi~~/test/moz/unescaped-ampersand.html": [		"http://hsivonen.com/test/moz/unescaped-ampersand.html": [
	{		{
	"firstColumn": 7,		"firstColumn": 7,
Line 89:		Line 89:
	Edit the range to start 3 column earlier and end 5 columns later:		Edit the range to start 3 column earlier and end 5 columns later:
	<pre>{		<pre>{
	"http://hsivonen.~~iki.fi~~/test/moz/unescaped-ampersand.html": [		"http://hsivonen.com/test/moz/unescaped-ampersand.html": [
	{		{
	"firstColumn": 4,		"firstColumn": 4,
Line 106:		Line 106:

	When the autogenerated reference does not need editing, you can use this shortcut:		When the autogenerated reference does not need editing, you can use this shortcut:
	<pre>python validator-tester.py adduri http://hsivonen.~~iki.fi~~/test/moz/en-UK.html</pre>		<pre>python validator-tester.py adduri http://hsivonen.com/test/moz/en-UK.html</pre>

	==Writing a Test Eliciting No Errors==		==Writing a Test Eliciting No Errors==
Line 115:		Line 115:

	You can run the test created above with this command:		You can run the test created above with this command:
	<pre>python validator-tester.py checkuri http://hsivonen.~~iki.fi~~/test/moz/unescaped-ampersand.html</pre>		<pre>python validator-tester.py checkuri http://hsivonen.com/test/moz/unescaped-ampersand.html</pre>

	There is no output if the test passes.		There is no output if the test passes.

Zcorpan at 15:42, 11 September 2008

2008-09-11T15:42:23Z

← Older revision		Revision as of 15:42, 11 September 2008
Line 41:		Line 41:
	<body>		<body>
	<p>&amgueao</p>		<p>&amgueao</p>
			<p>There should be an error.</p>
	</body>		</body>
	</html></pre>		</html></pre>

	Note that a test that isn’t testing the doctype or tag inference should have the doctype and explicit tags for <code>html</code>, <code>head</code>, <code>title</code> and <code>body</code> in order to avoid accidentally testing something related to tag inference. It is a good idea to make the content of		Note that a test that isn’t testing the doctype or tag inference should have the doctype and explicit tags for <code>html</code>, <code>head</code>, <code>title</code> and <code>body</code> in order to avoid accidentally testing something related to tag inference. It is a good idea to make the content of
	<code>title</code> hint at what is being tested.		<code>title</code> hint at what is being tested and include a paragraph saying whether an error is expected or not.

	===Placing the Test Case on a Server===		===Placing the Test Case on a Server===

Hsivonen: /* The Front End */ specify which SVN module this stuff is in

2008-09-09T14:40:25Z

The Front End: specify which SVN module this stuff is in

← Older revision		Revision as of 14:40, 9 September 2008
Line 5:		Line 5:
	==The Front End==		==The Front End==

	The front end for the system is the script named <code>validator-tester.py</code> in the <code>test-harness/</code> directory. The script requires a recent version of [http://pypi.python.org/pypi/simplejson simplejson].		The front end for the system is the script named <code>validator-tester.py</code> in the <code>test-harness/</code> directory of the <code>validator</code> SVN module. The script requires a recent version of [http://pypi.python.org/pypi/simplejson simplejson].

	The script is documented on [[Validator.nu validator-tester.py\|a separate page]].		The script is documented on [[Validator.nu validator-tester.py\|a separate page]].

Hsivonen: add category

2008-09-09T14:36:16Z

add category

← Older revision		Revision as of 14:36, 9 September 2008
Line 122:		Line 122:
	To run all tests that have reference results in the database, run:		To run all tests that have reference results in the database, run:
	<pre>python validator-tester.py checkall</pre>		<pre>python validator-tester.py checkall</pre>

			[[Category:Validator.nu Documentation]]

Hsivonen: /* The Front End */

2008-09-09T14:33:58Z

The Front End

← Older revision		Revision as of 14:33, 9 September 2008
Line 5:		Line 5:
	==The Front End==		==The Front End==

	The front end for the system is the script named <code>validator-tester.py</code> in the <code>test-harness/</code> directory.		The front end for the system is the script named <code>validator-tester.py</code> in the <code>test-harness/</code> directory. The script requires a recent version of [http://pypi.python.org/pypi/simplejson simplejson].

	The script is documented on [[Validator.nu validator-tester.py\|a separate page]].		The script is documented on [[Validator.nu validator-tester.py\|a separate page]].

Hsivonen: /* Writing a Test Eliciting No Errors */

2008-09-09T14:32:52Z

Writing a Test Eliciting No Errors

← Older revision		Revision as of 14:32, 9 September 2008
Line 110:		Line 110:

	When writing a test case that is expected to pass without errors, you need to write it and put in on the server as above. However, since there is nothing to edit in the reference results, you can always use the <code>adduri</code> shortcut command mentioned above.		When writing a test case that is expected to pass without errors, you need to write it and put in on the server as above. However, since there is nothing to edit in the reference results, you can always use the <code>adduri</code> shortcut command mentioned above.

			==Running a Test==

			You can run the test created above with this command:
			<pre>python validator-tester.py checkuri http://hsivonen.iki.fi/test/moz/unescaped-ampersand.html</pre>

			There is no output if the test passes.

			==Running All Tests==

			To run all tests that have reference results in the database, run:
			<pre>python validator-tester.py checkall</pre>

Hsivonen: /* Writing a Test Eliciting No Errors */

2008-09-09T14:29:58Z

Writing a Test Eliciting No Errors

← Older revision		Revision as of 14:29, 9 September 2008
Line 108:		Line 108:

	==Writing a Test Eliciting No Errors==		==Writing a Test Eliciting No Errors==

			When writing a test case that is expected to pass without errors, you need to write it and put in on the server as above. However, since there is nothing to edit in the reference results, you can always use the <code>adduri</code> shortcut command mentioned above.

Hsivonen at 13:54, 9 September 2008

2008-09-09T13:54:24Z

← Older revision		Revision as of 13:54, 9 September 2008
Line 106:		Line 106:
	When the autogenerated reference does not need editing, you can use this shortcut:		When the autogenerated reference does not need editing, you can use this shortcut:
	<pre>python validator-tester.py adduri http://hsivonen.iki.fi/test/moz/en-UK.html</pre>		<pre>python validator-tester.py adduri http://hsivonen.iki.fi/test/moz/en-UK.html</pre>

			==Writing a Test Eliciting No Errors==

Hsivonen: /* Generating a Reference Result */

2008-09-09T13:51:48Z

Generating a Reference Result

← Older revision		Revision as of 13:51, 9 September 2008
Line 103:		Line 103:
	Now you can save the file and commit the modified file into the reference database:		Now you can save the file and commit the modified file into the reference database:
	<pre>python validator-tester.py mergedb temp.json</pre>		<pre>python validator-tester.py mergedb temp.json</pre>

			When the autogenerated reference does not need editing, you can use this shortcut:
			<pre>python validator-tester.py adduri http://hsivonen.iki.fi/test/moz/en-UK.html</pre>