Why not conneg - Revision history

Mnot at 06:31, 11 June 2015

2015-06-11T06:31:49Z

← Older revision		Revision as of 06:31, 11 June 2015
Line 4:		Line 4:

	This page explains what’s wrong with each of the existing negotiation axes and then uses the HTML <code>video</code> element as an example for illustrating why HTTP-based negotiation in general is worse than letting the browser choose from alternative URLs by showing how an HTTP-based codec negotiation solution would be worse than the actual browser-side codec negotiation solution that <code>video</code> element uses.		This page explains what’s wrong with each of the existing negotiation axes and then uses the HTML <code>video</code> element as an example for illustrating why HTTP-based negotiation in general is worse than letting the browser choose from alternative URLs by showing how an HTTP-based codec negotiation solution would be worse than the actual browser-side codec negotiation solution that <code>video</code> element uses.

			See also the [http://httpwg.github.io/specs/rfc7231.html#content.negotiation HTTP specification], which itself lists a number of disadvantages to content negotiation (the type discussed here is called "proactive content negotiation" there).

	==Negotiation by compression==		==Negotiation by compression==

GPHemsley: Reverted edits by Annevk (talk) to last revision by Hsivonen

2014-10-10T10:44:26Z

Reverted edits by Annevk (talk) to last revision by Hsivonen

← Older revision		Revision as of 10:44, 10 October 2014
Line 1:		Line 1:
	The purpose of this page is to explain what's wrong with HTTP content negotiation and why you should not suggest HTTP content negotiation as a solution to a problem.		The purpose of this page is to explain what's wrong with HTTP content negotiation and why you should not suggest HTTP content negotiation as a solution to a problem.

	HTTP content negotiation has four axes: negotiation by format (<code>Accept</code>), negotiation by character encoding <code>Accept-Charset</code>, negotiation by natural language (<code>Accept-Language</code>) and negotiation by compression (<code>Accept-Encoding</code>). These axes need to be discussed separately. The only one of these that shouldn't be classified as a failure is negotiation by compression, but even it shouldn't be taken as a role model considering its verbosity for basically negotiating one bit of information.		HTTP content negotiation has four axes: negotiation by format (<code>Accept</code>), negotiation by character encoding <code>Accept-Charset</code>, negotiation by natural language (<code>Accept-Language</code>) and negotiation by compression (<code>Accept-Encoding</code>). These axes need to be discussed separately. The only one of these that shouldn't be classified as a failure is negotiation by compression, but even it shouldn't be taken as a role model considering its verbosity for basically negotiating one bit of information.

Annevk: make meaningless edit to clear cache

2014-10-08T10:29:00Z

make meaningless edit to clear cache

← Older revision		Revision as of 10:29, 8 October 2014
Line 1:		Line 1:
	The purpose of this page is to explain what's wrong with HTTP content negotiation and why you should not suggest HTTP content negotiation as a solution to a problem.		The purpose of this page is to explain what's wrong with HTTP content negotiation and why you should not suggest HTTP content negotiation as a solution to a problem.

	HTTP content negotiation has four axes: negotiation by format (<code>Accept</code>), negotiation by character encoding <code>Accept-Charset</code>, negotiation by natural language (<code>Accept-Language</code>) and negotiation by compression (<code>Accept-Encoding</code>). These axes need to be discussed separately. The only one of these that shouldn't be classified as a failure is negotiation by compression, but even it shouldn't be taken as a role model considering its verbosity for basically negotiating one bit of information.		HTTP content negotiation has four axes: negotiation by format (<code>Accept</code>), negotiation by character encoding <code>Accept-Charset</code>, negotiation by natural language (<code>Accept-Language</code>) and negotiation by compression (<code>Accept-Encoding</code>). These axes need to be discussed separately. The only one of these that shouldn't be classified as a failure is negotiation by compression, but even it shouldn't be taken as a role model considering its verbosity for basically negotiating one bit of information.

Hsivonen: /* Negotiating by format */ Fix typo from “can” to “can’t”, add punctuation

2013-08-19T06:48:23Z

Negotiating by format: Fix typo from “can” to “can’t”, add punctuation

← Older revision		Revision as of 06:48, 19 August 2013
Line 37:		Line 37:
	* You can’t rely on format negotiation as Web author, because there are always clients that accept something they don’t declare.		* You can’t rely on format negotiation as Web author, because there are always clients that accept something they don’t declare.

	* Due to the previous point, if you are a browser vendor and another vendor has shipped a browser that doesn’t declare that it support something it supports, it doesn't make sense for you to waste bytes declaring it, either, because Web authors ~~can~~ rely (solely) on the declaration anyway.		* Due to the previous point, if you are a browser vendor and another vendor has shipped a browser that doesn’t declare that it support something it supports, it doesn't make sense for you to waste bytes declaring it, either, because Web authors can’t rely (solely) on the declaration anyway.

	* Negotiated responses are so rare relative to all HTTP responses that the request bloat resulting from advertising the supported formats is almost always waste.		* Negotiated responses are so rare relative to all HTTP responses that the request bloat resulting from advertising the supported formats is almost always waste.

	* Past experience from the GIF to PNG transition suggests that authors are more likely to deploy one ~~format either~~ the old one (with worse capabilities) or the new one (forgoing support for old browsers) ~~than~~ actually bother using negotiation (as a generalization on the Web scale; there may be individual counterexamples that round to practical nothingness).		* Past experience from the GIF to PNG transition suggests that authors are more likely to deploy one format—either the old one (with worse capabilities) or the new one (forgoing support for old browsers)—than actually bother using negotiation (as a generalization on the Web scale; there may be individual counterexamples that round to practical nothingness).

	* It never pays to advertise a format once all current browsers support it but there are still old browsers that don't. (E.g. SVG as of 2013.) Since there are browsers that support the format that don't advertise it, the earlier point about not being able to rely on the declaration applies. However, since non-support is a past phenomenon, it is safe to UA sniff for the legacy browsers. If all current browsers support the format, there probably cannot be successful future browsers that don't support the format so the usual problem with you a sniffing where you assume that the current defect stays present in the future versions of a given browser does not apply.		* It never pays to advertise a format once all current browsers support it but there are still old browsers that don't. (E.g. SVG as of 2013.) Since there are browsers that support the format that don't advertise it, the earlier point about not being able to rely on the declaration applies. However, since non-support is a past phenomenon, it is safe to UA sniff for the legacy browsers. If all current browsers support the format, there probably cannot be successful future browsers that don't support the format so the usual problem with you a sniffing where you assume that the current defect stays present in the future versions of a given browser does not apply.

GPHemsley: added Category:Justifications using HotCat

2013-07-09T14:35:20Z

added Category:Justifications using HotCat

← Older revision		Revision as of 14:35, 9 July 2013
Line 91:		Line 91:
	less cache-friendly than letting the browser choose from		less cache-friendly than letting the browser choose from
	distinct URLs offered in the referring document.		distinct URLs offered in the referring document.

			[[Category:Justifications]]

Hsivonen: Emphasis wiki syntax, for real. Plus a clarification

2013-01-30T08:39:55Z

Emphasis wiki syntax, for real. Plus a clarification

← Older revision		Revision as of 08:39, 30 January 2013
Line 79:		Line 79:
	Suppose first a browser that declares support for WebM/VP8/Vorbis request the video URL. The intermediate cache has no data for that URL yet, so it forwards the request to the origin server and caches the result with the URL and the value of the <code>Accept-Codecs</code> header as the cache key. Then another browser that declares support for MP4/H.264/AAC fetches the same URL from the same intermediate cache. The value of the <code>Accept-Codecs</code> header does not match the value in the existing cache key, so the intermediary cannot just go and serve the existing cache entry. Instead, it forwards the request to the origin server, gets a different response and makes new cache entry for that response with the URL and the different <code>Accept-Codecs</code> header value as the cache key. So now the intermediary has in its cache all the distinct video files that the origin server has for this URL. If the solution was properly cache-friendly, all subsequent requests could be served straight from the cache without consulting with the origin server until the entries' time to live expires.		Suppose first a browser that declares support for WebM/VP8/Vorbis request the video URL. The intermediate cache has no data for that URL yet, so it forwards the request to the origin server and caches the result with the URL and the value of the <code>Accept-Codecs</code> header as the cache key. Then another browser that declares support for MP4/H.264/AAC fetches the same URL from the same intermediate cache. The value of the <code>Accept-Codecs</code> header does not match the value in the existing cache key, so the intermediary cannot just go and serve the existing cache entry. Instead, it forwards the request to the origin server, gets a different response and makes new cache entry for that response with the URL and the different <code>Accept-Codecs</code> header value as the cache key. So now the intermediary has in its cache all the distinct video files that the origin server has for this URL. If the solution was properly cache-friendly, all subsequent requests could be served straight from the cache without consulting with the origin server until the entries' time to live expires.

	But server-side negotiation isn't cache-friendly in that way. Suppose that now a browser that declares support for WebM/VP8/Vorbis ~~_and_~~ WebM/VP8/Opus makes a request for the same URL. The value of the <code>Accept-Codecs</code> header matches neither of the cache keys, so the intermediary needs to consult with the origin server even though it has cached all the possible variants already.		But server-side negotiation isn't cache-friendly in that way. Suppose that now a browser that declares support for WebM/VP8/Vorbis ''and'' WebM/VP8/Opus makes a request for the same URL. The value of the <code>Accept-Codecs</code> header matches neither of the cache keys, so the intermediary needs to consult with the origin server even though it has cached all the possible variants already and one of them would be acceptable to this browser.

	With the actual design of the video element, this problem does not occur. Each alternative video		With the actual design of the video element, this problem does not occur. Each alternative video

Hsivonen: Emphasis wiki syntax

2013-01-30T08:37:53Z

Emphasis wiki syntax

← Older revision		Revision as of 08:37, 30 January 2013
Line 79:		Line 79:
	Suppose first a browser that declares support for WebM/VP8/Vorbis request the video URL. The intermediate cache has no data for that URL yet, so it forwards the request to the origin server and caches the result with the URL and the value of the <code>Accept-Codecs</code> header as the cache key. Then another browser that declares support for MP4/H.264/AAC fetches the same URL from the same intermediate cache. The value of the <code>Accept-Codecs</code> header does not match the value in the existing cache key, so the intermediary cannot just go and serve the existing cache entry. Instead, it forwards the request to the origin server, gets a different response and makes new cache entry for that response with the URL and the different <code>Accept-Codecs</code> header value as the cache key. So now the intermediary has in its cache all the distinct video files that the origin server has for this URL. If the solution was properly cache-friendly, all subsequent requests could be served straight from the cache without consulting with the origin server until the entries' time to live expires.		Suppose first a browser that declares support for WebM/VP8/Vorbis request the video URL. The intermediate cache has no data for that URL yet, so it forwards the request to the origin server and caches the result with the URL and the value of the <code>Accept-Codecs</code> header as the cache key. Then another browser that declares support for MP4/H.264/AAC fetches the same URL from the same intermediate cache. The value of the <code>Accept-Codecs</code> header does not match the value in the existing cache key, so the intermediary cannot just go and serve the existing cache entry. Instead, it forwards the request to the origin server, gets a different response and makes new cache entry for that response with the URL and the different <code>Accept-Codecs</code> header value as the cache key. So now the intermediary has in its cache all the distinct video files that the origin server has for this URL. If the solution was properly cache-friendly, all subsequent requests could be served straight from the cache without consulting with the origin server until the entries' time to live expires.

	But server-side negotiation isn't cache-friendly in that way. Suppose that now a browser that declares support for WebM/VP8/Vorbis ~~and~~ WebM/VP8/Opus makes a request for the same URL. The value of the <code>Accept-Codecs</code> header matches neither of the cache keys, so the intermediary needs to consult with the origin server even though it has cached all the possible variants already.		But server-side negotiation isn't cache-friendly in that way. Suppose that now a browser that declares support for WebM/VP8/Vorbis _and_ WebM/VP8/Opus makes a request for the same URL. The value of the <code>Accept-Codecs</code> header matches neither of the cache keys, so the intermediary needs to consult with the origin server even though it has cached all the possible variants already.

	With the actual design of the video element, this problem does not occur. Each alternative video		With the actual design of the video element, this problem does not occur. Each alternative video

Hsivonen: Mention codec example in intro

2013-01-30T08:35:40Z

Mention codec example in intro

← Older revision		Revision as of 08:35, 30 January 2013
Line 2:		Line 2:

	HTTP content negotiation has four axes: negotiation by format (<code>Accept</code>), negotiation by character encoding <code>Accept-Charset</code>, negotiation by natural language (<code>Accept-Language</code>) and negotiation by compression (<code>Accept-Encoding</code>). These axes need to be discussed separately. The only one of these that shouldn't be classified as a failure is negotiation by compression, but even it shouldn't be taken as a role model considering its verbosity for basically negotiating one bit of information.		HTTP content negotiation has four axes: negotiation by format (<code>Accept</code>), negotiation by character encoding <code>Accept-Charset</code>, negotiation by natural language (<code>Accept-Language</code>) and negotiation by compression (<code>Accept-Encoding</code>). These axes need to be discussed separately. The only one of these that shouldn't be classified as a failure is negotiation by compression, but even it shouldn't be taken as a role model considering its verbosity for basically negotiating one bit of information.

			This page explains what’s wrong with each of the existing negotiation axes and then uses the HTML <code>video</code> element as an example for illustrating why HTTP-based negotiation in general is worse than letting the browser choose from alternative URLs by showing how an HTTP-based codec negotiation solution would be worse than the actual browser-side codec negotiation solution that <code>video</code> element uses.

	==Negotiation by compression==		==Negotiation by compression==

Hsivonen: Explain why all conneg in less cache-friendly than letting the browser choose

2013-01-30T08:32:38Z

Explain why all conneg in less cache-friendly than letting the browser choose

← Older revision		Revision as of 08:32, 30 January 2013
Line 42:		Line 42:

	* It never pays to advertise a format once all current browsers support it but there are still old browsers that don't. (E.g. SVG as of 2013.) Since there are browsers that support the format that don't advertise it, the earlier point about not being able to rely on the declaration applies. However, since non-support is a past phenomenon, it is safe to UA sniff for the legacy browsers. If all current browsers support the format, there probably cannot be successful future browsers that don't support the format so the usual problem with you a sniffing where you assume that the current defect stays present in the future versions of a given browser does not apply.		* It never pays to advertise a format once all current browsers support it but there are still old browsers that don't. (E.g. SVG as of 2013.) Since there are browsers that support the format that don't advertise it, the earlier point about not being able to rely on the declaration applies. However, since non-support is a past phenomenon, it is safe to UA sniff for the legacy browsers. If all current browsers support the format, there probably cannot be successful future browsers that don't support the format so the usual problem with you a sniffing where you assume that the current defect stays present in the future versions of a given browser does not apply.

			==Server-side choice is worse for intermediate caches than browser-side choice==

			Aside from script-driven HTTP requests (XHR), the HTTP requests made by browsers fall into two main categories: requests that arise from navigation
			(HTML most of the time) and requests that arise from resource
			inclusions (images, videos, audio, style sheets, fonts and scripts).

			For resource inclusions, the server has always already sent
			(unless the site uses SPDY push) an HTML document to the browser and
			then the browser makes the requests in response to what it sees in
			that document. That is, the server has had an opportunity to say something
			to the browser already.

			It is more friendly for intermediate caches for the resource (typically HTML document) that the server has alread sent to the browser to list alternative URLs for a resource inclusion request with information that allows the browser to choose which one of the URLs to fetch than to give the browser one URL and have the server vary what gets served from that URL.

			Consider the actual design of the video element versus a hypothetical HTTP-based negotiation design
			like this: There are no <code>source</code> elements. There is just the <code>src</code>
			attribute on the <code>video</code> element. When making the request for the URL
			given in the <code>src</code> attribute, the browser sends an <code>Accept-Codecs</code> HTTP
			header that declares the supported container and codec combinations.
			The server decides which video file to respond with depending on the
			request header. The server sets the <code>Vary: Accept-Codecs</code> header on the response.

			Now, intermediate caches have to use the value of the <code>Accept-Codecs</code>
			header as part of the cache key.

			Suppose the origin server has
			two alternative video files MP4/H.264/AAC and WebM/VP8/Vorbis.
			Ideally, this should result in at most two distinct cache entries and two cache keys in
			an intermede cache. However, in this scenario, the worst case number for
			the cache keys in the intermediary is the number of distinct
			<code>Accept-Codec</code> values sent by the browsers out there.

			Suppose first a browser that declares support for WebM/VP8/Vorbis request the video URL. The intermediate cache has no data for that URL yet, so it forwards the request to the origin server and caches the result with the URL and the value of the <code>Accept-Codecs</code> header as the cache key. Then another browser that declares support for MP4/H.264/AAC fetches the same URL from the same intermediate cache. The value of the <code>Accept-Codecs</code> header does not match the value in the existing cache key, so the intermediary cannot just go and serve the existing cache entry. Instead, it forwards the request to the origin server, gets a different response and makes new cache entry for that response with the URL and the different <code>Accept-Codecs</code> header value as the cache key. So now the intermediary has in its cache all the distinct video files that the origin server has for this URL. If the solution was properly cache-friendly, all subsequent requests could be served straight from the cache without consulting with the origin server until the entries' time to live expires.

			But server-side negotiation isn't cache-friendly in that way. Suppose that now a browser that declares support for WebM/VP8/Vorbis and WebM/VP8/Opus makes a request for the same URL. The value of the <code>Accept-Codecs</code> header matches neither of the cache keys, so the intermediary needs to consult with the origin server even though it has cached all the possible variants already.

			With the actual design of the video element, this problem does not occur. Each alternative video
			file has a distinct URL, there is no need for the origin server to set
			the Vary header and the result is as cache-friendly as possible. That
			is, the worst case number of distinct cache keys is the number of
			distinct videos on the origin server, so after the intermediary has cached them all, there is no need to consult with the origin server until the cache entries expire due to their age.

			This example illustrates that solving the problem of choosing
			between alternative resources on the HTTP level ends up being
			less cache-friendly than letting the browser choose from
			distinct URLs offered in the referring document.

Hsivonen: Mention accept-charset, discuss HTTP versioning some more

2013-01-17T12:20:14Z

Mention accept-charset, discuss HTTP versioning some more

← Older revision		Revision as of 12:20, 17 January 2013
Line 1:		Line 1:
	The purpose of this page is to explain what's wrong with HTTP content negotiation and why you should not suggest HTTP content negotiation as a solution to a problem.		The purpose of this page is to explain what's wrong with HTTP content negotiation and why you should not suggest HTTP content negotiation as a solution to a problem.

	HTTP content negotiation has ~~three~~ axes: negotiation by format (<code>Accept</code>), negotiation by natural language (<code>Accept-Language</code>) and negotiation by compression (<code>Accept-Encoding</code>). These axes need to be discussed separately. The only one of these that shouldn't be classified as a failure is negotiation by compression, but even it shouldn't be taken as a role model considering its verbosity for basically negotiating one bit of information.		HTTP content negotiation has four axes: negotiation by format (<code>Accept</code>), negotiation by character encoding <code>Accept-Charset</code>, negotiation by natural language (<code>Accept-Language</code>) and negotiation by compression (<code>Accept-Encoding</code>). These axes need to be discussed separately. The only one of these that shouldn't be classified as a failure is negotiation by compression, but even it shouldn't be taken as a role model considering its verbosity for basically negotiating one bit of information.

	==Negotiation by compression==		==Negotiation by compression==
Line 8:		Line 8:

	These days, all major browsers support gzipped responses. In that sense, the feature is a success. However, it is terribly wasteful that each request ends up containing 23 bytes of boilerplate. It would have been much more efficient if HTTP 1.1 had made the ability to accept gzipped responses a mandatory feature so that servers would have known that they are eligible to send a gzipped response if the request says HTTP/1.1 (or higher) rather than HTTP/1.0.		These days, all major browsers support gzipped responses. In that sense, the feature is a success. However, it is terribly wasteful that each request ends up containing 23 bytes of boilerplate. It would have been much more efficient if HTTP 1.1 had made the ability to accept gzipped responses a mandatory feature so that servers would have known that they are eligible to send a gzipped response if the request says HTTP/1.1 (or higher) rather than HTTP/1.0.

			One might argue that it was important for getting from the HTTP/1.0 world to the world we have now by allowing new features to be opted into one by one and that format and protocol versions are an anti-pattern on the Web. In any case, it’s sad for the 1 bit of information to take 23 bytes—especially if HTTP is to have version number anyway and some features tied to the version number (like support for chunked responses).

			==Negotiation by character encoding==

			UTF-8 was introduced in 1993. By the time <code>Accept-Charset</code> was introduced, it was already obsolete in the sense that any server capable of converting to multiple encoding could have used its conversion capabilities to just convert to UTF-8. (Granted, UTF-8 support in browsers didn’t appear right away, but supporting UTF-8 sooner would have been more worthwhile than sending <code>Accept-Charset</code>.) Fortunately, we have been able to get rid of this one. Of the major browsers, only Chrome sends <code>Accept-Charset</code> anymore.

	==Negotiation by natural language==		==Negotiation by natural language==