Adaptive images

Summary

A markup-based means of delivering alternate image sources based on device capabilities to prevent wasted bandwidth and optimize display for both screen and print.

The idea is to use the video tag’s markup pattern as the inspiration, as it’s specced to allow the use of media queries within attributes on its source elements and reliably displays the markup inside the tag in any browser that doesn’t recognize it. Through use of media attributes we would not only be able to reduce wasteful image requests for the sake of users with smaller displays, but we would be able tailor our images’ resolutions for users with high-res displays or for print.

Thank you for your consideration, Mat Marquis and the Responsive Images Community Group

Example Use Cases

Serving full-bleed images within a flexible layout or a layout dictated by media queries requires a source image with the largest necessary inherent size and scaling it down through CSS. On smaller displays such as phones and tablets—where bandwidth can be at a premium—this means an exceptionally wasteful request.

While there are currently “responsive images” solutions that deliver smaller images by default and conditionally load a larger image above a certain window size, all of these involve a redundant request and rely fully on JavaScript. On large displays, an image’s src will be prefetched prior to any logic that swaps the image in many modern browsers. This is further detailed in this post.

This limitation could be avoided in a native implementation.

Mobile-First Development

<picture alt="">
	<!-- Matches by default: -->
	<source src="mobile.jpg" /> 
	<source src="medium.jpg" media="min-width: 600px" /> 	
	<source src="fullsize.jpg" media="min-width: 900px" />
	<img src="mobile.jpg" />
</picture>

Assuming a 960px wide window at the time the page is requested and the sample markup pattern in section 3.1, the UA should make a single request for “fullsize.jpg.” Any window/screen smaller than 600px is served “mobile.jpg”, which—as a completely alternate source—could be cropped as well as resized in order to preserve the focus of the image at smaller sizes.

This example could be made more specific through the orientation, device-width, and device-height media queries.

Desktop-First Development

Following the reverse of the logic above, authors may want to ensure that the largest possible images are served by default. For example: a site intended for browsing across a range of “smart televisions” where one assumes a large viewport, but assets could be optimized in the event that media queries are supported. This use case could be further tailored through the aspect-ratio and device-aspect-ratio media queries.

<picture alt="">
	<!-- Matches by default: -->
	<source src="fullsize.jpg" /> 
	<source src="medium.jpg" media="max-width: 1400px" /> 	
	<source src="smaller.jpg" media="max-width: 1280px" />
	<img src="mobile.jpg" />
</picture>

Relative Units

A common practice in creating flexible layouts is to specify the value of one’s media queries in relative units: em, rem, vw/vh etc. This is most commonly seen with ems in order to reflow layouts based on users’ zoom preferences, or to resize elements through JavaScript by dynamically altering a font-size value.

Through the use of media queries this layout flexibility would extend to images as well, ensuring that a source always remains appropriate to the size of its containing element.

<picture alt="">
	<!-- Matches by default: -->
	<source src="mobile.jpg" /> 
	<source src="medium.jpg" media="min-width: 37.5em" /> 	
	<source src="fullsize.jpg" media="min-width: 56.25em" />
	<img src="mobile.jpg" />
</picture>

High-Resolution Displays

High resolution screens such as Apple’s Retina display will require high-resolution images, leaving us with a situation similar to the above: either serving larger, high-resolution images to displays that can’t take advantage of them, or forcing high-density displays to first download a low-resolution image then replace it with a high-resolution image. The latter—the approach currently used on Apple’s website—is far from ideal, and the former is so fraught with concerns that it has recently been adressed in such mainstream publications as the New York Times.

<picture alt="">
	<!-- Matches by default: -->
	<source src="standard-res.jpg" /> 	
	<source src="high-res.jpg" media="min-device-pixel-ratio: 1.5" />
	<source src="highest-res.jpg" media="min-device-pixel-ratio: 2.5" />
	<img src="standard-res.jpg" />
</picture>

In this instance, the standard resolution image is served by default and as a fallback in cases where <picture> is unsupported. The high resolution image is served instead only in cases where the pixel-ratio media query matches. Naturally, this can be used in combination with min-width and max-width media queries to ensure the most appropriate possible image source.

Alternate Print Sources

Where media queries allow conditional logic based on whether the page is being rendered for display on screen or for print, one could extend this same logic to provide a print source other than the one rendered on the screen. For example: a photo sharing site could provide a bandwidth-optimized image for display on screen, but a high resolution image for print.

<picture alt="">
	<source src="standard-res.jpg" /> 	
	<source src="high-res.jpg" media="print, min-device-pixel-ratio: 1.5" />
	<img src="standard-res.jpg" />
</picture>

DOM Scripting and Bandwidth

In treating sources as independent elements, we make on-the-fly manipulation of elements via the DOM infinitely more simple compared to parsing and manipulating a string of sources contained in a single attribute. Without an implemented version of picture this remains purely speculation, but simplicity in manipulating these sources combined with the rapid adoption of navigator.connection may open the door for sources more tailored to a user’s internet connection.

Of course—given the flexibility and constant evolution of media queries—we may even be able to specify connection speed within our media attributes one day.

Proposal Details

Any combination of existing media queries can be used to determine the appropriate picture source, through a media attribute on source elements. This is identical to the specced behavior of media attributes on the video’s source elements, as outlined here. Any implementation of the picture tag should allow for the inclusion of fallback markup that is completely ignored by any UA that supports picture, and is only displayed in browsers that do not recognize the new tag. Note that older browsers can be polyfilled with behavior similar to a native implementation.

Example Markup Pattern

<picture alt="The alt attribute’s content should accurately describe the image represented by all sources, though cropping and zooming of sources may differ.">
	<!-- Matches by default: -->
	<source src="mobile.jpg" /> 
	
	<!-- Overrides the previous source for windows greater than 600px -->
	<source src="medium.jpg" media="min-width: 600px" /> 
	
	<!-- Overrides the previous source for windows greater than 900px -->
	<source src="fullsize.jpg" media="min-width: 900px" /> 
	
	 <!-- Fallback content, only displayed in the event the <picture> tag is unsupported by the browser: --> 
	<img src="mobile.jpg" />
</picture>

Functional Polyfills

Currently two polyfills exist to bring the proposed functionality of the picture element to non-supporting browsers:

• Scott Jehl’s Picturefill
• Abban Dunne’s jQuery Picture

A reliable native fallback is provided in the event that the new functionality isn’t supported and no polyfill is applied, using the same markup-based pattern as the video tag.

Requirements

• The appropriate asset MUST be fetched by way of a single request. A change in window size causing the media attribute to match an alternate source SHOULD trigger a request for said source (to be retrieved from the browser cache, if possible).
• As with the <video> and <audio> tags, this solution MUST NOT require any client-side scripting, server-side technologies, or headers to reliably deliver content tailored for the end user’s context.
• Similar to the <video> tag, fallback markup MUST be rendered in any browser that does not recognize the <picture> element. The example in 3.1 uses the “mobile”-sized image as the fallback content, which is the recommended approach: barring the use of a polyfill, the smaller/low-res image should be provided as a fallback to prevent incurring a costly download in contexts that may see no benefit.
• The specification MUST provide at least the same level of accessibility as <img>, with an alt attribute readily accessible to assistive technology.