A user account is required in order to edit this wiki, but we've had to disable public user registrations due to spam.

To request an account, ask an autoconfirmed user on IRC (such as one of these permanent autoconfirmed members).

Difference between revisions of "ImageBitmap Options"

From WHATWG Wiki
Jump to: navigation, search
(Created page with " This page is a Work in Progress '''Problem Statement''' Loading and preparing image resources for rendering to 2D canvas contexts and WebGL contexts is often inefficient us...")
 
Line 3: Line 3:
  
  
'''Problem Statement'''
+
'''Problem to be solved''': Loading and preparing image resources for rendering to 2D canvas contexts and WebGL contexts is often inefficient using existing APIs.
Loading and preparing image resources for rendering to 2D canvas contexts and WebGL contexts is often inefficient using existing web APIs. Many resource preparation steps are implementable in JavaScript, but could be performed much faster and more conveniently with proper API support and native implementations provided by the browser. Examples of such tasks: color correction, image flipping, alpha premultiplication.  
 
  
 
== Use Case Description ==
 
== Use Case Description ==
* A web application needs to load an image and may or may not need to flip the image vertically, depending on whether client-side code has selected to draw using a 2D canvas context or a WebGL context.
+
:WebGL applications often use image media for transferring non-image data (illumination maps, vector maps, etc.) For such applications the image data must usually be decoded without colorspace conversions and without alpha premultiplications.
* WebGL applications often use image media for transferring non-image data (illumination maps, vector maps, etc.) For such applications the image data must usually be decoded without colorspace conversions and without alpha premultiplications.
 
  
 
=== Current Usage and Workarounds ===
 
=== Current Usage and Workarounds ===
  
:Currently, there are several ways to apply the transformations on the client side, all of which involve intermediate copies of the image data. Some solutions involve DataURLs, temporary canvas, or direct manipulation of image data arrays, all of which are inefficient. Preparing the resources asynchronously also presents an additional challenge to the web developer.
+
:Currently, there are several ways to apply arbitrary transformations on the client side, many of which involve intermediate copies of the image data. Some solutions involve DataURLs, temporary canvases, or direct manipulation of image data arrays, which are inefficient. Preparing the resources asynchronously also presents an additional challenge to the web developer.
  
:WebGL applications can request that colorspace conversions be skipped at texture upload time by specifying pixel storage parameters, but this currently causes a synchronous image decode, because it is not known in advance that the image needs to be decoded in a special way. This undue latency can cause animation to be janky.
+
:WebGL applications can request that colorspace conversions be skipped at texture upload time by specifying pixel storage parameters, but this currently causes a synchronous image decode, because it is not known in advance that the image will need to be decoded in a special way. This undue latency can cause animations to be janky.
  
 
=== Benefits ===
 
=== Benefits ===
  
:With the right API support, these image preparation steps could be performed by native code, and in many case they can be integrated directly into the browser's image decoding process without any additional memory consumption and little additional computational burden.
+
:With the right API support, image resources can be prepared asynchronously and/or in advance to avoid rendering latency issues. Furthermore, the preparation steps could be performed by native code, and in many case they can be integrated directly into the image decoding step without any additional memory consumption and little or no additional computational burden.
  
 
=== Requests for this Feature ===
 
=== Requests for this Feature ===
Line 34: Line 32:
 
* boolean flipY, default = false <blockquote><p>If set to true, the image is flipped vertically</p></blockquote>
 
* boolean flipY, default = false <blockquote><p>If set to true, the image is flipped vertically</p></blockquote>
 
* boolean colorspaceConversion, default = true <blockquote><p>If set to false, the ImageBitmap must be created by decoding the source image media without performing any colorspace conversions. This means that the image decoding algorithm must ignore color profile meta-data embedded in the image source as well as the display device color profile.  colorspaceConversion only applies when creating ImageBitmaps from bitmap media data (i.e. Blobs and Image elements)</p></blockquote>
 
* boolean colorspaceConversion, default = true <blockquote><p>If set to false, the ImageBitmap must be created by decoding the source image media without performing any colorspace conversions. This means that the image decoding algorithm must ignore color profile meta-data embedded in the image source as well as the display device color profile.  colorspaceConversion only applies when creating ImageBitmaps from bitmap media data (i.e. Blobs and Image elements)</p></blockquote>
* boolean premultiplyAlpha, default = true <blockquote><p>If set to false, the image is decoded without multiplying color components by the alpha component. Only applies when creating ImageBitmaps from bitmap media data. Alpha multiplication is never performed type of image sources  </p></blockquote>
+
* boolean premultiplyAlpha, default = true <blockquote><p>If set to false, the image is decoded without multiplying color components by the alpha component. Only applies when creating ImageBitmaps from bitmap media data. Alpha multiplication is never performed on other types of image sources  </p></blockquote>
  
 
==== Other potentially useful options ====
 
==== Other potentially useful options ====
Line 53: Line 51:
  
 
==== Adoption ====  
 
==== Adoption ====  
* Easier to develop asychronous image resource loading and preparation in JavaScript
+
* Easier to develop asychronous image resource loading and preparation
 
* Better performance that what can be implemented using current APIs
 
* Better performance that what can be implemented using current APIs
 +
* Applications can rely on the browser resource management facilities for caching decoded image data with the desired options baked-in and for evicting pre-decoded resources in low memory conditions to avoid out-of-memory crashes.
  
 
[[Category:Proposals]]
 
[[Category:Proposals]]

Revision as of 18:49, 17 July 2013

This page is a Work in Progress


Problem to be solved: Loading and preparing image resources for rendering to 2D canvas contexts and WebGL contexts is often inefficient using existing APIs.

Use Case Description

WebGL applications often use image media for transferring non-image data (illumination maps, vector maps, etc.) For such applications the image data must usually be decoded without colorspace conversions and without alpha premultiplications.

Current Usage and Workarounds

Currently, there are several ways to apply arbitrary transformations on the client side, many of which involve intermediate copies of the image data. Some solutions involve DataURLs, temporary canvases, or direct manipulation of image data arrays, which are inefficient. Preparing the resources asynchronously also presents an additional challenge to the web developer.
WebGL applications can request that colorspace conversions be skipped at texture upload time by specifying pixel storage parameters, but this currently causes a synchronous image decode, because it is not known in advance that the image will need to be decoded in a special way. This undue latency can cause animations to be janky.

Benefits

With the right API support, image resources can be prepared asynchronously and/or in advance to avoid rendering latency issues. Furthermore, the preparation steps could be performed by native code, and in many case they can be integrated directly into the image decoding step without any additional memory consumption and little or no additional computational burden.

Requests for this Feature

  • Source

    I would like this feature ...

Proposed Solutions

ImageBitmap Construction Options

Add an optional options argument to createImageBitmap to specify image treatments to be applied when creating the ImageBitmap object.

Strongly desired options

  • boolean flipY, default = false

    If set to true, the image is flipped vertically

  • boolean colorspaceConversion, default = true

    If set to false, the ImageBitmap must be created by decoding the source image media without performing any colorspace conversions. This means that the image decoding algorithm must ignore color profile meta-data embedded in the image source as well as the display device color profile. colorspaceConversion only applies when creating ImageBitmaps from bitmap media data (i.e. Blobs and Image elements)

  • boolean premultiplyAlpha, default = true

    If set to false, the image is decoded without multiplying color components by the alpha component. Only applies when creating ImageBitmaps from bitmap media data. Alpha multiplication is never performed on other types of image sources

Other potentially useful options

  • frameIndex: for creating an image bitmap from multi-frame image media
  • flipX: like flipY but horizontal
  • colorspace: to decode images to a colorspace other than the device colorspace (sRGB, Adobe RGB, YUV, YCbCr, etc.)
  • custom color tranforms: color matrix, 3D Look up tables, per component look up tables, bias, gain, ...

Processing Model

TBD Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.

Limitations

TBD Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.

Implementation

TBD Description of how and why browser vendors would take advantage of this feature.

Adoption

  • Easier to develop asychronous image resource loading and preparation
  • Better performance that what can be implemented using current APIs
  • Applications can rely on the browser resource management facilities for caching decoded image data with the desired options baked-in and for evicting pre-decoded resources in low memory conditions to avoid out-of-memory crashes.