https://wiki.whatwg.org/api.php?action=feedcontributions&feedformat=atom&user=KenrussellWHATWG Wiki - User contributions [en]2026-05-23T06:03:58ZUser contributionsMediaWiki 1.39.3https://wiki.whatwg.org/index.php?title=OffscreenCanvas.requestAnimationFrame&diff=10287OffscreenCanvas.requestAnimationFrame2019-05-10T17:26:48Z<p>Kenrussell: Added note indicating this is obsoleted</p>
<hr />
<div>:''This proposal aims to provide a reliable mechanism for driving animations using OffscreenCanvas in a Worker''<br />
<br />
<br />
== NOTE: OBSOLETED ==<br />
<br />
The proposal below was [https://html.spec.whatwg.org/multipage/imagebitmap-and-animations.html#animation-frames refined and specified] as requestAnimationFrame on the DedicatedWorkerGlobalScope. This page is maintained only to maintain the history of the discussion.<br />
<br />
<br />
== Use Case Description ==<br />
An OffscreenCanvas is used in a worker to produce a sequence of frames that constitute an animation. The frames need to be produced at regular intervals that match the frame rate of the display<br />
<br />
=== Current Limitations ===<br />
<br />
=== Current Usage and Workarounds ===<br />
Currently, it is possible to use setTimeout or setInterval in a worker to invoke an animation callback at a regular interval. However, since this mechanism is not driven by the display, the following issues arise:<br />
* The display device's refresh rate needs to be guessed.<br />
* Even if the correct rate is guessed, drift in the timing will cause dropped frames and skipped frames.<br />
* In GPU-accelerated use cases, it is possible for the rendering script to run at a rate that the GPU cannot keep-up with, which may cause the accumulation of a multi-frame rendering backlog, resulting in high latency and eventually OOM crashes. A possible solution to this is for the user agent to implement a throttling mechanism, in which case the worker's event loop may be periodically de-scheduled while the GPU catches up. Such de-scheduling is bad because it prevents the worker from doing other work.<br />
<br />
Another solution is to have a requestAnimationFrame loop in the browsing context's event loop that posts a message to the worker at each animation iteration.<br />
* This mechanism may add undue latency to the signal, especially when the browsing context's event loop is busy, which completely destroys one of the key advantages of using OffscreenCanvas in a worker.<br />
* The the frame rate in the browsing context's event loop may be higher than the worker can keep up which which will require a throttling mechanism to be implemented in script<br />
* As with setTimeout/setInterval, it is possible for the rendering script to run at a rate that the GPU cannot keep-up with.<br />
<br />
=== Benefits ===<br />
<br />
New API to drive OffscreenCanvas animations can ensure optimal frame rate, minimize animation jank, prevent overdraw, and minimize display latency.<br />
<br />
=== Requests for this Feature ===<br />
<br />
* <cite>[https://github.com/w3ctag/spec-reviews/issues/141 W3C TAG review for ImageBitmapRenderingContext]</cite><blockquote><p>(...) there seems to be a good bit of concern about lack of things like requestAnimationFrame (...) -- L. David Barron, Mozilla</p></blockquote><br />
* <cite>[https://lists.w3.org/Archives/Public/public-whatwg-archive/2015Aug/0019.html whatwg mailing list thread "Worker requestAnimationFrame"]</cite> <blockquote><p>For OffscreenCanvas we need a way for a Worker to draw once per composited frame. --Robert O'Callahan, Mozilla<br />
</p></blockquote><br />
<br />
== Proposed Solutions ==<br />
<br />
=== OffscreenCanvas.requestAnimationFrame ===<br />
<br />
Works much like window.requestAnimationFrame, except that the scheduling of callbacks is independent of the browsing context event loop, and therefore is not necessarily synchronized with graphics updates from the browsing context.<br />
<br />
==== Processing Model ====<br />
<br />
The requestAnimationFrame() and cancelAnimationFrame() methods shall be spec'ed almost identically to their Window interface counterparts, except that the callback list would be stored in the OffscreenCanvas object.<br />
<br />
The main difference with respect to the Window.requestAnimationFrame processing model is in how the callbacks are scheduled. In a browsing context, the animation callbacks are coordinated with the graphics update in the [https://html.spec.whatwg.org/multipage/webappapis.html#event-loop-processing-model event loop processing model]. Such coordination shall not exist in workers. <br />
<br />
For OffscreenCanvases, the user agent will schedule an "animation task" to run all of the OffscreenCanvas's pending animation callbacks. in a way the respects the following constraints:<br />
* No overdraw: An animation frame committed from an animation task shall not replace an animation frame from a previous animation task until that previous frame has been rendered to the display device.<br />
* animation tasks shall be scheduled at the highest possible rate that can be maintained without going into overdraw and without accumulating a backlog of more than one pending frame.<br />
<br />
===== Special Cases =====<br />
<br />
* When rAF is invoked on an OffscreenCanvas that does not have a placeholder canvas and is not linked to a VRLayer, throw an InvalidStateError. The OffscreenCanvas content must be composited for the rAF processing model to make sense.<br />
* Attempting to transfer an OffscreenCanvas object with a non-empty animation callback list throws an InvalidStateError.<br />
* Attempting to construct a VRLayer using an OffscreenCanvas object with a non-empty animation callback list throws an InvalidStateError.<br />
* When the OffscreenCanvas is associated with a VRLayer, all calls to {request|cancel}AnimationFrame must be forwarded to the VRLayer's VRDisplay's {request|cancel}AnimationFrame methods. This implies that when the OffscreenCanvas simultaneously is visible through a placeholder canvas and a VR device, the animation loop is driven by the VR device.<br />
* The animations tasks for different OffscreenCanvas objects that live in the same event loop are not necessarily synchronized. <br />
<br />
==== Open issues ==== <br />
Calling commit() on a given OffscreenCanvas multiple times in the same animation frame is problematic. Possible way of handling the situation:<br />
* Drop all commits but the last one (or the first one?)<br />
* Queue multiple frames and wait for all of them to have be displayed before scheduling the next animation task<br />
* Throw an exception<br />
<br />
What to do if commit() is not called from within the animation callback? This is problematic because <br />
* Do an implicit commit()<br />
* Repeat the previous frame<br />
* Schedule the next animation frame immediately.<br />
* Prevent this from ever happening: Let OffscreenCanvas object have a needsCommit flag that is initially false. Set needsCommit to true at the beginning of an animation task. Set needsCommit to false when commit is called. When requestAnimationFrame is called, throw an exception if needsCommit is true.<br />
<br />
Should it be possible to commit() the contents of other canvases from within a rAF callback? <br />
<br />
=== OffscreenCanvas.commit() to return a promise ===<br />
<br />
An alternate solution would be to have commit() return a promise that gets resolved when it is time to begin rendering the next frame. This single API entry-point provides the necessary flexibility to handle continuous animations as well as sporadic updates.<br />
<br />
Continuous animation example:<br />
<br />
<nowiki>function animationLoop() {<br />
// draw stuff<br />
(...)<br />
ctx.commit().then(animationLoop);<br />
// do post commit work<br />
(...)<br />
}</nowiki><br />
<br />
Another possibility is to use the async/await syntax:<br />
<br />
<nowiki>async function animationLoop() {<br />
var promise;<br />
do {<br />
//draw stuff<br />
(...)<br />
promise = ctx.commit()<br />
// do post commit work<br />
(...)<br />
} while (await promise);<br />
}</nowiki><br />
<br />
To animate multiple canvases in lock-step, one could do this, for eaxample:<br />
<br />
<nowiki>function animationLoop() {<br />
// draw stuff<br />
(...)<br />
Promise.all([ctx1.commit(), ctx2.commit()]).then(animationLoop);<br />
}</nowiki><br />
<br />
<br />
For occasional update use cases, it is just a matter of ignoring the promise returned by commit() and to drive the animation using another signal, for example a network event. In the case where multiple calls to commit are made in the same frame interval, the user agent skips frames in order to avoid accumulating a multi-frame backlog, as described in the processing model below.<br />
<br />
==== Processing Model ====<br />
<br />
An OffscreenCanvas object has a ''pendingFrame'' internal slot that stores a reference to the frame that was captured by the last call to commit(). The reference is held until the frame is actually committed. 'pendingFrame' is initially unset.<br />
<br />
An OffscreenCanvas object has a ''pendingPromise'' internal slot that stores a reference to the promise that was returned by the last call to commit(). [[pendingPromise]] is initially unset, and its reference is only retained while the promise is in the unresolved state.<br />
<br />
The ''BeginFrame'' signal is a signal that is dispatched by the UserAgent to a specific OffscreenCanvas when it is time to render the next animation frame for the OffscreenCanvas.<br />
<br />
When commit() is called:<br />
# Let ''frame'' be a copy of the current contents of the canvas.<br />
# If ''pendingPromise'' is set, then run theses substeps:<br />
## Set ''pendingFrame'' to be a reference to ''frame''.<br />
## Return ''pendingPromise''.<br />
# Set ''pendingPromise'' to be a newly created unresolved promise object.<br />
# Run the steps to '''commit a frame''', passing ''frame'' as an argument.<br />
# Return ''pendingPromise''.<br />
<br />
When the ''BeginFrame'' signal is to be dispatched to an OffscreenCanvas object, the UserAgent must queue a task on the OffscreenCanvas object's event loop that runs the following steps: <br />
# If ''pendingFrame'' is set, then run the following substeps:<br />
## Run the steps to '''commit a frame''', passing [[pendingFrame]] as an argument.<br />
## Unset ''pendingFrame''.<br />
## Abort these steps.<br />
# If ''pendingPromise'' is not set then abort these steps.<br />
# Resolve the promise referenced by ''pendingPromise''.<br />
# Unset ''pendingPromise''.<br />
<br />
When the user agent is required to run the steps to '''commit a frame''', it must do what is currently spec'ed as the steps for commit().<br />
<br />
This processing model takes care the unresolved issues with the OffscreenCanvas.requestAnimationFrame solution because it makes it safe to call commit at any time by providing the following guarantees:<br />
# In cases of overdraw (commit() called at a rate higher than can be displayed), frames may be dropped to ensure low latency (no more than one frame of backlog).<br />
# The frame captured by the last call to commit after the end of an animation sequence is never dropped. In other words, when animation stops, it is always the most recent frame that is displayed.<br />
<br />
==== Adoption ====<br />
<br />
The idea of the commit API was discussed at a meeting of the WebVR working group and has support from multiple browser vendors.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=10068OffscreenCanvas2016-05-09T17:54:16Z<p>Kenrussell: Added link to ImageBitmapRenderingContext in current WHATWG spec.</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
* (From the Google Docs team): need to be able to layout and render text from a worker using CanvasRenderingContext2D and display those results on the main thread.<br />
* (From the Google Slides team): want to layout and render the slide thumbnails from a worker. During initial load and heavy collaboration these update frequently, and currently cause slowdowns on the main thread.<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] mostly addresses these two use cases, but some implementers objected to the mechanism for displaying the rendering results in image elements. The specific objection was that image elements already have complex internal state (for example, the management of the image's "loaded" state), and this would make it more complex. It also did not precisely address the use case of producing new frames both on the main thread and in workers.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface OffscreenCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// OffscreenCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the OffscreenCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the OffscreenCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
OffscreenCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
// It's crucial that there be a way to explicitly dispose of ImageBitmaps<br />
// since they refer to potentially large graphics resources. Some uses<br />
// of this API proposal will result in repeated allocations of ImageBitmaps,<br />
// and garbage collection will not reliably reclaim them quickly enough. <br />
// Here we reuse close(), which also exists on another Transferable type,<br />
// MessagePort. Potentially, all Transferable types should inherit from a<br />
// new interface type "Closeable". <br />
partial interface ImageBitmap {<br />
// Dispose of all graphical resources associated with this ImageBitmap.<br />
void close(); <br />
};<br />
<br />
partial interface HTMLCanvasElement {<br />
OffscreenCanvas transferControlToOffscreen();<br />
};<br />
<br />
// Note that CanvasRenderingContext2D already has a commit() method<br />
// from the CanvasProxy spec which this proposal obsoletes.<br />
partial interface CanvasRenderingContext2D {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
};<br />
<br />
partial interface WebGLRenderingContextBase {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
<br />
// If this context is associated with an OffscreenCanvas that was<br />
// created by HTMLCanvasElement's transferControlToOffscreen method,<br />
// causes this context's current rendering results to be pushed<br />
// to that canvas element. This has the same effect as returning<br />
// control to the main loop in a single-threaded application. Otherwise,<br />
// this call has no effect.<br />
void commit();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
//<br />
// Note: this interface has already been incorporated into the current WHATWG<br />
// specification at https://html.spec.whatwg.org/multipage/scripting.html#the-imagebitmap-rendering-context .<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferFromImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of OffscreenCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the OffscreenCanvas. The application generates new frames using the RenderingContext obtained from the OffscreenCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the OffscreenCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single OffscreenCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the OffscreenCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToOffscreeen</code>. <code>getContext</code> is used to obtain a rendering context for that OffscreenCanvas, either on the main thread, or on a worker. The application calls <code>commit</code> against that rendering context in order to push frames to the original HTMLCanvasElement. In this rendering model, it is not defined when those frames become visible in the original canvas element. However, if the following situations apply:<br />
<br />
* It is a worker thread which is calling commit(), and<br />
* The worker is calling commit() repeatedly against exactly one rendering context<br />
<br />
then it is required that the user agent synchronize the calls to commit() to the vsync interval. Calls to commit() conceptually enqueue frames for display, and after an implementation-defined number of frames have been enqueued, further calls to commit() will block until earlier frames have been presented to the screen. (This requirement allows porting of applications which drive their own main loop rather than using an event-driven loop.)<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
* Some parts of the CanvasRenderingContext2D interface shall not be supported due OffscreenCanvas objects having no relation to the DOM or Frame: HitRegions, scrollPathIntoView, drawFocusIfNeeded.<br />
* Due to technical challenges, some implementors [https://bugzilla.mozilla.org/show_bug.cgi?id=801176#c29 (Google and Mozilla)] have expressed a desire to ship without initially supporting text rendering in 2D contexts. Open Issue: Should text support be formally excluded from the specification until implementors are prepared to ship it (or until a more feasible API is designed)?<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed upon the basic form of the API, so it is likely it will be implemented widely and compatibly.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
==== Example code ====<br />
<br />
Jeff Gilbert from Mozilla has crafted some example code utilizing this API:<br />
<br />
* [https://github.com/jdashg/snippets/tree/master/webgl-from-worker Rendering WebGL from a worker using the commit() API]<br />
* [https://github.com/jdashg/snippets/blob/master/webgl-one-to-many/index.html Using one WebGL context to render to many Canvas elements]<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=10060OffscreenCanvas2016-04-22T02:09:31Z<p>Kenrussell: Renamed ImageBitmapRenderingContext.transferImageBitmap to transferFromImageBitmap per discussion with all Canvas implementers in WebGL working group.</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
* (From the Google Docs team): need to be able to layout and render text from a worker using CanvasRenderingContext2D and display those results on the main thread.<br />
* (From the Google Slides team): want to layout and render the slide thumbnails from a worker. During initial load and heavy collaboration these update frequently, and currently cause slowdowns on the main thread.<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] mostly addresses these two use cases, but some implementers objected to the mechanism for displaying the rendering results in image elements. The specific objection was that image elements already have complex internal state (for example, the management of the image's "loaded" state), and this would make it more complex. It also did not precisely address the use case of producing new frames both on the main thread and in workers.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface OffscreenCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// OffscreenCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the OffscreenCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the OffscreenCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
OffscreenCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
// It's crucial that there be a way to explicitly dispose of ImageBitmaps<br />
// since they refer to potentially large graphics resources. Some uses<br />
// of this API proposal will result in repeated allocations of ImageBitmaps,<br />
// and garbage collection will not reliably reclaim them quickly enough. <br />
// Here we reuse close(), which also exists on another Transferable type,<br />
// MessagePort. Potentially, all Transferable types should inherit from a<br />
// new interface type "Closeable". <br />
partial interface ImageBitmap {<br />
// Dispose of all graphical resources associated with this ImageBitmap.<br />
void close(); <br />
};<br />
<br />
partial interface HTMLCanvasElement {<br />
OffscreenCanvas transferControlToOffscreen();<br />
};<br />
<br />
// Note that CanvasRenderingContext2D already has a commit() method<br />
// from the CanvasProxy spec which this proposal obsoletes.<br />
partial interface CanvasRenderingContext2D {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
};<br />
<br />
partial interface WebGLRenderingContextBase {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
<br />
// If this context is associated with an OffscreenCanvas that was<br />
// created by HTMLCanvasElement's transferControlToOffscreen method,<br />
// causes this context's current rendering results to be pushed<br />
// to that canvas element. This has the same effect as returning<br />
// control to the main loop in a single-threaded application. Otherwise,<br />
// this call has no effect.<br />
void commit();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferFromImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of OffscreenCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the OffscreenCanvas. The application generates new frames using the RenderingContext obtained from the OffscreenCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the OffscreenCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single OffscreenCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the OffscreenCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToOffscreeen</code>. <code>getContext</code> is used to obtain a rendering context for that OffscreenCanvas, either on the main thread, or on a worker. The application calls <code>commit</code> against that rendering context in order to push frames to the original HTMLCanvasElement. In this rendering model, it is not defined when those frames become visible in the original canvas element. However, if the following situations apply:<br />
<br />
* It is a worker thread which is calling commit(), and<br />
* The worker is calling commit() repeatedly against exactly one rendering context<br />
<br />
then it is required that the user agent synchronize the calls to commit() to the vsync interval. Calls to commit() conceptually enqueue frames for display, and after an implementation-defined number of frames have been enqueued, further calls to commit() will block until earlier frames have been presented to the screen. (This requirement allows porting of applications which drive their own main loop rather than using an event-driven loop.)<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
* Some parts of the CanvasRenderingContext2D interface shall not be supported due OffscreenCanvas objects having no relation to the DOM or Frame: HitRegions, scrollPathIntoView, drawFocusIfNeeded.<br />
* Due to technical challenges, some implementors [https://bugzilla.mozilla.org/show_bug.cgi?id=801176#c29 (Google and Mozilla)] have expressed a desire to ship without initially supporting text rendering in 2D contexts. Open Issue: Should text support be formally excluded from the specification until implementors are prepared to ship it (or until a more feasible API is designed)?<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed upon the basic form of the API, so it is likely it will be implemented widely and compatibly.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
==== Example code ====<br />
<br />
Jeff Gilbert from Mozilla has crafted some example code utilizing this API:<br />
<br />
* [https://github.com/jdashg/snippets/tree/master/webgl-from-worker Rendering WebGL from a worker using the commit() API]<br />
* [https://github.com/jdashg/snippets/blob/master/webgl-one-to-many/index.html Using one WebGL context to render to many Canvas elements]<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9989OffscreenCanvas2015-09-18T23:32:40Z<p>Kenrussell: Added comments to asynchronous rendering model about commit() blocking to vsync interval based on feedback from Mozilla.</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
* (From the Google Docs team): need to be able to layout and render text from a worker using CanvasRenderingContext2D and display those results on the main thread.<br />
* (From the Google Slides team): want to layout and render the slide thumbnails from a worker. During initial load and heavy collaboration these update frequently, and currently cause slowdowns on the main thread.<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] mostly addresses these two use cases, but some implementers objected to the mechanism for displaying the rendering results in image elements. The specific objection was that image elements already have complex internal state (for example, the management of the image's "loaded" state), and this would make it more complex. It also did not precisely address the use case of producing new frames both on the main thread and in workers.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface OffscreenCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// OffscreenCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the OffscreenCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the OffscreenCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
OffscreenCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
// It's crucial that there be a way to explicitly dispose of ImageBitmaps<br />
// since they refer to potentially large graphics resources. Some uses<br />
// of this API proposal will result in repeated allocations of ImageBitmaps,<br />
// and garbage collection will not reliably reclaim them quickly enough. <br />
// Here we reuse close(), which also exists on another Transferable type,<br />
// MessagePort. Potentially, all Transferable types should inherit from a<br />
// new interface type "Closeable". <br />
partial interface ImageBitmap {<br />
// Dispose of all graphical resources associated with this ImageBitmap.<br />
void close(); <br />
};<br />
<br />
partial interface HTMLCanvasElement {<br />
OffscreenCanvas transferControlToOffscreen();<br />
};<br />
<br />
// Note that CanvasRenderingContext2D already has a commit() method<br />
// from the CanvasProxy spec which this proposal obsoletes.<br />
partial interface CanvasRenderingContext2D {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
};<br />
<br />
partial interface WebGLRenderingContextBase {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
<br />
// If this context is associated with an OffscreenCanvas that was<br />
// created by HTMLCanvasElement's transferControlToOffscreen method,<br />
// causes this context's current rendering results to be pushed<br />
// to that canvas element. This has the same effect as returning<br />
// control to the main loop in a single-threaded application. Otherwise,<br />
// this call has no effect.<br />
void commit();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of OffscreenCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the OffscreenCanvas. The application generates new frames using the RenderingContext obtained from the OffscreenCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the OffscreenCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single OffscreenCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the OffscreenCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToOffscreeen</code>. <code>getContext</code> is used to obtain a rendering context for that OffscreenCanvas, either on the main thread, or on a worker. The application calls <code>commit</code> against that rendering context in order to push frames to the original HTMLCanvasElement. In this rendering model, it is not defined when those frames become visible in the original canvas element. However, if the following situations apply:<br />
<br />
* It is a worker thread which is calling commit(), and<br />
* The worker is calling commit() repeatedly against exactly one rendering context<br />
<br />
then it is required that the user agent synchronize the calls to commit() to the vsync interval. Calls to commit() conceptually enqueue frames for display, and after an implementation-defined number of frames have been enqueued, further calls to commit() will block until earlier frames have been presented to the screen. (This requirement allows porting of applications which drive their own main loop rather than using an event-driven loop.)<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed upon the basic form of the API, so it is likely it will be implemented widely and compatibly.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
==== Example code ====<br />
<br />
Jeff Gilbert from Mozilla has crafted some example code utilizing this API:<br />
<br />
* [https://github.com/jdashg/snippets/tree/master/webgl-from-worker Rendering WebGL from a worker using the commit() API]<br />
* [https://github.com/jdashg/snippets/blob/master/webgl-one-to-many/index.html Using one WebGL context to render to many Canvas elements]<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9988OffscreenCanvas2015-09-18T23:22:50Z<p>Kenrussell: </p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
* (From the Google Docs team): need to be able to layout and render text from a worker using CanvasRenderingContext2D and display those results on the main thread.<br />
* (From the Google Slides team): want to layout and render the slide thumbnails from a worker. During initial load and heavy collaboration these update frequently, and currently cause slowdowns on the main thread.<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] mostly addresses these two use cases, but some implementers objected to the mechanism for displaying the rendering results in image elements. The specific objection was that image elements already have complex internal state (for example, the management of the image's "loaded" state), and this would make it more complex. It also did not precisely address the use case of producing new frames both on the main thread and in workers.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface OffscreenCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// OffscreenCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the OffscreenCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the OffscreenCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
OffscreenCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
// It's crucial that there be a way to explicitly dispose of ImageBitmaps<br />
// since they refer to potentially large graphics resources. Some uses<br />
// of this API proposal will result in repeated allocations of ImageBitmaps,<br />
// and garbage collection will not reliably reclaim them quickly enough. <br />
// Here we reuse close(), which also exists on another Transferable type,<br />
// MessagePort. Potentially, all Transferable types should inherit from a<br />
// new interface type "Closeable". <br />
partial interface ImageBitmap {<br />
// Dispose of all graphical resources associated with this ImageBitmap.<br />
void close(); <br />
};<br />
<br />
partial interface HTMLCanvasElement {<br />
OffscreenCanvas transferControlToOffscreen();<br />
};<br />
<br />
// Note that CanvasRenderingContext2D already has a commit() method<br />
// from the CanvasProxy spec which this proposal obsoletes.<br />
partial interface CanvasRenderingContext2D {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
};<br />
<br />
partial interface WebGLRenderingContextBase {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
<br />
// If this context is associated with an OffscreenCanvas that was<br />
// created by HTMLCanvasElement's transferControlToOffscreen method,<br />
// causes this context's current rendering results to be pushed<br />
// to that canvas element. This has the same effect as returning<br />
// control to the main loop in a single-threaded application. Otherwise,<br />
// this call has no effect.<br />
void commit();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of OffscreenCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the OffscreenCanvas. The application generates new frames using the RenderingContext obtained from the OffscreenCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the OffscreenCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single OffscreenCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the OffscreenCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToOffscreeen</code>. <code>getContext</code> is used to obtain a rendering context for that OffscreenCanvas, either on the main thread, or on a worker. The application calls <code>commit</code> against that rendering context in order to push frames to the original HTMLCanvasElement. In this rendering model, it is not defined when those frames become visible in the original canvas element.<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed upon the basic form of the API, so it is likely it will be implemented widely and compatibly.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
==== Example code ====<br />
<br />
Jeff Gilbert from Mozilla has crafted some example code utilizing this API:<br />
<br />
* [https://github.com/jdashg/snippets/tree/master/webgl-from-worker Rendering WebGL from a worker using the commit() API]<br />
* [https://github.com/jdashg/snippets/blob/master/webgl-one-to-many/index.html Using one WebGL context to render to many Canvas elements]<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9937OffscreenCanvas2015-04-11T00:00:42Z<p>Kenrussell: Updated Google Slides use case</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
* (From the Google Docs team): need to be able to layout and render text from a worker using CanvasRenderingContext2D and display those results on the main thread.<br />
* (From the Google Slides team): want to layout and render the slide thumbnails from a worker. During initial load and heavy collaboration these update frequently, and currently cause slowdowns on the main thread.<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] mostly addresses these two use cases, but some implementers objected to the mechanism for displaying the rendering results in image elements. The specific objection was that image elements already have complex internal state (for example, the management of the image's "loaded" state), and this would make it more complex. It also did not precisely address the use case of producing new frames both on the main thread and in workers.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface OffscreenCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// OffscreenCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the OffscreenCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the OffscreenCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
OffscreenCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
// It's crucial that there be a way to explicitly dispose of ImageBitmaps<br />
// since they refer to potentially large graphics resources. Some uses<br />
// of this API proposal will result in repeated allocations of ImageBitmaps,<br />
// and garbage collection will not reliably reclaim them quickly enough. <br />
// Here we reuse close(), which also exists on another Transferable type,<br />
// MessagePort. Potentially, all Transferable types should inherit from a<br />
// new interface type "Closeable". <br />
partial interface ImageBitmap {<br />
// Dispose of all graphical resources associated with this ImageBitmap.<br />
void close(); <br />
};<br />
<br />
partial interface HTMLCanvasElement {<br />
OffscreenCanvas transferControlToOffscreen();<br />
};<br />
<br />
// Note that CanvasRenderingContext2D already has a commit() method<br />
// from the CanvasProxy spec which this proposal obsoletes.<br />
partial interface CanvasRenderingContext2D {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
};<br />
<br />
partial interface WebGLRenderingContextBase {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
<br />
// If this context is associated with an OffscreenCanvas that was<br />
// created by HTMLCanvasElement's transferControlToOffscreen method,<br />
// causes this context's current rendering results to be pushed<br />
// to that canvas element. This has the same effect as returning<br />
// control to the main loop in a single-threaded application. Otherwise,<br />
// this call has no effect.<br />
void commit();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of OffscreenCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the OffscreenCanvas. The application generates new frames using the RenderingContext obtained from the OffscreenCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the OffscreenCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single OffscreenCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the OffscreenCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToOffscreeen</code>. <code>getContext</code> is used to obtain a rendering context for that OffscreenCanvas, either on the main thread, or on a worker. The application calls <code>commit</code> against that rendering context in order to push frames to the original HTMLCanvasElement. In this rendering model, it is not defined when those frames become visible in the original canvas element.<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed upon the basic form of the API, so it is likely it will be implemented widely and compatibly.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9936OffscreenCanvas2015-04-11T00:00:12Z<p>Kenrussell: Added use case from Google Slides team</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
* (From the Google Docs team): need to be able to layout and render text from a worker using CanvasRenderingContext2D and display those results on the main thread.<br />
* (From the Google Slides team): want to layout and render the slide thumbnails from a worker. During heavy collaboration these update frequently, and currently cause slowdowns on the main thread.<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] mostly addresses these two use cases, but some implementers objected to the mechanism for displaying the rendering results in image elements. The specific objection was that image elements already have complex internal state (for example, the management of the image's "loaded" state), and this would make it more complex. It also did not precisely address the use case of producing new frames both on the main thread and in workers.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface OffscreenCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// OffscreenCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the OffscreenCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the OffscreenCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
OffscreenCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
// It's crucial that there be a way to explicitly dispose of ImageBitmaps<br />
// since they refer to potentially large graphics resources. Some uses<br />
// of this API proposal will result in repeated allocations of ImageBitmaps,<br />
// and garbage collection will not reliably reclaim them quickly enough. <br />
// Here we reuse close(), which also exists on another Transferable type,<br />
// MessagePort. Potentially, all Transferable types should inherit from a<br />
// new interface type "Closeable". <br />
partial interface ImageBitmap {<br />
// Dispose of all graphical resources associated with this ImageBitmap.<br />
void close(); <br />
};<br />
<br />
partial interface HTMLCanvasElement {<br />
OffscreenCanvas transferControlToOffscreen();<br />
};<br />
<br />
// Note that CanvasRenderingContext2D already has a commit() method<br />
// from the CanvasProxy spec which this proposal obsoletes.<br />
partial interface CanvasRenderingContext2D {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
};<br />
<br />
partial interface WebGLRenderingContextBase {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
<br />
// If this context is associated with an OffscreenCanvas that was<br />
// created by HTMLCanvasElement's transferControlToOffscreen method,<br />
// causes this context's current rendering results to be pushed<br />
// to that canvas element. This has the same effect as returning<br />
// control to the main loop in a single-threaded application. Otherwise,<br />
// this call has no effect.<br />
void commit();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of OffscreenCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the OffscreenCanvas. The application generates new frames using the RenderingContext obtained from the OffscreenCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the OffscreenCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single OffscreenCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the OffscreenCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToOffscreeen</code>. <code>getContext</code> is used to obtain a rendering context for that OffscreenCanvas, either on the main thread, or on a worker. The application calls <code>commit</code> against that rendering context in order to push frames to the original HTMLCanvasElement. In this rendering model, it is not defined when those frames become visible in the original canvas element.<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed upon the basic form of the API, so it is likely it will be implemented widely and compatibly.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9932OffscreenCanvas2015-04-08T21:30:26Z<p>Kenrussell: Added Google Docs use case. Commented that CanvasRenderingContext2D already has a commit method. Added and documented ImageBitmap.close().</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
* (From the Google Docs team): need to be able to layout and render text from a worker using CanvasRenderingContext2D and display those results on the main thread.<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] mostly addresses these two use cases, but some implementers objected to the mechanism for displaying the rendering results in image elements. The specific objection was that image elements already have complex internal state (for example, the management of the image's "loaded" state), and this would make it more complex. It also did not precisely address the use case of producing new frames both on the main thread and in workers.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface OffscreenCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// OffscreenCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the OffscreenCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the OffscreenCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
OffscreenCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
// It's crucial that there be a way to explicitly dispose of ImageBitmaps<br />
// since they refer to potentially large graphics resources. Some uses<br />
// of this API proposal will result in repeated allocations of ImageBitmaps,<br />
// and garbage collection will not reliably reclaim them quickly enough. <br />
// Here we reuse close(), which also exists on another Transferable type,<br />
// MessagePort. Potentially, all Transferable types should inherit from a<br />
// new interface type "Closeable". <br />
partial interface ImageBitmap {<br />
// Dispose of all graphical resources associated with this ImageBitmap.<br />
void close(); <br />
};<br />
<br />
partial interface HTMLCanvasElement {<br />
OffscreenCanvas transferControlToOffscreen();<br />
};<br />
<br />
// Note that CanvasRenderingContext2D already has a commit() method<br />
// from the CanvasProxy spec which this proposal obsoletes.<br />
partial interface CanvasRenderingContext2D {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
};<br />
<br />
partial interface WebGLRenderingContextBase {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
<br />
// If this context is associated with an OffscreenCanvas that was<br />
// created by HTMLCanvasElement's transferControlToOffscreen method,<br />
// causes this context's current rendering results to be pushed<br />
// to that canvas element. This has the same effect as returning<br />
// control to the main loop in a single-threaded application. Otherwise,<br />
// this call has no effect.<br />
void commit();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of OffscreenCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the OffscreenCanvas. The application generates new frames using the RenderingContext obtained from the OffscreenCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the OffscreenCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single OffscreenCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the OffscreenCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToOffscreeen</code>. <code>getContext</code> is used to obtain a rendering context for that OffscreenCanvas, either on the main thread, or on a worker. The application calls <code>commit</code> against that rendering context in order to push frames to the original HTMLCanvasElement. In this rendering model, it is not defined when those frames become visible in the original canvas element.<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed upon the basic form of the API, so it is likely it will be implemented widely and compatibly.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9931OffscreenCanvas2015-04-08T21:06:46Z<p>Kenrussell: Added details on objection to using images to display offscreen canvases' rendering results</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] mostly addresses these two use cases, but some implementers objected to the mechanism for displaying the rendering results in image elements. The specific objection was that image elements already have complex internal state (for example, the management of the image's "loaded" state), and this would make it more complex. It also did not precisely address the use case of producing new frames both on the main thread and in workers.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface OffscreenCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// OffscreenCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the OffscreenCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the OffscreenCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
OffscreenCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
OffscreenCanvas transferControlToOffscreen();<br />
};<br />
<br />
partial interface CanvasRenderingContext2D {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
};<br />
<br />
partial interface WebGLRenderingContextBase {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
<br />
// If this context is associated with an OffscreenCanvas that was<br />
// created by HTMLCanvasElement's transferControlToOffscreen method,<br />
// causes this context's current rendering results to be pushed<br />
// to that canvas element. This has the same effect as returning<br />
// control to the main loop in a single-threaded application. Otherwise,<br />
// this call has no effect.<br />
void commit();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of OffscreenCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the OffscreenCanvas. The application generates new frames using the RenderingContext obtained from the OffscreenCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the OffscreenCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single OffscreenCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the OffscreenCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToOffscreeen</code>. <code>getContext</code> is used to obtain a rendering context for that OffscreenCanvas, either on the main thread, or on a worker. The application calls <code>commit</code> against that rendering context in order to push frames to the original HTMLCanvasElement. In this rendering model, it is not defined when those frames become visible in the original canvas element.<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed upon the basic form of the API, so it is likely it will be implemented widely and compatibly.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9930OffscreenCanvas2015-04-07T23:25:13Z<p>Kenrussell: </p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] mostly addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings. It also did not precisely address the use case of producing new frames both on the main thread and in workers.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface OffscreenCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// OffscreenCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the OffscreenCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the OffscreenCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
OffscreenCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
OffscreenCanvas transferControlToOffscreen();<br />
};<br />
<br />
partial interface CanvasRenderingContext2D {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
};<br />
<br />
partial interface WebGLRenderingContextBase {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
<br />
// If this context is associated with an OffscreenCanvas that was<br />
// created by HTMLCanvasElement's transferControlToOffscreen method,<br />
// causes this context's current rendering results to be pushed<br />
// to that canvas element. This has the same effect as returning<br />
// control to the main loop in a single-threaded application. Otherwise,<br />
// this call has no effect.<br />
void commit();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of OffscreenCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the OffscreenCanvas. The application generates new frames using the RenderingContext obtained from the OffscreenCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the OffscreenCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single OffscreenCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the OffscreenCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToOffscreeen</code>. <code>getContext</code> is used to obtain a rendering context for that OffscreenCanvas, either on the main thread, or on a worker. The application calls <code>commit</code> against that rendering context in order to push frames to the original HTMLCanvasElement. In this rendering model, it is not defined when those frames become visible in the original canvas element.<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed upon the basic form of the API, so it is likely it will be implemented widely and compatibly.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9929OffscreenCanvas2015-04-07T23:24:56Z<p>Kenrussell: /* Web IDL */ Use union types for "canvas" back-references in CanvasRenderingContext2D and WebGLRenderingContextBase</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] mostly addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings. It also did not precisely address the use case of producing new frames both on the main thread and in workers.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface OffscreenCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// OffscreenCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the OffscreenCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the OffscreenCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
OffscreenCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
OffscreenCanvas transferControlToOffscreen();<br />
};<br />
<br />
partial interface CanvasRenderingContext2D {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
};<br />
<br />
partial interface WebGLRenderingContextBase {<br />
// back-reference to the canvas<br />
readonly attribute (HTMLCanvasElement or OffscreenCanvas) canvas;<br />
<br />
// If this context is associated with an OffscreenCanvas that was<br />
// created by HTMLCanvasElement's transferControlToOffscreen method,<br />
// causes this context's current rendering results to be pushed<br />
// to that canvas element. This has the same effect as returning<br />
// control to the main loop in a single-threaded application. Otherwise,<br />
// this call has no effect.<br />
void commit();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of OffscreenCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the OffscreenCanvas. The application generates new frames using the RenderingContext obtained from the OffscreenCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the OffscreenCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single OffscreenCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the OffscreenCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToOffscreeen</code>. <code>getContext</code> is used to obtain a rendering context for that OffscreenCanvas, either on the main thread, or on a worker. The application calls <code>commit</code> against that rendering context in order to push frames to the original HTMLCanvasElement. In this rendering model, it is not defined when those frames become visible in the original canvas element.<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed upon the basic form of the API, so it is likely it will be implemented widely and compatibly.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9928OffscreenCanvas2015-04-07T23:21:33Z<p>Kenrussell: Renamed WorkerCanvas to OffscreenCanvas on feedback from annevk@</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] mostly addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings. It also did not precisely address the use case of producing new frames both on the main thread and in workers.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface OffscreenCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// OffscreenCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the OffscreenCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the OffscreenCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
OffscreenCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
OffscreenCanvas transferControlToOffscreen();<br />
};<br />
<br />
partial interface WebGLRenderingContextBase {<br />
// If this context is associated with an OffscreenCanvas that was<br />
// created by HTMLCanvasElement's transferControlToOffscreen method,<br />
// causes this context's current rendering results to be pushed<br />
// to that canvas element. This has the same effect as returning<br />
// control to the main loop in a single-threaded application. Otherwise,<br />
// this call has no effect.<br />
void commit();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of OffscreenCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the OffscreenCanvas. The application generates new frames using the RenderingContext obtained from the OffscreenCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the OffscreenCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single OffscreenCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the OffscreenCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToOffscreeen</code>. <code>getContext</code> is used to obtain a rendering context for that OffscreenCanvas, either on the main thread, or on a worker. The application calls <code>commit</code> against that rendering context in order to push frames to the original HTMLCanvasElement. In this rendering model, it is not defined when those frames become visible in the original canvas element.<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed upon the basic form of the API, so it is likely it will be implemented widely and compatibly.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=WorkerCanvas2&diff=9927WorkerCanvas22015-04-07T23:16:12Z<p>Kenrussell: Kenrussell moved page WorkerCanvas2 to OffscreenCanvas: Per feedback from annevk@, renaming the main type in this proposal.</p>
<hr />
<div>#REDIRECT [[OffscreenCanvas]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9926OffscreenCanvas2015-04-07T23:16:12Z<p>Kenrussell: Kenrussell moved page WorkerCanvas2 to OffscreenCanvas: Per feedback from annevk@, renaming the main type in this proposal.</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// WorkerCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the WorkerCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the WorkerCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
partial interface WebGLRenderingContextBase {<br />
// If this context is associated with a WorkerCanvas that was<br />
// created by HTMLCanvasElement's transferControlToWorker method,<br />
// causes this context's current rendering results to be pushed<br />
// to that canvas element. This has the same effect as returning<br />
// control to the main loop in a single-threaded application.<br />
void commit();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of WorkerCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the WorkerCanvas. The application generates new frames using the RenderingContext obtained from the WorkerCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the WorkerCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single WorkerCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the WorkerCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToWorker</code>. <code>getContext</code> is used to obtain a rendering context for that WorkerCanvas, either on the main thread, or on a worker. The application calls <code>commit</code> against that rendering context in order to push frames to the original HTMLCanvasElement. In this rendering model, it is not defined when those frames become visible in the original canvas element.<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed upon the basic form of the API, so it is likely it will be implemented widely and compatibly.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9918OffscreenCanvas2015-04-07T04:02:04Z<p>Kenrussell: /* Implementation */ Rephrased browser vendors' support</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// WorkerCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the WorkerCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the WorkerCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
partial interface WebGLRenderingContextBase {<br />
// If this context is associated with a WorkerCanvas that was<br />
// created by HTMLCanvasElement's transferControlToWorker method,<br />
// causes this context's current rendering results to be pushed<br />
// to that canvas element. This has the same effect as returning<br />
// control to the main loop in a single-threaded application.<br />
void commit();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of WorkerCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the WorkerCanvas. The application generates new frames using the RenderingContext obtained from the WorkerCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the WorkerCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single WorkerCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the WorkerCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToWorker</code>. <code>getContext</code> is used to obtain a rendering context for that WorkerCanvas, either on the main thread, or on a worker. The application calls <code>commit</code> against that rendering context in order to push frames to the original HTMLCanvasElement. In this rendering model, it is not defined when those frames become visible in the original canvas element.<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed upon the basic form of the API, so it is likely it will be implemented widely and compatibly.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9917OffscreenCanvas2015-04-07T03:58:47Z<p>Kenrussell: /* Limitations */ Proactively defined commit() method in WebGLRenderingContextBase</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// WorkerCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the WorkerCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the WorkerCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
partial interface WebGLRenderingContextBase {<br />
// If this context is associated with a WorkerCanvas that was<br />
// created by HTMLCanvasElement's transferControlToWorker method,<br />
// causes this context's current rendering results to be pushed<br />
// to that canvas element. This has the same effect as returning<br />
// control to the main loop in a single-threaded application.<br />
void commit();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of WorkerCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the WorkerCanvas. The application generates new frames using the RenderingContext obtained from the WorkerCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the WorkerCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single WorkerCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the WorkerCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToWorker</code>. <code>getContext</code> is used to obtain a rendering context for that WorkerCanvas, either on the main thread, or on a worker. The application calls <code>commit</code> against that rendering context in order to push frames to the original HTMLCanvasElement. In this rendering model, it is not defined when those frames become visible in the original canvas element.<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed that the basic form of the API is palatable and implementable.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9916OffscreenCanvas2015-04-07T03:58:31Z<p>Kenrussell: /* Processing Model */ Proactively added commit() method to WebGLRenderingContext</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// WorkerCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the WorkerCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the WorkerCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
partial interface WebGLRenderingContextBase {<br />
// If this context is associated with a WorkerCanvas that was<br />
// created by HTMLCanvasElement's transferControlToWorker method,<br />
// causes this context's current rendering results to be pushed<br />
// to that canvas element. This has the same effect as returning<br />
// control to the main loop in a single-threaded application.<br />
void commit();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of WorkerCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the WorkerCanvas. The application generates new frames using the RenderingContext obtained from the WorkerCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the WorkerCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single WorkerCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the WorkerCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToWorker</code>. <code>getContext</code> is used to obtain a rendering context for that WorkerCanvas, either on the main thread, or on a worker. The application calls <code>commit</code> against that rendering context in order to push frames to the original HTMLCanvasElement. In this rendering model, it is not defined when those frames become visible in the original canvas element.<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
<br />
* The mechanism by which frames are committed in the asynchronous processing model must be defined.<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed that the basic form of the API is palatable and implementable.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9915OffscreenCanvas2015-04-07T03:54:49Z<p>Kenrussell: /* Web IDL */ Proactively added commit method to WebGLRenderingContextBase</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// WorkerCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the WorkerCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the WorkerCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
partial interface WebGLRenderingContextBase {<br />
// If this context is associated with a WorkerCanvas that was<br />
// created by HTMLCanvasElement's transferControlToWorker method,<br />
// causes this context's current rendering results to be pushed<br />
// to that canvas element. This has the same effect as returning<br />
// control to the main loop in a single-threaded application.<br />
void commit();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of WorkerCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the WorkerCanvas. The application generates new frames using the RenderingContext obtained from the WorkerCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the WorkerCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single WorkerCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the WorkerCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToWorker</code>. At that point, all rendering operations performed into the WorkerCanvas are displayed in the original HTMLCanvasElement at an indeterminate future time. ''(TODO: define when frames are captured from the WorkerCanvas. Introduce a <code>commit</code> method onto WebGLRenderingContext as well? Define that frames are captured when control returns to the main loop of the thread driving the WorkerCanvas?)''<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
<br />
* The mechanism by which frames are committed in the asynchronous processing model must be defined.<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed that the basic form of the API is palatable and implementable.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9914OffscreenCanvas2015-04-07T03:50:05Z<p>Kenrussell: /* Web IDL */ updated transferToImageBitmap origin-clean docs</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// WorkerCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the WorkerCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag like other<br />
// CanvasImageSource types do, such as HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the WorkerCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of WorkerCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the WorkerCanvas. The application generates new frames using the RenderingContext obtained from the WorkerCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the WorkerCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single WorkerCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the WorkerCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToWorker</code>. At that point, all rendering operations performed into the WorkerCanvas are displayed in the original HTMLCanvasElement at an indeterminate future time. ''(TODO: define when frames are captured from the WorkerCanvas. Introduce a <code>commit</code> method onto WebGLRenderingContext as well? Define that frames are captured when control returns to the main loop of the thread driving the WorkerCanvas?)''<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
<br />
* The mechanism by which frames are committed in the asynchronous processing model must be defined.<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed that the basic form of the API is palatable and implementable.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9913OffscreenCanvas2015-04-07T03:47:41Z<p>Kenrussell: /* My Solution */ Renamed to "This Solution" and filled out all sections</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// WorkerCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the WorkerCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag similarly to other<br />
// CanvasImageSource types, like HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the WorkerCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== This Solution ===<br />
<br />
This proposed API can be used in several ways to satisfy the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It supports using a single WebGLRenderingContext or Canvas2DRenderingContext to efficiently render into multiple regions on the web page.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
* It supports asynchronous encoding of WorkerCanvases' rendering results into Blobs which can be consumed by various other web platform APIs.<br />
<br />
==== Processing Model ====<br />
<br />
This proposal introduces two primary processing models. The first involves ''synchronous'' display of new frames produced by the WorkerCanvas. The application generates new frames using the RenderingContext obtained from the WorkerCanvas. When the application is finished rendering each new frame, it calls transferToImageBitmap to "tear off" the most recently rendered image from the WorkerCanvas -- like a Post-It note. The resulting ImageBitmap can then be used in any API receiving that data type; notably, it can be displayed in a second canvas without introducing a copy. An ImageBitmapRenderingContext is obtained from the second canvas by calling <code>getContext('bitmaprenderer')</code>. Each frame is displayed in the second canvas using the <code>transferImageBitmap</code> method on this rendering context. Note that the threads producing and consuming the frames may be the same, or they may be different. Note also that a single WorkerCanvas may transfer frames into an arbitrary number of other ImageBitmapRenderingContexts.<br />
<br />
The second processing model involves ''asynchronous'' display of new frames produced by the WorkerCanvas. The main thread instantiates an HTMLCanvasElement and calls <code>transferControlToWorker</code>. At that point, all rendering operations performed into the WorkerCanvas are displayed in the original HTMLCanvasElement at an indeterminate future time. ''(TODO: define when frames are captured from the WorkerCanvas. Introduce a <code>commit</code> method onto WebGLRenderingContext as well? Define that frames are captured when control returns to the main loop of the thread driving the WorkerCanvas?)''<br />
<br />
==== Limitations ==== <br />
<br />
* A known good way to drive an animation loop from a worker is needed. requestAnimationFrame or a similar API needs to be defined on worker threads.<br />
<br />
* The mechanism by which frames are committed in the asynchronous processing model must be defined.<br />
<br />
==== Implementation ==== <br />
<br />
This proposal has been vetted by developers of Apple's Safari, Google's Chrome, Microsoft's Internet Explorer, and Mozilla's Firefox browsers. All vendors agreed that the basic form of the API is palatable and implementable.<br />
<br />
==== Adoption ==== <br />
<br />
Web page authors have demanded increased parallelism support from the web platform for multiple years. If support for multithreaded rendering is added, it is likely it will be rapidly adopted.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9912OffscreenCanvas2015-04-07T03:22:02Z<p>Kenrussell: /* Web IDL */</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
<br />
// WorkerCanvas, like HTMLCanvasElement, maintains an origin-clean flag.<br />
// ImageBitmaps created by calling this method also have an<br />
// origin-clean flag which is set to the value of the WorkerCanvas's<br />
// flag at the time of their construction. Uses of the ImageBitmap<br />
// in other APIs, such as CanvasRenderingContext2D or<br />
// WebGLRenderingContext, propagate this flag similarly to other<br />
// CanvasImageSource types, like HTMLImageElement.<br />
ImageBitmap transferToImageBitmap();<br />
<br />
// Throws a SecurityError if the WorkerCanvas's origin-clean flag<br />
// is set to false.<br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== My Solution ===<br />
<br />
The API described above can be used in several ways to satisfy all of the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
==== Processing Model ====<br />
:''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
:''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
:''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
:''Reasons why page authors would use this solution.''<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9911OffscreenCanvas2015-04-07T02:58:09Z<p>Kenrussell: /* My Solution */ Description of how the abovementioned API satisfies the use cases</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
ImageBitmap transferToImageBitmap();<br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== My Solution ===<br />
<br />
The API described above can be used in several ways to satisfy all of the use cases described above:<br />
<br />
* It supports zero-copy transfer of canvases' rendering results between threads, for example from a worker to the main thread. In this model, the main thread controls when to display new frames produced by the worker, so synchronization with other DOM updates is achieved.<br />
<br />
* It supports fully asynchronous rendering by a worker into a canvas displayed on the main thread. This satisfies certain Emscripten developers' full-screen use cases.<br />
<br />
* It introduces ImageBitmapRenderingContext, a new canvas context type whose sole purpose is to efficiently display ImageBitmaps. This supersedes the [[WorkerCanvas]] proposal's use of HTMLImageElement for this purpose.<br />
<br />
==== Processing Model ====<br />
:''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
:''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
:''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
:''Reasons why page authors would use this solution.''<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9910OffscreenCanvas2015-04-07T02:46:04Z<p>Kenrussell: /* Web IDL */ Updated Web IDL with results of WebGL working group discussion</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height),<br />
Exposed=(Window,Worker)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
Promise<Blob> toBlob(optional DOMString type, any... arguments); <br />
ImageBitmap transferToImageBitmap();<br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
// The new ImageBitmapRenderingContext is a canvas rendering context<br />
// which only provides the functionality to replace the canvas's<br />
// contents with the given ImageBitmap. Its context id (the first argument<br />
// to getContext) is "bitmaprenderer".<br />
[Exposed=(Window,Worker)]<br />
interface ImageBitmapRenderingContext {<br />
// Displays the given ImageBitmap in the canvas associated with this<br />
// rendering context. Ownership of the ImageBitmap is transferred to<br />
// the canvas. The caller may not use its reference to the ImageBitmap<br />
// after making this call. (This semantic is crucial to enable prompt<br />
// reclamation of expensive graphics resources, rather than relying on<br />
// garbage collection to do so.)<br />
//<br />
// The ImageBitmap conceptually replaces the canvas's bitmap, but<br />
// it does not change the canvas's intrinsic width or height.<br />
//<br />
// The ImageBitmap, when displayed, is clipped to the rectangle<br />
// defined by the canvas's instrinsic width and height. Pixels that<br />
// would be covered by the canvas's bitmap which are not covered by<br />
// the supplied ImageBitmap are rendered transparent black. Any CSS<br />
// styles affecting the display of the canvas are applied as usual.<br />
void transferImageBitmap(ImageBitmap bitmap);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== My Solution ===<br />
:''Brief description of the solution and of how it address the problem at hand.''<br />
<br />
==== Processing Model ====<br />
:''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
:''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
:''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
:''Reasons why page authors would use this solution.''<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9907OffscreenCanvas2015-04-03T22:20:58Z<p>Kenrussell: /* Requests for this Feature */</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
* Many others<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
void toBlob(FileCallback? _callback, optional DOMString type, any... arguments);<br />
ImageBitmap transferToImageBitmap();<br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
partial interface ImageBitmap {<br />
void transferToImage(HTMLImageElement image);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== My Solution ===<br />
:''Brief description of the solution and of how it address the problem at hand.''<br />
<br />
==== Processing Model ====<br />
:''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
:''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
:''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
:''Reasons why page authors would use this solution.''<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9906OffscreenCanvas2015-04-03T22:20:47Z<p>Kenrussell: /* Requests for this Feature */</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
See the abovementioned use cases:<br />
<br />
* Google's Maps team<br />
* Emscripten users such as Epic Games and Unity<br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
void toBlob(FileCallback? _callback, optional DOMString type, any... arguments);<br />
ImageBitmap transferToImageBitmap();<br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
partial interface ImageBitmap {<br />
void transferToImage(HTMLImageElement image);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== My Solution ===<br />
:''Brief description of the solution and of how it address the problem at hand.''<br />
<br />
==== Processing Model ====<br />
:''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
:''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
:''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
:''Reasons why page authors would use this solution.''<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9905OffscreenCanvas2015-04-03T22:18:16Z<p>Kenrussell: /* Benefits */</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
Making canvas rendering contexts available to workers will increase parallelism in web applications, leading to increased performance on multi-core systems.<br />
<br />
=== Requests for this Feature ===<br />
<br />
* <cite>[http://example.com Source]</cite> <blockquote><p>I would like this feature ...</p></blockquote><br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
void toBlob(FileCallback? _callback, optional DOMString type, any... arguments);<br />
ImageBitmap transferToImageBitmap();<br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
partial interface ImageBitmap {<br />
void transferToImage(HTMLImageElement image);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== My Solution ===<br />
:''Brief description of the solution and of how it address the problem at hand.''<br />
<br />
==== Processing Model ====<br />
:''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
:''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
:''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
:''Reasons why page authors would use this solution.''<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9904OffscreenCanvas2015-04-03T22:16:46Z<p>Kenrussell: /* Current Usage and Workarounds */ Linked to Alon Zakai's blog post on WebGL in workers</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
[https://blog.mozilla.org/research/2014/07/22/webgl-in-web-workers-today-and-faster-than-expected/ WebGL in Web Workers] details some work attempted in the Emscripten toolchain to address the lack of WebGL in workers. Due to the high volume of calls and large amount of data that is transferred to the graphics card in a typical high-end WebGL application, this approach is not sustainable. It's necessary for workers to be able to call the WebGL API directly, and present those results to the screen in a manner that does not introduce any copies of the rendering results.<br />
<br />
=== Benefits ===<br />
<br />
''Explanation of how and why new markup would be useful.''<br />
<br />
=== Requests for this Feature ===<br />
<br />
* <cite>[http://example.com Source]</cite> <blockquote><p>I would like this feature ...</p></blockquote><br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
void toBlob(FileCallback? _callback, optional DOMString type, any... arguments);<br />
ImageBitmap transferToImageBitmap();<br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
partial interface ImageBitmap {<br />
void transferToImage(HTMLImageElement image);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== My Solution ===<br />
:''Brief description of the solution and of how it address the problem at hand.''<br />
<br />
==== Processing Model ====<br />
:''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
:''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
:''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
:''Reasons why page authors would use this solution.''<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9903OffscreenCanvas2015-04-03T22:14:01Z<p>Kenrussell: /* Current Limitations */</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy] does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings.<br />
<br />
=== Current Usage and Workarounds ===<br />
''Some evidence that this feature is desperately needed on the web. You may provide a separate examples page for listing these.''<br />
<br />
=== Benefits ===<br />
<br />
''Explanation of how and why new markup would be useful.''<br />
<br />
=== Requests for this Feature ===<br />
<br />
* <cite>[http://example.com Source]</cite> <blockquote><p>I would like this feature ...</p></blockquote><br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
void toBlob(FileCallback? _callback, optional DOMString type, any... arguments);<br />
ImageBitmap transferToImageBitmap();<br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
partial interface ImageBitmap {<br />
void transferToImage(HTMLImageElement image);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== My Solution ===<br />
:''Brief description of the solution and of how it address the problem at hand.''<br />
<br />
==== Processing Model ====<br />
:''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
:''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
:''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
:''Reasons why page authors would use this solution.''<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9902OffscreenCanvas2015-04-03T22:13:45Z<p>Kenrussell: /* Current Limitations */ Detailed the reasons for creating this proposal</p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
* (From adopters of the Push API): need to be able to dynamically create images to use as notification icons, such as compositing avatars, or adding an unread count<br />
<br />
=== Current Limitations ===<br />
<br />
* [https://html.spec.whatwg.org/multipage/scripting.html#proxying-canvases-to-workers CanvasProxy]: does not provide sufficient control to allow synchronization between workers' rendering and DOM updates on the main thread. Keeping this rendering in sync is a requirement from Google's Maps team.<br />
* [[CanvasInWorkers]] does not allow a worker to render directly into a canvas on the main thread without running code on the main thread. Allowing completely unsynchronized rendering is a requirement from Mozilla and users of Emscripten such as Epic Games and Unity, in which the desire is to execute all of the game's rendering on a worker thread.<br />
* [[WorkerCanvas]] addresses these two use cases, but the specific mechanism for displaying the rendering results (in image elements) was not palatable to some implementers in recent face-to-face meetings.<br />
<br />
=== Current Usage and Workarounds ===<br />
''Some evidence that this feature is desperately needed on the web. You may provide a separate examples page for listing these.''<br />
<br />
=== Benefits ===<br />
<br />
''Explanation of how and why new markup would be useful.''<br />
<br />
=== Requests for this Feature ===<br />
<br />
* <cite>[http://example.com Source]</cite> <blockquote><p>I would like this feature ...</p></blockquote><br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
void toBlob(FileCallback? _callback, optional DOMString type, any... arguments);<br />
ImageBitmap transferToImageBitmap();<br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
partial interface ImageBitmap {<br />
void transferToImage(HTMLImageElement image);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== My Solution ===<br />
:''Brief description of the solution and of how it address the problem at hand.''<br />
<br />
==== Processing Model ====<br />
:''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
:''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
:''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
:''Reasons why page authors would use this solution.''<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9799OffscreenCanvas2015-01-27T20:36:35Z<p>Kenrussell: </p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
<br />
=== Current Limitations ===<br />
''Explanation of why the current markup is insufficient.''<br />
<br />
=== Current Usage and Workarounds ===<br />
''Some evidence that this feature is desperately needed on the web. You may provide a separate examples page for listing these.''<br />
<br />
=== Benefits ===<br />
<br />
''Explanation of how and why new markup would be useful.''<br />
<br />
=== Requests for this Feature ===<br />
<br />
* <cite>[http://example.com Source]</cite> <blockquote><p>I would like this feature ...</p></blockquote><br />
<br />
== Web IDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
void toBlob(FileCallback? _callback, optional DOMString type, any... arguments);<br />
ImageBitmap transferToImageBitmap();<br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
partial interface ImageBitmap {<br />
void transferToImage(HTMLImageElement image);<br />
};<br />
<br />
== Proposed Solutions ==<br />
<br />
=== My Solution ===<br />
:''Brief description of the solution and of how it address the problem at hand.''<br />
<br />
==== Processing Model ====<br />
:''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
:''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
:''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
:''Reasons why page authors would use this solution.''<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9798OffscreenCanvas2015-01-27T20:34:52Z<p>Kenrussell: </p>
<hr />
<div>:''Provides more control over how canvases are rendered. This is a follow-on to the [[WorkerCanvas]] proposal and will be merged once agreement is reached.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
<br />
=== Current Limitations ===<br />
''Explanation of why the current markup is insufficient.''<br />
<br />
=== Current Usage and Workarounds ===<br />
''Some evidence that this feature is desperately needed on the web. You may provide a separate examples page for listing these.''<br />
<br />
=== Benefits ===<br />
<br />
''Explanation of how and why new markup would be useful.''<br />
<br />
=== Requests for this Feature ===<br />
<br />
* <cite>[http://example.com Source]</cite> <blockquote><p>I would like this feature ...</p></blockquote><br />
<br />
== Proposed Solutions ==<br />
<br />
=== My Solution ===<br />
:''Brief description of the solution and of how it address the problem at hand.''<br />
<br />
==== Processing Model ====<br />
:''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
:''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
:''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
:''Reasons why page authors would use this solution.''<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=Category:Proposals&diff=9797Category:Proposals2015-01-27T20:33:36Z<p>Kenrussell: Deleting temporary WorkerCanvas2 link</p>
<hr />
<div>This is a collection of proposals for [[specs]]. Before drafting a proposal, please read [[FAQ#Is_there_a_process_for_adding_new_features_to_a_specification.3F|Is there a process for adding new features to a specification?]]. If you want to add a feature request, start by copying the [[Problem Solving]] template page onto a new page and fill out as much information as you can.<br />
<br />
You do not have to provide detailed answers for everything straight away. The most important information to provide at first is the problem description. Once we have detailed descriptions, use cases and an understanding of the limitations with existing markup, we can then begin to discuss the best way in which to address the problems and work out more of the more technical details.<br />
<br />
[[Category:Spec coordination]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=OffscreenCanvas&diff=9796OffscreenCanvas2015-01-27T20:32:59Z<p>Kenrussell: Creating WorkerCanvas2 page from template</p>
<hr />
<div>This is an empty page that describes what should be documented about a problem and the different solutions proposed to address it. Feel free to copy this template to when creating a new page.<br />
<br />
:''Brief description of the problem to solve.''<br />
<br />
== Use Case Description ==<br />
''Complete description of the use case.''<br />
<br />
=== Current Limitations ===<br />
''Explanation of why the current markup is insufficient.''<br />
<br />
=== Current Usage and Workarounds ===<br />
''Some evidence that this feature is desperately needed on the web. You may provide a separate examples page for listing these.''<br />
<br />
=== Benefits ===<br />
<br />
''Explanation of how and why new markup would be useful.''<br />
<br />
=== Requests for this Feature ===<br />
<br />
* <cite>[http://example.com Source]</cite> <blockquote><p>I would like this feature ...</p></blockquote><br />
<br />
== Proposed Solutions ==<br />
<br />
=== My Solution ===<br />
:''Brief description of the solution and of how it address the problem at hand.''<br />
<br />
==== Processing Model ====<br />
:''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
:''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
:''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
:''Reasons why page authors would use this solution.''<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=Category:Proposals&diff=9795Category:Proposals2015-01-27T20:32:26Z<p>Kenrussell: Temporary link to create a new page</p>
<hr />
<div>This is a collection of proposals for [[specs]]. Before drafting a proposal, please read [[FAQ#Is_there_a_process_for_adding_new_features_to_a_specification.3F|Is there a process for adding new features to a specification?]]. If you want to add a feature request, start by copying the [[Problem Solving]] template page onto a new page and fill out as much information as you can.<br />
<br />
You do not have to provide detailed answers for everything straight away. The most important information to provide at first is the problem description. Once we have detailed descriptions, use cases and an understanding of the limitations with existing markup, we can then begin to discuss the best way in which to address the problems and work out more of the more technical details.<br />
<br />
[[Category:Spec coordination]]<br />
[[WorkerCanvas2]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=WorkerCanvas&diff=9756WorkerCanvas2014-11-05T18:46:19Z<p>Kenrussell: Added use case description</p>
<hr />
<div>:''Provides more control over how canvases are rendered.''<br />
<br />
== Use Case Description ==<br />
<br />
Feedback from web application authors using canvases have shown the need for the following controls:<br />
<br />
* (From ShaderToy, Sketchfab, Verold): need to be able to render to multiple regions on the page efficiently using a single canvas context. 3D model warehouse sites desire to show multiple live interactive models on the page, but creating multiple WebGL contexts per page is too inefficient. A single context should be able to render to multiple regions on the page.<br />
* (From Google Maps): need to be able to render WebGL from a worker, transfer the rendered image to the main thread without making any copy of it, and composite it with other HTML on the page, guaranteeing that the updates are all seen in the same rendered frame.<br />
* (From Mozilla and partners using Emscripten and asm.js): need to be able to render WebGL entirely asynchronously from a worker, displaying the results in a canvas owned by the main thread, without any synchronization with the main thread. In this mode, the entire application runs in the worker. The main thread only receives input events and sends them to the worker for processing.<br />
<br />
== WebIDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
void toBlob(FileCallback? _callback, optional DOMString type, any... arguments);<br />
ImageBitmap transferToImageBitmap();<br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
partial interface ImageBitmap {<br />
void transferToImage(HTMLImageElement image);<br />
};<br />
<br />
== Spec changes ==<br />
<br />
Transferring of ImageBitmaps has to be defined. It should neuter the ImageBitmap in the sending thread. Neutering sets the ImageBitmap's width and height to 0.<br />
<br />
HTMLCanvasElement.transferControlToWorker behaves like transferControlToProxy in the current WHATWG spec. WorkerCanvas is Transferable, but transfer fails if transferred other than from the main thread to a worker. All its methods throw if not called on a worker, or if it's neutered.<br />
<br />
ImageBitmap.transferToImage removes the image element's "src" attribute and makes the image display the contents of the ImageBitmap (until the next transferToImage to that image, or until the image's "src" attribute is set). The ImageBitmap is neutered.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=ImageBitmap_Constructor_taking_URL&diff=9755ImageBitmap Constructor taking URL2014-11-05T18:31:21Z<p>Kenrussell: </p>
<hr />
<div>:''Allows creating an ImageBitmap from the URL of an image.''<br />
<br />
== Use Case Description ==<br />
<br />
ImageBitmap provides a useful abstraction for image data that can be drawn to a 2D canvas context or uploaded to a texture in a WebGL canvas context. It attempts to move all of the image decoding and preparation work out of the critical path, so that by the time an ImageBitmap is constructed, the data it contains is in exactly the right format for rendering. [[ImageBitmap Options]] provides control over the image decoding process so that the resulting bitmap is in exactly the desired form. For example, it can be used to disable the user agent's premultiplication of the alpha channel into the color channels.<br />
<br />
Currently an ImageBitmap can only be constructed from a fully-loaded media resource such as an HTMLImageElement. Unfortunately, most user agents decode the image data before calling HTMLImageElement's onload handler, and it is only legal to construct an ImageBitmap after the HTMLImageElement's onload handler is called. If the ImageBitmap options would specify different decoding parameters than the browser's default, the image must be decoded twice, which is inefficient. Further, some user agents discard the compressed image data from HTMLImageElements before firing the onload handler.<br />
<br />
This proposal adds URL to the [[https://html.spec.whatwg.org/multipage/webappapis.html#imagebitmapsource ImageBitmapSource union type]], which is the first argument to [[https://html.spec.whatwg.org/multipage/webappapis.html#dom-createimagebitmap createImageBitmap]]. The image is downloaded from that URL, decoded, and processed as part of the construction of the ImageBitmap, taking into account the ImageBitmap options. If any errors occur during the downloading or decoding process, including if the image is not single-origin after honoring the crossOrigin attribute of the ImageBitmap options, the Promise returned from createImageBitmap is rejected. If image downloading and decoding succeeds, then the promise is resolved with the new ImageBitmap as its value.<br />
<br />
=== Current Limitations ===<br />
<br />
The intent of ImageBitmap is to prepare the image data in the desired form for drawing or texture uploading before resolving the promise returned from createImageBitmap. Unfortunately, most user agents decode the image before calling the onload handler, and it's only legal to pass an HTMLImageElement to createImageBitmap once the image's onload handler has been called. This means that at a minimum, in order to respect the [[ImageBitmap Options]] argument to createImageBitmap, two decodes of the image must be done, which is inefficient.<br />
<br />
Further, some user agents discard the compressed image data for an HTMLImageElement after decoding it, and before calling the onload handler. Honoring [[ImageBitmap Options]] in these user agents is not possible without rearchitecting their image decoding subsystem.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
Most WebGL applications encode normal maps or other non-image data into images, and upload these images to textures using WebGL-specific options that skip the alpha premultiplication or color correction the user agent may perform. Some WebGL implementers have pointed out that the current mechanisms usually result in a synchronous image re-decode. Switching to use ImageBitmap to provide control over these image decoding options will be more efficient, and allow the WebGL mechanisms to be removed in the next version of that specification.<br />
<br />
=== Benefits ===<br />
<br />
Provides the most efficient image decoding path for canvas applications requiring control over image decoding.<br />
<br />
<br />
=== Requests for this Feature ===<br />
<br />
* Microsoft has requested this feature in order to provide an efficient path for uploading WebGL images to textures.<br />
* Other WebGL implementers (Apple, Google, Mozilla) support adding this feature and will implement it.<br />
* The WebGL 2.0 specification under development will remove the WebGL-specific image decoding options and rely on this mechanism instead.<br />
<br />
== Proposed Solutions ==<br />
<br />
==== Processing Model ====<br />
<br />
Compared to constructing an ImageBitmap from an HTMLImageElement, this alternative construction path downloads and decodes the image data as part of resolving the promise returned by createImageBitmap.<br />
<br />
==== Limitations ==== <br />
<br />
Only supports single images. There's no provision for downloading other types of media from URLs.<br />
<br />
==== Implementation ==== <br />
<br />
Browser vendors would trigger their usual image downloading and decoding code paths upon creating an ImageBitmap from a URL. Browsers would implement this new API in order to provide more efficient image handling for 2D and WebGL canvas applications.<br />
<br />
==== Adoption ==== <br />
<br />
WebGL application authors in particular would readily update their code in order to use this new API because it will provide more efficient uploading of images into textures. It's straightforward to add conditional code which prefers this new, more efficient, mechanism, and which uses the older texture upload paths if it's not available.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=ImageBitmap_Constructor_taking_URL&diff=9754ImageBitmap Constructor taking URL2014-11-05T18:30:30Z<p>Kenrussell: </p>
<hr />
<div>:''Allows creating an ImageBitmap from the URL of an image.''<br />
<br />
== Use Case Description ==<br />
<br />
ImageBitmap provides a useful abstraction for image data that can be drawn to a 2D canvas context or uploaded to a texture in a WebGL canvas context. It attempts to move all of the image decoding and preparation work out of the critical path, so that by the time an ImageBitmap is constructed, the data it contains is in exactly the right format for rendering. [[ImageBitmap Options]] provides control over the image decoding process so that the resulting bitmap is in exactly the desired form. For example, it can be used to disable the user agent's premultiplication of the alpha channel into the color channels.<br />
<br />
Currently an ImageBitmap can only be constructed from a fully-loaded media resource such as an HTMLImageElement. Unfortunately, most user agents decode the image data before calling HTMLImageElement's onload handler, and it is only legal to construct an ImageBitmap after the HTMLImageElement's onload handler is called. If the ImageBitmap options would specify different decoding parameters than the browser's default, the image must be decoded twice, which is inefficient. Further, some user agents discard the compressed image data from HTMLImageElements before firing the onload handler.<br />
<br />
This proposal adds URL to the [[https://html.spec.whatwg.org/multipage/webappapis.html#imagebitmapsource ImageBitmapSource union type]], which is the first argument to [[https://html.spec.whatwg.org/multipage/webappapis.html#dom-createimagebitmap createImageBitmap]]. The image is downloaded from that URL, decoded, and processed as part of the construction of the ImageBitmap, taking into account the ImageBitmap options. If any errors occur during the downloading or decoding process, including if the image is not single-origin after honoring the crossOrigin attribute of the ImageBitmap options, the Promise returned from createImageBitmap is rejected. If image downloading and decoding succeeds, then the promise is resolved with the new ImageBitmap as its value.<br />
<br />
=== Current Limitations ===<br />
<br />
The intent of ImageBitmap is to prepare the image data in the desired form for drawing or texture uploading before resolving the promise returned from createImageBitmap. Unfortunately, most user agents decode the image before calling the onload handler, and it's only legal to pass an HTMLImageElement to createImageBitmap once the image's onload handler has been called. This means that at a minimum, in order to respect the [[ImageBitmap Options]] argument to createImageBitmap, two decodes of the image must be done, which is inefficient.<br />
<br />
Further, some user agents discard the compressed image data for an HTMLImageElement after decoding it, and before calling the onload handler. Honoring [[ImageBitmap Options]] in these user agents is not possible without rearchitecting their image decoding subsystem.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
Most WebGL applications encode normal maps or other non-image data into images, and upload these images to textures using WebGL-specific options that skip the alpha premultiplication or color correction the user agent may perform. Some WebGL implementers have pointed out that the current WebGL mechanisms are inefficient, and usually result in a synchronous image re-decode. Switching to use ImageBitmap to provide control over these image decoding options will be more efficient, and allow the WebGL mechanisms to be removed in the next version of the WebGL specification.<br />
<br />
=== Benefits ===<br />
<br />
Provides the most efficient image decoding path for canvas applications requiring control over image decoding.<br />
<br />
<br />
=== Requests for this Feature ===<br />
<br />
* Microsoft has requested this feature in order to provide an efficient path for uploading WebGL images to textures.<br />
* Other WebGL implementers (Apple, Google, Mozilla) support adding this feature and will implement it.<br />
* The WebGL 2.0 specification under development will remove the WebGL-specific image decoding options and rely on this mechanism instead.<br />
<br />
== Proposed Solutions ==<br />
<br />
==== Processing Model ====<br />
<br />
Compared to constructing an ImageBitmap from an HTMLImageElement, this alternative construction path downloads and decodes the image data as part of resolving the promise returned by createImageBitmap.<br />
<br />
==== Limitations ==== <br />
<br />
Only supports single images. There's no provision for downloading other types of media from URLs.<br />
<br />
==== Implementation ==== <br />
<br />
Browser vendors would trigger their usual image downloading and decoding code paths upon creating an ImageBitmap from a URL. Browsers would implement this new API in order to provide more efficient image handling for 2D and WebGL canvas applications.<br />
<br />
==== Adoption ==== <br />
<br />
WebGL application authors in particular would readily update their code in order to use this new API because it will provide more efficient uploading of images into textures. It's straightforward to add conditional code which prefers this new, more efficient, mechanism, and which uses the older texture upload paths if it's not available.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=ImageBitmap_Constructor_taking_URL&diff=9753ImageBitmap Constructor taking URL2014-11-05T18:26:58Z<p>Kenrussell: </p>
<hr />
<div>:''Allows creating an ImageBitmap from the URL of an image.''<br />
<br />
== Use Case Description ==<br />
<br />
ImageBitmap provides a useful abstraction for image data that can be drawn to a 2D canvas context or uploaded to a texture in a WebGL canvas context. It attempts to move all of the image decoding and preparation work out of the critical path, so that by the time an ImageBitmap is constructed, the data it contains is in exactly the right format for rendering. [[ImageBitmap Options]] provides control over the image decoding process so that the resulting bitmap is in exactly the desired form. For example, it can be used to disable the user agent's premultiplication of the alpha channel into the color channels.<br />
<br />
Currently an ImageBitmap can only be constructed from a fully-loaded media resource such as an HTMLImageElement. Unfortunately, some user agents discard the compressed image data from HTMLImageElements before firing the onload handler. (An ImageBitmap can only be constructed from an HTMLImageElement after the image element's onload handler has fired.) These user agents perform all image decompression before firing the onload handler. The only way that the ImageBitmap options can be taken into account is during the decoding process. Transformations done during decoding, like premultiplying the alpha channel into the color channels, are not reversible.<br />
<br />
This proposal adds URL to the [[https://html.spec.whatwg.org/multipage/webappapis.html#imagebitmapsource ImageBitmapSource union type]], which is the first argument to [[https://html.spec.whatwg.org/multipage/webappapis.html#dom-createimagebitmap createImageBitmap]]. The image is downloaded from that URL, decoded, and processed as part of the construction of the ImageBitmap, taking into account the ImageBitmap options. If any errors occur during the downloading or decoding process, including if the image is not single-origin after honoring the crossOrigin attribute of the ImageBitmap options, the Promise returned from createImageBitmap is rejected. If image downloading and decoding succeeds, then the promise is resolved with the new ImageBitmap as its value.<br />
<br />
=== Current Limitations ===<br />
<br />
The intent of ImageBitmap is to prepare the image data in the desired form for drawing or texture uploading before resolving the promise returned from createImageBitmap. Unfortunately, most user agents decode the image before calling the onload handler, and it's only legal to pass an HTMLImageElement to createImageBitmap once the image's onload handler has been called. This means that at a minimum, in order to respect the [[ImageBitmap Options]] argument to createImageBitmap, two decodes of the image must be done, which is inefficient.<br />
<br />
Further, some user agents discard the compressed image data for an HTMLImageElement after decoding it, and before calling the onload handler. Honoring [[ImageBitmap Options]] in these user agents is not possible without rearchitecting their image decoding subsystem.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
Most WebGL applications encode normal maps or other non-image data into images, and upload these images to textures using WebGL-specific options that skip the alpha premultiplication or color correction the user agent may perform. Some WebGL implementers have pointed out that the current WebGL mechanisms are inefficient, and usually result in a synchronous image re-decode. Switching to use ImageBitmap to provide control over these image decoding options will be more efficient, and allow the WebGL mechanisms to be removed in the next version of the WebGL specification.<br />
<br />
=== Benefits ===<br />
<br />
Provides the most efficient image decoding path for canvas applications requiring control over image decoding.<br />
<br />
<br />
=== Requests for this Feature ===<br />
<br />
* Microsoft has requested this feature in order to provide an efficient path for uploading WebGL images to textures.<br />
* Other WebGL implementers (Apple, Google, Mozilla) support adding this feature and will implement it.<br />
* The WebGL 2.0 specification under development will remove the WebGL-specific image decoding options and rely on this mechanism instead.<br />
<br />
== Proposed Solutions ==<br />
<br />
==== Processing Model ====<br />
<br />
Compared to constructing an ImageBitmap from an HTMLImageElement, this alternative construction path downloads and decodes the image data as part of resolving the promise returned by createImageBitmap.<br />
<br />
==== Limitations ==== <br />
<br />
Only supports single images. There's no provision for downloading other types of media from URLs.<br />
<br />
==== Implementation ==== <br />
<br />
Browser vendors would trigger their usual image downloading and decoding code paths upon creating an ImageBitmap from a URL. Browsers would implement this new API in order to provide more efficient image handling for 2D and WebGL canvas applications.<br />
<br />
==== Adoption ==== <br />
<br />
WebGL application authors in particular would readily update their code in order to use this new API because it will provide more efficient uploading of images into textures. It's straightforward to add conditional code which prefers this new, more efficient, mechanism, and which uses the older texture upload paths if it's not available.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=ImageBitmap_Constructor_taking_URL&diff=9752ImageBitmap Constructor taking URL2014-11-05T18:23:43Z<p>Kenrussell: /* Requests for this Feature */</p>
<hr />
<div>:''Allows creating an ImageBitmap from the URL of an image.''<br />
<br />
== Use Case Description ==<br />
<br />
ImageBitmap provides a useful abstraction for image data that can be drawn to a 2D canvas context or uploaded to a texture in a WebGL canvas context. It attempts to move all of the image decoding and preparation work out of the critical path, so that by the time an ImageBitmap is constructed, the data it contains is in exactly the right format for rendering. [[ImageBitmap Options]] provides control over the image decoding process so that the resulting bitmap is in exactly the desired form. For example, it can be used to disable the user agent's premultiplication of the alpha channel into the color channels.<br />
<br />
Currently an ImageBitmap can only be constructed from a fully-loaded media resource such as an HTMLImageElement. Unfortunately, some user agents discard the compressed image data from HTMLImageElements before firing the onload handler. (An ImageBitmap can only be constructed from an HTMLImageElement after the image element's onload handler has fired.) These user agents perform all of the image decompression before firing the onload handler. The only way that the ImageBitmap options can be taken into account is during the decoding process. Transformations done during decoding, like premultiplying the alpha channel into the color channels, are not reversible.<br />
<br />
This proposal adds URL to the ImageBitmapSource union type, which is the first argument to createImageBitmap. The image is downloaded from that URL, decoded, and processed as part of the construction of the ImageBitmap, taking into account the ImageBitmap options. If any errors occur during the downloading or decoding process, including if the image is not single-origin after honoring the crossOrigin attribute of the ImageBitmap options, the Promise returned from createImageBitmap is rejected. If image downloading and decoding succeeds, then the promise is resolved with the new ImageBitmap as its value.<br />
<br />
=== Current Limitations ===<br />
<br />
The intent of ImageBitmap is to prepare the image data in the desired form for drawing or texture uploading before resolving the promise returned from createImageBitmap. Unfortunately, most user agents decode the image before calling the onload handler, and it's only legal to pass an HTMLImageElement to createImageBitmap once the image's onload handler has been called. This means that at a minimum, in order to respect the [[ImageBitmap Options]] argument to createImageBitmap, two decodes of the image must be done, which is inefficient.<br />
<br />
Further, some user agents discard the compressed image data for an HTMLImageElement after decoding it, and before calling the onload handler. Honoring [[ImageBitmap Options]] in these user agents is not possible without rearchitecting their image decoding subsystem.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
Most WebGL applications encode normal maps or other non-image data into images, and upload these images to textures using WebGL-specific options that skip the alpha premultiplication or color correction the user agent may perform. Some WebGL implementers have pointed out that the current WebGL mechanisms are inefficient, and usually result in a synchronous image re-decode. Switching to use ImageBitmap to provide control over these image decoding options will be more efficient, and allow the WebGL mechanisms to be removed in the next version of the WebGL specification.<br />
<br />
=== Benefits ===<br />
<br />
Provides the most efficient image decoding path for canvas applications requiring control over image decoding.<br />
<br />
<br />
=== Requests for this Feature ===<br />
<br />
* Microsoft has requested this feature in order to provide an efficient path for uploading WebGL images to textures.<br />
* Other WebGL implementers (Apple, Google, Mozilla) support adding this feature and will implement it.<br />
* The WebGL 2.0 specification under development will remove the WebGL-specific image decoding options and rely on this mechanism instead.<br />
<br />
== Proposed Solutions ==<br />
<br />
==== Processing Model ====<br />
<br />
Compared to constructing an ImageBitmap from an HTMLImageElement, this alternative construction path downloads and decodes the image data as part of resolving the promise returned by createImageBitmap.<br />
<br />
==== Limitations ==== <br />
<br />
Only supports single images. There's no provision for downloading other types of media from URLs.<br />
<br />
==== Implementation ==== <br />
<br />
Browser vendors would trigger their usual image downloading and decoding code paths upon creating an ImageBitmap from a URL. Browsers would implement this new API in order to provide more efficient image handling for 2D and WebGL canvas applications.<br />
<br />
==== Adoption ==== <br />
<br />
WebGL application authors in particular would readily update their code in order to use this new API because it will provide more efficient uploading of images into textures. It's straightforward to add conditional code which prefers this new, more efficient, mechanism, and which uses the older texture upload paths if it's not available.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=ImageBitmap_Options&diff=9751ImageBitmap Options2014-11-05T18:22:35Z<p>Kenrussell: </p>
<hr />
<div><br />
This page is a Work in Progress<br />
<br />
<br />
'''Problem to be solved''': Loading and preparing image resources for rendering to 2D canvas contexts and WebGL contexts is often inefficient using existing APIs.<br />
<br />
== Use Case Description ==<br />
:WebGL applications often use image media for transferring non-image data (illumination maps, vector maps, etc.) For such applications the image data must usually be decoded without colorspace conversions and without alpha premultiplications.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
:Currently, there are several ways to apply arbitrary transformations on the client side, many of which involve intermediate copies of the image data. Some solutions involve DataURLs, temporary canvases, or direct manipulation of image data arrays, which are inefficient. Preparing the resources asynchronously also presents an additional challenge to the web developer.<br />
<br />
:WebGL applications can request that colorspace conversions be skipped at texture upload time by specifying pixel storage parameters, but this currently causes a synchronous image decode, because it is not known in advance that the image will need to be decoded in a special way. This latency is a significant problem for applications that upload many images to the GPU, such as Google Maps.<br />
<br />
:ImageBitmap offers an opportunity to provide exactly the desired image decoding options when the image is fetched, so that when the ImageBitmap is ready, it is in exactly the desired form for upload to the GPU.<br />
<br />
=== Benefits ===<br />
<br />
:With the right API support, image resources can be prepared asynchronously and/or in advance to avoid rendering latency issues. Furthermore, the preparation steps could be performed by native code, and in many case they can be integrated directly into the image decoding step without any additional memory consumption and little or no additional computational burden.<br />
<br />
=== Requests for this Feature ===<br />
<br />
* <cite>[http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-June/039793.html June 2013 whatwg thread]</cite><br />
* <cite>[http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-July/039983.html July 2013 whatwg thread]</cite><br />
<br />
== Proposed Solutions ==<br />
<br />
=== ImageBitmap Construction Options ===<br />
<br />
:Add an optional ''options'' argument to createImageBitmap to specify image treatments to be applied when creating the ImageBitmap object.<br />
<br />
==== Strongly desired options ====<br />
<br />
* enum imageOrientation, default = 'topLeft' <blockquote><p>'none': Do not change orientation.</p><p>'flipY': Flip image vertically, disregarding image orientation meta data (EXIF or other), if any.</p><p>'topLeft': identify visual orientation from image media meta data and re-order data so top-down, left-to-right processing for display results in correct visual orientation. If the image source is an ImageBitmap, the image data will be flipped vertically if and only if the source was created with bottomLeft orientation. If the image source holds no orientation information, the image orientation is not changed.</p><p>'bottomLeft': identify visual orientation from image media meta data and re-order data so bottom-up, left-to-right processing for display results in correct visual orientation. If the image source is an ImageBitmap, the image data will be flipped vertically if and only if the source was created with topLeft orientation. If the image source holds no orientation information, the image is flipped vertically.</p></blockquote><br />
* boolean colorspaceConversion, default = true <blockquote><p>If set to false, the ImageBitmap must be created by decoding the source image media without performing any colorspace conversions. This means that the image decoding algorithm must ignore color profile meta-data embedded in the image source as well as the display device color profile. colorspaceConversion only applies when creating ImageBitmaps from bitmap media data (i.e. blobs and image elements with non-vector images)</p></blockquote><br />
* boolean premultiplyAlpha, default = true <blockquote><p>If set to false, image sources that are premultiplied by alpha will have their color components divided by alpha and image sources that are not premultiplied will be left untouched.</p><p>If set to true, image sources that are not premultiplied by alpha will have their color components multiplied by alpha and image sources that are premultiplied will be left untouched.</p><p>When premultiplyAlpha is set to false, the division by alpha may result in loss of precision, which must be avoided whenever possible. For example, if the image source is a blob or an image element, the decoded image may already be available in premultipled form. Rather than divide the already decoded data, the user agent must re-decode the image without premultiplying, thus ensuring that the ImageBitmap is delivered with the highest quality data.</p><p>Image sources that do not have an alpha channel are not affected by this option.</p><p>Caution: Canvases are assumed to be premultiplied image sources, which may not always be accurate.</p><br />
</blockquote><br />
* DOMString? crossOrigin, default = null <blockquote><p>The crossOrigin option is a [[https://html.spec.whatwg.org/multipage/infrastructure.html#cors-settings-attribute CORS settings attribute]]. It is consulted when [[ImageBitmap Constructor taking URL|constructing an ImageBitmap from the URL of an image]]. Because ImageBitmaps can only be constructed from same-origin resources, in order to construct an ImageBitmap from the URL of a cross-origin image resource, the crossOrigin option must be specified, and the server must grant the request. Otherwise the Promise returned from createImageBitmap will be rejected.<br />
</blockquote><br />
<br />
==== Other potentially useful options ====<br />
<br />
* resizedWidth, resizedHeight: dimension to which the image should be resized. Could also be expressed using scale factors<br />
* resizeFilter: filtering method used for scaling the image (pixelated, low quality, high quality, etc.)<br />
* frameIndex: for creating an image bitmap from multi-frame image media<br />
* flipX: like flipY but horizontal<br />
* colorspace: to decode images to a colorspace other than the device colorspace (sRGB, Adobe RGB, YUV, YCbCr, etc.)<br />
* custom color tranforms: color matrix, 3D Look up tables, per component look up tables, bias, gain, ... <br />
<br />
==== Processing Model ====<br />
: TBD ''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
: TBD ''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
: TBD ''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
* Easier to develop asychronous image resource loading and preparation<br />
* Better performance that what can be implemented using current APIs<br />
* Applications can rely on the browser resource management facilities for caching decoded image data with the desired options baked-in and for evicting pre-decoded resources in low memory conditions to avoid out-of-memory crashes.<br />
<br />
==== See Also ====<br />
<br />
Related is the [[ImageBitmap Constructor taking URL]].<br />
<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=ImageBitmap_Options&diff=9750ImageBitmap Options2014-11-05T18:22:06Z<p>Kenrussell: /* Strongly desired options */</p>
<hr />
<div><br />
This page is a Work in Progress<br />
<br />
<br />
'''Problem to be solved''': Loading and preparing image resources for rendering to 2D canvas contexts and WebGL contexts is often inefficient using existing APIs.<br />
<br />
== Use Case Description ==<br />
:WebGL applications often use image media for transferring non-image data (illumination maps, vector maps, etc.) For such applications the image data must usually be decoded without colorspace conversions and without alpha premultiplications.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
:Currently, there are several ways to apply arbitrary transformations on the client side, many of which involve intermediate copies of the image data. Some solutions involve DataURLs, temporary canvases, or direct manipulation of image data arrays, which are inefficient. Preparing the resources asynchronously also presents an additional challenge to the web developer.<br />
<br />
:WebGL applications can request that colorspace conversions be skipped at texture upload time by specifying pixel storage parameters, but this currently causes a synchronous image decode, because it is not known in advance that the image will need to be decoded in a special way. This latency is a significant problem for applications that upload many images to the GPU, such as Google Maps.<br />
<br />
:ImageBitmap offers an opportunity to provide exactly the desired image decoding options when the image is fetched, so that when the ImageBitmap is ready, it is in exactly the desired form for upload to the GPU.<br />
<br />
=== Benefits ===<br />
<br />
:With the right API support, image resources can be prepared asynchronously and/or in advance to avoid rendering latency issues. Furthermore, the preparation steps could be performed by native code, and in many case they can be integrated directly into the image decoding step without any additional memory consumption and little or no additional computational burden.<br />
<br />
=== Requests for this Feature ===<br />
<br />
* <cite>[http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-June/039793.html June 2013 whatwg thread]</cite><br />
* <cite>[http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-July/039983.html July 2013 whatwg thread]</cite><br />
<br />
== Proposed Solutions ==<br />
<br />
=== ImageBitmap Construction Options ===<br />
<br />
:Add an optional ''options'' argument to createImageBitmap to specify image treatments to be applied when creating the ImageBitmap object.<br />
<br />
==== Strongly desired options ====<br />
<br />
* enum imageOrientation, default = 'topLeft' <blockquote><p>'none': Do not change orientation.</p><p>'flipY': Flip image vertically, disregarding image orientation meta data (EXIF or other), if any.</p><p>'topLeft': identify visual orientation from image media meta data and re-order data so top-down, left-to-right processing for display results in correct visual orientation. If the image source is an ImageBitmap, the image data will be flipped vertically if and only if the source was created with bottomLeft orientation. If the image source holds no orientation information, the image orientation is not changed.</p><p>'bottomLeft': identify visual orientation from image media meta data and re-order data so bottom-up, left-to-right processing for display results in correct visual orientation. If the image source is an ImageBitmap, the image data will be flipped vertically if and only if the source was created with topLeft orientation. If the image source holds no orientation information, the image is flipped vertically.</p></blockquote><br />
* boolean colorspaceConversion, default = true <blockquote><p>If set to false, the ImageBitmap must be created by decoding the source image media without performing any colorspace conversions. This means that the image decoding algorithm must ignore color profile meta-data embedded in the image source as well as the display device color profile. colorspaceConversion only applies when creating ImageBitmaps from bitmap media data (i.e. blobs and image elements with non-vector images)</p></blockquote><br />
* boolean premultiplyAlpha, default = true <blockquote><p>If set to false, image sources that are premultiplied by alpha will have their color components divided by alpha and image sources that are not premultiplied will be left untouched.</p><p>If set to true, image sources that are not premultiplied by alpha will have their color components multiplied by alpha and image sources that are premultiplied will be left untouched.</p><p>When premultiplyAlpha is set to false, the division by alpha may result in loss of precision, which must be avoided whenever possible. For example, if the image source is a blob or an image element, the decoded image may already be available in premultipled form. Rather than divide the already decoded data, the user agent must re-decode the image without premultiplying, thus ensuring that the ImageBitmap is delivered with the highest quality data.</p><p>Image sources that do not have an alpha channel are not affected by this option.</p><p>Caution: Canvases are assumed to be premultiplied image sources, which may not always be accurate.</p><br />
</blockquote><br />
* DOMString? crossOrigin, default = null <blockquote><p>The crossOrigin option is a [[https://html.spec.whatwg.org/multipage/infrastructure.html#cors-settings-attribute CORS settings attribute]]. It is consulted when [[ImageBitmap Constructor taking URL|constructing an ImageBitmap from the URL of an image]]. Because ImageBitmaps can only be constructed from same-origin resources, in order to construct an ImageBitmap from a URL of a cross-origin image resources, the crossOrigin option must be specified, and the server must grant the request. Otherwise the Promise returned from createImageBitmap will be rejected.<br />
</blockquote><br />
<br />
==== Other potentially useful options ====<br />
<br />
* resizedWidth, resizedHeight: dimension to which the image should be resized. Could also be expressed using scale factors<br />
* resizeFilter: filtering method used for scaling the image (pixelated, low quality, high quality, etc.)<br />
* frameIndex: for creating an image bitmap from multi-frame image media<br />
* flipX: like flipY but horizontal<br />
* colorspace: to decode images to a colorspace other than the device colorspace (sRGB, Adobe RGB, YUV, YCbCr, etc.)<br />
* custom color tranforms: color matrix, 3D Look up tables, per component look up tables, bias, gain, ... <br />
<br />
==== Processing Model ====<br />
: TBD ''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
: TBD ''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
: TBD ''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
* Easier to develop asychronous image resource loading and preparation<br />
* Better performance that what can be implemented using current APIs<br />
* Applications can rely on the browser resource management facilities for caching decoded image data with the desired options baked-in and for evicting pre-decoded resources in low memory conditions to avoid out-of-memory crashes.<br />
<br />
==== See Also ====<br />
<br />
Related is the [[ImageBitmap Constructor taking URL]].<br />
<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=ImageBitmap_Options&diff=9749ImageBitmap Options2014-11-05T18:20:46Z<p>Kenrussell: /* Strongly desired options */</p>
<hr />
<div><br />
This page is a Work in Progress<br />
<br />
<br />
'''Problem to be solved''': Loading and preparing image resources for rendering to 2D canvas contexts and WebGL contexts is often inefficient using existing APIs.<br />
<br />
== Use Case Description ==<br />
:WebGL applications often use image media for transferring non-image data (illumination maps, vector maps, etc.) For such applications the image data must usually be decoded without colorspace conversions and without alpha premultiplications.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
:Currently, there are several ways to apply arbitrary transformations on the client side, many of which involve intermediate copies of the image data. Some solutions involve DataURLs, temporary canvases, or direct manipulation of image data arrays, which are inefficient. Preparing the resources asynchronously also presents an additional challenge to the web developer.<br />
<br />
:WebGL applications can request that colorspace conversions be skipped at texture upload time by specifying pixel storage parameters, but this currently causes a synchronous image decode, because it is not known in advance that the image will need to be decoded in a special way. This latency is a significant problem for applications that upload many images to the GPU, such as Google Maps.<br />
<br />
:ImageBitmap offers an opportunity to provide exactly the desired image decoding options when the image is fetched, so that when the ImageBitmap is ready, it is in exactly the desired form for upload to the GPU.<br />
<br />
=== Benefits ===<br />
<br />
:With the right API support, image resources can be prepared asynchronously and/or in advance to avoid rendering latency issues. Furthermore, the preparation steps could be performed by native code, and in many case they can be integrated directly into the image decoding step without any additional memory consumption and little or no additional computational burden.<br />
<br />
=== Requests for this Feature ===<br />
<br />
* <cite>[http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-June/039793.html June 2013 whatwg thread]</cite><br />
* <cite>[http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-July/039983.html July 2013 whatwg thread]</cite><br />
<br />
== Proposed Solutions ==<br />
<br />
=== ImageBitmap Construction Options ===<br />
<br />
:Add an optional ''options'' argument to createImageBitmap to specify image treatments to be applied when creating the ImageBitmap object.<br />
<br />
==== Strongly desired options ====<br />
<br />
* enum imageOrientation, default = 'topLeft' <blockquote><p>'none': Do not change orientation.</p><p>'flipY': Flip image vertically, disregarding image orientation meta data (EXIF or other), if any.</p><p>'topLeft': identify visual orientation from image media meta data and re-order data so top-down, left-to-right processing for display results in correct visual orientation. If the image source is an ImageBitmap, the image data will be flipped vertically if and only if the source was created with bottomLeft orientation. If the image source holds no orientation information, the image orientation is not changed.</p><p>'bottomLeft': identify visual orientation from image media meta data and re-order data so bottom-up, left-to-right processing for display results in correct visual orientation. If the image source is an ImageBitmap, the image data will be flipped vertically if and only if the source was created with topLeft orientation. If the image source holds no orientation information, the image is flipped vertically.</p></blockquote><br />
* boolean colorspaceConversion, default = true <blockquote><p>If set to false, the ImageBitmap must be created by decoding the source image media without performing any colorspace conversions. This means that the image decoding algorithm must ignore color profile meta-data embedded in the image source as well as the display device color profile. colorspaceConversion only applies when creating ImageBitmaps from bitmap media data (i.e. blobs and image elements with non-vector images)</p></blockquote><br />
* boolean premultiplyAlpha, default = true <blockquote><p>If set to false, image sources that are premultiplied by alpha will have their color components divided by alpha and image sources that are not premultiplied will be left untouched.</p><p>If set to true, image sources that are not premultiplied by alpha will have their color components multiplied by alpha and image sources that are premultiplied will be left untouched.</p><p>When premultiplyAlpha is set to false, the division by alpha may result in loss of precision, which must be avoided whenever possible. For example, if the image source is a blob or an image element, the decoded image may already be available in premultipled form. Rather than divide the already decoded data, the user agent must re-decode the image without premultiplying, thus ensuring that the ImageBitmap is delivered with the highest quality data.</p><p>Image sources that do not have an alpha channel are not affected by this option.</p><p>Caution: Canvases are assumed to be premultiplied image sources, which may not always be accurate.</p><br />
</blockquote><br />
* DOMString? crossOrigin, default = null <blockquote><p>The crossOrigin option is a [[https://html.spec.whatwg.org/multipage/infrastructure.html#cors-settings-attribute CORS settings attribute]]. It is consulted when [[ImageBitmap Constructor taking URL|constructing an ImageBitmap from the URL of an image]]. Because ImageBitmaps can only be constructed from same-origin resources, the crossOrigin option is required in order to create an ImageBitmap from a URL of a cross-origin image resource, and further, the server must grant the request.<br />
</blockquote><br />
<br />
==== Other potentially useful options ====<br />
<br />
* resizedWidth, resizedHeight: dimension to which the image should be resized. Could also be expressed using scale factors<br />
* resizeFilter: filtering method used for scaling the image (pixelated, low quality, high quality, etc.)<br />
* frameIndex: for creating an image bitmap from multi-frame image media<br />
* flipX: like flipY but horizontal<br />
* colorspace: to decode images to a colorspace other than the device colorspace (sRGB, Adobe RGB, YUV, YCbCr, etc.)<br />
* custom color tranforms: color matrix, 3D Look up tables, per component look up tables, bias, gain, ... <br />
<br />
==== Processing Model ====<br />
: TBD ''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
: TBD ''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
: TBD ''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
* Easier to develop asychronous image resource loading and preparation<br />
* Better performance that what can be implemented using current APIs<br />
* Applications can rely on the browser resource management facilities for caching decoded image data with the desired options baked-in and for evicting pre-decoded resources in low memory conditions to avoid out-of-memory crashes.<br />
<br />
==== See Also ====<br />
<br />
Related is the [[ImageBitmap Constructor taking URL]].<br />
<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=ImageBitmap_Options&diff=9748ImageBitmap Options2014-11-05T18:19:17Z<p>Kenrussell: /* Strongly desired options */</p>
<hr />
<div><br />
This page is a Work in Progress<br />
<br />
<br />
'''Problem to be solved''': Loading and preparing image resources for rendering to 2D canvas contexts and WebGL contexts is often inefficient using existing APIs.<br />
<br />
== Use Case Description ==<br />
:WebGL applications often use image media for transferring non-image data (illumination maps, vector maps, etc.) For such applications the image data must usually be decoded without colorspace conversions and without alpha premultiplications.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
:Currently, there are several ways to apply arbitrary transformations on the client side, many of which involve intermediate copies of the image data. Some solutions involve DataURLs, temporary canvases, or direct manipulation of image data arrays, which are inefficient. Preparing the resources asynchronously also presents an additional challenge to the web developer.<br />
<br />
:WebGL applications can request that colorspace conversions be skipped at texture upload time by specifying pixel storage parameters, but this currently causes a synchronous image decode, because it is not known in advance that the image will need to be decoded in a special way. This latency is a significant problem for applications that upload many images to the GPU, such as Google Maps.<br />
<br />
:ImageBitmap offers an opportunity to provide exactly the desired image decoding options when the image is fetched, so that when the ImageBitmap is ready, it is in exactly the desired form for upload to the GPU.<br />
<br />
=== Benefits ===<br />
<br />
:With the right API support, image resources can be prepared asynchronously and/or in advance to avoid rendering latency issues. Furthermore, the preparation steps could be performed by native code, and in many case they can be integrated directly into the image decoding step without any additional memory consumption and little or no additional computational burden.<br />
<br />
=== Requests for this Feature ===<br />
<br />
* <cite>[http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-June/039793.html June 2013 whatwg thread]</cite><br />
* <cite>[http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-July/039983.html July 2013 whatwg thread]</cite><br />
<br />
== Proposed Solutions ==<br />
<br />
=== ImageBitmap Construction Options ===<br />
<br />
:Add an optional ''options'' argument to createImageBitmap to specify image treatments to be applied when creating the ImageBitmap object.<br />
<br />
==== Strongly desired options ====<br />
<br />
* enum imageOrientation, default = 'topLeft' <blockquote><p>'none': Do not change orientation.</p><p>'flipY': Flip image vertically, disregarding image orientation meta data (EXIF or other), if any.</p><p>'topLeft': identify visual orientation from image media meta data and re-order data so top-down, left-to-right processing for display results in correct visual orientation. If the image source is an ImageBitmap, the image data will be flipped vertically if and only if the source was created with bottomLeft orientation. If the image source holds no orientation information, the image orientation is not changed.</p><p>'bottomLeft': identify visual orientation from image media meta data and re-order data so bottom-up, left-to-right processing for display results in correct visual orientation. If the image source is an ImageBitmap, the image data will be flipped vertically if and only if the source was created with topLeft orientation. If the image source holds no orientation information, the image is flipped vertically.</p></blockquote><br />
* boolean colorspaceConversion, default = true <blockquote><p>If set to false, the ImageBitmap must be created by decoding the source image media without performing any colorspace conversions. This means that the image decoding algorithm must ignore color profile meta-data embedded in the image source as well as the display device color profile. colorspaceConversion only applies when creating ImageBitmaps from bitmap media data (i.e. blobs and image elements with non-vector images)</p></blockquote><br />
* boolean premultiplyAlpha, default = true <blockquote><p>If set to false, image sources that are premultiplied by alpha will have their color components divided by alpha and image sources that are not premultiplied will be left untouched.</p><p>If set to true, image sources that are not premultiplied by alpha will have their color components multiplied by alpha and image sources that are premultiplied will be left untouched.</p><p>When premultiplyAlpha is set to false, the division by alpha may result in loss of precision, which must be avoided whenever possible. For example, if the image source is a blob or an image element, the decoded image may already be available in premultipled form. Rather than divide the already decoded data, the user agent must re-decode the image without premultiplying, thus ensuring that the ImageBitmap is delivered with the highest quality data.</p><p>Image sources that do not have an alpha channel are not affected by this option.</p><p>Caution: Canvases are assumed to be premultiplied image sources, which may not always be accurate.</p><br />
</blockquote><br />
* DOMString? crossOrigin, default = null <blockquote><p>The crossOrigin option is a <a href="https://html.spec.whatwg.org/multipage/infrastructure.html#cors-settings-attribute">CORS settings attribute</a>. It is consulted when [[ImageBitmap Constructor taking URL|constructing an ImageBitmap from the URL of an image]]. Because ImageBitmaps can only be constructed from same-origin resources, the crossOrigin option is required in order to create an ImageBitmap from a URL of a cross-origin image resource, and further, the server must grant the request.<br />
</blockquote><br />
<br />
==== Other potentially useful options ====<br />
<br />
* resizedWidth, resizedHeight: dimension to which the image should be resized. Could also be expressed using scale factors<br />
* resizeFilter: filtering method used for scaling the image (pixelated, low quality, high quality, etc.)<br />
* frameIndex: for creating an image bitmap from multi-frame image media<br />
* flipX: like flipY but horizontal<br />
* colorspace: to decode images to a colorspace other than the device colorspace (sRGB, Adobe RGB, YUV, YCbCr, etc.)<br />
* custom color tranforms: color matrix, 3D Look up tables, per component look up tables, bias, gain, ... <br />
<br />
==== Processing Model ====<br />
: TBD ''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
: TBD ''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
: TBD ''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
* Easier to develop asychronous image resource loading and preparation<br />
* Better performance that what can be implemented using current APIs<br />
* Applications can rely on the browser resource management facilities for caching decoded image data with the desired options baked-in and for evicting pre-decoded resources in low memory conditions to avoid out-of-memory crashes.<br />
<br />
==== See Also ====<br />
<br />
Related is the [[ImageBitmap Constructor taking URL]].<br />
<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=ImageBitmap_Options&diff=9747ImageBitmap Options2014-11-05T18:18:55Z<p>Kenrussell: Added crossOrigin option, required for creating ImageBitmaps from URLs.</p>
<hr />
<div><br />
This page is a Work in Progress<br />
<br />
<br />
'''Problem to be solved''': Loading and preparing image resources for rendering to 2D canvas contexts and WebGL contexts is often inefficient using existing APIs.<br />
<br />
== Use Case Description ==<br />
:WebGL applications often use image media for transferring non-image data (illumination maps, vector maps, etc.) For such applications the image data must usually be decoded without colorspace conversions and without alpha premultiplications.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
:Currently, there are several ways to apply arbitrary transformations on the client side, many of which involve intermediate copies of the image data. Some solutions involve DataURLs, temporary canvases, or direct manipulation of image data arrays, which are inefficient. Preparing the resources asynchronously also presents an additional challenge to the web developer.<br />
<br />
:WebGL applications can request that colorspace conversions be skipped at texture upload time by specifying pixel storage parameters, but this currently causes a synchronous image decode, because it is not known in advance that the image will need to be decoded in a special way. This latency is a significant problem for applications that upload many images to the GPU, such as Google Maps.<br />
<br />
:ImageBitmap offers an opportunity to provide exactly the desired image decoding options when the image is fetched, so that when the ImageBitmap is ready, it is in exactly the desired form for upload to the GPU.<br />
<br />
=== Benefits ===<br />
<br />
:With the right API support, image resources can be prepared asynchronously and/or in advance to avoid rendering latency issues. Furthermore, the preparation steps could be performed by native code, and in many case they can be integrated directly into the image decoding step without any additional memory consumption and little or no additional computational burden.<br />
<br />
=== Requests for this Feature ===<br />
<br />
* <cite>[http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-June/039793.html June 2013 whatwg thread]</cite><br />
* <cite>[http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-July/039983.html July 2013 whatwg thread]</cite><br />
<br />
== Proposed Solutions ==<br />
<br />
=== ImageBitmap Construction Options ===<br />
<br />
:Add an optional ''options'' argument to createImageBitmap to specify image treatments to be applied when creating the ImageBitmap object.<br />
<br />
==== Strongly desired options ====<br />
<br />
* enum imageOrientation, default = 'topLeft' <blockquote><p>'none': Do not change orientation.</p><p>'flipY': Flip image vertically, disregarding image orientation meta data (EXIF or other), if any.</p><p>'topLeft': identify visual orientation from image media meta data and re-order data so top-down, left-to-right processing for display results in correct visual orientation. If the image source is an ImageBitmap, the image data will be flipped vertically if and only if the source was created with bottomLeft orientation. If the image source holds no orientation information, the image orientation is not changed.</p><p>'bottomLeft': identify visual orientation from image media meta data and re-order data so bottom-up, left-to-right processing for display results in correct visual orientation. If the image source is an ImageBitmap, the image data will be flipped vertically if and only if the source was created with topLeft orientation. If the image source holds no orientation information, the image is flipped vertically.</p></blockquote><br />
* boolean colorspaceConversion, default = true <blockquote><p>If set to false, the ImageBitmap must be created by decoding the source image media without performing any colorspace conversions. This means that the image decoding algorithm must ignore color profile meta-data embedded in the image source as well as the display device color profile. colorspaceConversion only applies when creating ImageBitmaps from bitmap media data (i.e. blobs and image elements with non-vector images)</p></blockquote><br />
* boolean premultiplyAlpha, default = true <blockquote><p>If set to false, image sources that are premultiplied by alpha will have their color components divided by alpha and image sources that are not premultiplied will be left untouched.</p><p>If set to true, image sources that are not premultiplied by alpha will have their color components multiplied by alpha and image sources that are premultiplied will be left untouched.</p><p>When premultiplyAlpha is set to false, the division by alpha may result in loss of precision, which must be avoided whenever possible. For example, if the image source is a blob or an image element, the decoded image may already be available in premultipled form. Rather than divide the already decoded data, the user agent must re-decode the image without premultiplying, thus ensuring that the ImageBitmap is delivered with the highest quality data.</p><p>Image sources that do not have an alpha channel are not affected by this option.</p><p>Caution: Canvases are assumed to be premultiplied image sources, which may not always be accurate.</p><br />
</blockquote><br />
* DOMString? crossOrigin, default = null <blockquote><br />
<p>The crossOrigin option is a <a href="https://html.spec.whatwg.org/multipage/infrastructure.html#cors-settings-attribute">CORS settings attribute</a>. It is consulted when [[ImageBitmap Constructor taking URL|constructing an ImageBitmap from the URL of an image]]. Because ImageBitmaps can only be constructed from same-origin resources, the crossOrigin option is required in order to create an ImageBitmap from a URL of a cross-origin image resource, and further, the server must grant the request.<br />
</blockquote><br />
<br />
==== Other potentially useful options ====<br />
<br />
* resizedWidth, resizedHeight: dimension to which the image should be resized. Could also be expressed using scale factors<br />
* resizeFilter: filtering method used for scaling the image (pixelated, low quality, high quality, etc.)<br />
* frameIndex: for creating an image bitmap from multi-frame image media<br />
* flipX: like flipY but horizontal<br />
* colorspace: to decode images to a colorspace other than the device colorspace (sRGB, Adobe RGB, YUV, YCbCr, etc.)<br />
* custom color tranforms: color matrix, 3D Look up tables, per component look up tables, bias, gain, ... <br />
<br />
==== Processing Model ====<br />
: TBD ''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
: TBD ''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
: TBD ''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
* Easier to develop asychronous image resource loading and preparation<br />
* Better performance that what can be implemented using current APIs<br />
* Applications can rely on the browser resource management facilities for caching decoded image data with the desired options baked-in and for evicting pre-decoded resources in low memory conditions to avoid out-of-memory crashes.<br />
<br />
==== See Also ====<br />
<br />
Related is the [[ImageBitmap Constructor taking URL]].<br />
<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=ImageBitmap_Constructor_taking_URL&diff=9746ImageBitmap Constructor taking URL2014-11-05T18:11:03Z<p>Kenrussell: </p>
<hr />
<div>:''Allows creating an ImageBitmap from the URL of an image.''<br />
<br />
== Use Case Description ==<br />
<br />
ImageBitmap provides a useful abstraction for image data that can be drawn to a 2D canvas context or uploaded to a texture in a WebGL canvas context. It attempts to move all of the image decoding and preparation work out of the critical path, so that by the time an ImageBitmap is constructed, the data it contains is in exactly the right format for rendering. [[ImageBitmap Options]] provides control over the image decoding process so that the resulting bitmap is in exactly the desired form. For example, it can be used to disable the user agent's premultiplication of the alpha channel into the color channels.<br />
<br />
Currently an ImageBitmap can only be constructed from a fully-loaded media resource such as an HTMLImageElement. Unfortunately, some user agents discard the compressed image data from HTMLImageElements before firing the onload handler. (An ImageBitmap can only be constructed from an HTMLImageElement after the image element's onload handler has fired.) These user agents perform all of the image decompression before firing the onload handler. The only way that the ImageBitmap options can be taken into account is during the decoding process. Transformations done during decoding, like premultiplying the alpha channel into the color channels, are not reversible.<br />
<br />
This proposal adds URL to the ImageBitmapSource union type, which is the first argument to createImageBitmap. The image is downloaded from that URL, decoded, and processed as part of the construction of the ImageBitmap, taking into account the ImageBitmap options. If any errors occur during the downloading or decoding process, including if the image is not single-origin after honoring the crossOrigin attribute of the ImageBitmap options, the Promise returned from createImageBitmap is rejected. If image downloading and decoding succeeds, then the promise is resolved with the new ImageBitmap as its value.<br />
<br />
=== Current Limitations ===<br />
<br />
The intent of ImageBitmap is to prepare the image data in the desired form for drawing or texture uploading before resolving the promise returned from createImageBitmap. Unfortunately, most user agents decode the image before calling the onload handler, and it's only legal to pass an HTMLImageElement to createImageBitmap once the image's onload handler has been called. This means that at a minimum, in order to respect the [[ImageBitmap Options]] argument to createImageBitmap, two decodes of the image must be done, which is inefficient.<br />
<br />
Further, some user agents discard the compressed image data for an HTMLImageElement after decoding it, and before calling the onload handler. Honoring [[ImageBitmap Options]] in these user agents is not possible without rearchitecting their image decoding subsystem.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
Most WebGL applications encode normal maps or other non-image data into images, and upload these images to textures using WebGL-specific options that skip the alpha premultiplication or color correction the user agent may perform. Some WebGL implementers have pointed out that the current WebGL mechanisms are inefficient, and usually result in a synchronous image re-decode. Switching to use ImageBitmap to provide control over these image decoding options will be more efficient, and allow the WebGL mechanisms to be removed in the next version of the WebGL specification.<br />
<br />
=== Benefits ===<br />
<br />
Provides the most efficient image decoding path for canvas applications requiring control over image decoding.<br />
<br />
<br />
=== Requests for this Feature ===<br />
<br />
* Microsoft has requested this feature in order to provide an efficient path for uploading WebGL images to textures.<br />
* Other WebGL implementers (Apple, Google, Mozilla) support adding this feature and will implement it.<br />
<br />
== Proposed Solutions ==<br />
<br />
==== Processing Model ====<br />
<br />
Compared to constructing an ImageBitmap from an HTMLImageElement, this alternative construction path downloads and decodes the image data as part of resolving the promise returned by createImageBitmap.<br />
<br />
==== Limitations ==== <br />
<br />
Only supports single images. There's no provision for downloading other types of media from URLs.<br />
<br />
==== Implementation ==== <br />
<br />
Browser vendors would trigger their usual image downloading and decoding code paths upon creating an ImageBitmap from a URL. Browsers would implement this new API in order to provide more efficient image handling for 2D and WebGL canvas applications.<br />
<br />
==== Adoption ==== <br />
<br />
WebGL application authors in particular would readily update their code in order to use this new API because it will provide more efficient uploading of images into textures. It's straightforward to add conditional code which prefers this new, more efficient, mechanism, and which uses the older texture upload paths if it's not available.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=ImageBitmap_Constructor_taking_URL&diff=9745ImageBitmap Constructor taking URL2014-11-05T18:10:50Z<p>Kenrussell: </p>
<hr />
<div>:''Allows creating an ImageBitmap from the URL of an image.''<br />
<br />
== Use Case Description ==<br />
<br />
ImageBitmap provides a useful abstraction for image data that can be drawn to a 2D canvas context or uploaded to a texture in a WebGL canvas context. It attempts to move all of the image decoding and preparation work out of the critical path, so that by the time an ImageBitmap is constructed, the data it contains is in exactly the right format for rendering. [[ImageBitmap Options]] provides control over the image decoding process so that the resulting bitmap is in exactly the desired form. For example, it can be used to disable the user agent's premultiplication of the alpha channel into the color channels.<br />
<br />
Currently an ImageBitmap can only be constructed from a fully-loaded media resource such as an HTMLImageElement. Unfortunately, some user agents discard the compressed image data from HTMLImageElements before firing the onload handler. (An ImageBitmap can only be constructed from an HTMLImageElement after the image element's onload handler has fired.) These user agents perform all of the image decompression before firing the onload handler. The only way that the ImageBitmap options can be taken into account is during the decoding process. Transformations done during decoding, like premultiplying the alpha channel into the color channels, are not reversible.<br />
<br />
This proposal adds URL to the ImageBitmapSource union type: the first argument to createImageBitmap. The image is downloaded from that URL, decoded, and processed as part of the construction of the ImageBitmap, taking into account the ImageBitmap options. If any errors occur during the downloading or decoding process, including if the image is not single-origin after honoring the crossOrigin attribute of the ImageBitmap options, the Promise returned from createImageBitmap is rejected. If image downloading and decoding succeeds, then the promise is resolved with the new ImageBitmap as its value.<br />
<br />
=== Current Limitations ===<br />
<br />
The intent of ImageBitmap is to prepare the image data in the desired form for drawing or texture uploading before resolving the promise returned from createImageBitmap. Unfortunately, most user agents decode the image before calling the onload handler, and it's only legal to pass an HTMLImageElement to createImageBitmap once the image's onload handler has been called. This means that at a minimum, in order to respect the [[ImageBitmap Options]] argument to createImageBitmap, two decodes of the image must be done, which is inefficient.<br />
<br />
Further, some user agents discard the compressed image data for an HTMLImageElement after decoding it, and before calling the onload handler. Honoring [[ImageBitmap Options]] in these user agents is not possible without rearchitecting their image decoding subsystem.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
Most WebGL applications encode normal maps or other non-image data into images, and upload these images to textures using WebGL-specific options that skip the alpha premultiplication or color correction the user agent may perform. Some WebGL implementers have pointed out that the current WebGL mechanisms are inefficient, and usually result in a synchronous image re-decode. Switching to use ImageBitmap to provide control over these image decoding options will be more efficient, and allow the WebGL mechanisms to be removed in the next version of the WebGL specification.<br />
<br />
=== Benefits ===<br />
<br />
Provides the most efficient image decoding path for canvas applications requiring control over image decoding.<br />
<br />
<br />
=== Requests for this Feature ===<br />
<br />
* Microsoft has requested this feature in order to provide an efficient path for uploading WebGL images to textures.<br />
* Other WebGL implementers (Apple, Google, Mozilla) support adding this feature and will implement it.<br />
<br />
== Proposed Solutions ==<br />
<br />
==== Processing Model ====<br />
<br />
Compared to constructing an ImageBitmap from an HTMLImageElement, this alternative construction path downloads and decodes the image data as part of resolving the promise returned by createImageBitmap.<br />
<br />
==== Limitations ==== <br />
<br />
Only supports single images. There's no provision for downloading other types of media from URLs.<br />
<br />
==== Implementation ==== <br />
<br />
Browser vendors would trigger their usual image downloading and decoding code paths upon creating an ImageBitmap from a URL. Browsers would implement this new API in order to provide more efficient image handling for 2D and WebGL canvas applications.<br />
<br />
==== Adoption ==== <br />
<br />
WebGL application authors in particular would readily update their code in order to use this new API because it will provide more efficient uploading of images into textures. It's straightforward to add conditional code which prefers this new, more efficient, mechanism, and which uses the older texture upload paths if it's not available.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=ImageBitmap_Constructor_taking_URL&diff=9744ImageBitmap Constructor taking URL2014-11-05T18:03:25Z<p>Kenrussell: Added proposal for constructing ImageBitmaps from URLs.</p>
<hr />
<div>:''Allows creating an ImageBitmap from the URL of an image.''<br />
<br />
== Use Case Description ==<br />
<br />
ImageBitmap provides a useful abstraction for image data that can be drawn to a 2D canvas context or uploaded to a texture in a WebGL canvas context. It attempts to move all of the image decoding and preparation work out of the critical path, so that by the time an ImageBitmap is constructed, the data it contains is in exactly the right format for rendering. [[ImageBitmap Options]] provides control over the image decoding process so that the resulting bitmap is in exactly the desired form. For example, it can be used to disable the premultiplication of the alpha channel into the color channels which most user agents perform upon image decoding (all known ones, at the time of this writing).<br />
<br />
Currently an ImageBitmap can only be constructed from a fully-loaded media resource such as an HTMLImageElement. Unfortunately, some user agents discard the compressed image data from HTMLImageElements before firing the onload handler. (An ImageBitmap can only be constructed from an HTMLImageElement after the image element's onload handler has fired.) These user agents perform all of the image decompression before firing the onload handler. The only way that the ImageBitmap options can be taken into account is during the decoding process. Transformations like premultiplying the alpha channel into the color channels are not reversible.<br />
<br />
This proposal adds URL to the ImageBitmapSource union type: the first argument to createImageBitmap. The image is downloaded from that URL, decoded, and processed as part of the construction of the ImageBitmap, taking into account the ImageBitmap options. If any errors occur during the downloading or decoding process, including if the image is not single-origin after honoring the crossOrigin attribute of the ImageBitmap options, the Promise returned from createImageBitmap is rejected. If image downloading and decoding succeeds, then the promise is resolved with the new ImageBitmap as its value.<br />
<br />
=== Current Limitations ===<br />
<br />
The intent of ImageBitmap is to prepare the image data in the desired form for drawing or texture uploading before resolving the promise returned from createImageBitmap. Unfortunately, most user agents decode the image before calling the onload handler, and it's only legal to pass an HTMLImageElement to createImageBitmap once the image's onload handler has been called. This means that at a minimum, in order to respect the [[ImageBitmap Options]] argument to createImageBitmap, two decodes of the image must be done, which is inefficient.<br />
<br />
Further, some user agents discard the compressed image data for an HTMLImageElement after decoding it, and before calling the onload handler. Honoring [[ImageBitmap Options]] in these user agents is not possible without rearchitecting their image decoding subsystem.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
Most WebGL applications encode normal maps or other non-image data into images, and upload these images to textures using WebGL-specific options that skip the alpha premultiplication or color correction the user agent may perform. Some WebGL implementers have pointed out that the current WebGL mechanisms are inefficient, and usually result in a synchronous image re-decode. Switching to use ImageBitmap to provide control over these image decoding options will be more efficient, and allow the WebGL mechanisms to be removed in the next version of the WebGL specification.<br />
<br />
=== Benefits ===<br />
<br />
Provides the most efficient image decoding path for canvas applications requiring control over image decoding.<br />
<br />
<br />
=== Requests for this Feature ===<br />
<br />
* Microsoft has requested this feature in order to provide an efficient path for uploading WebGL images to textures.<br />
* Other WebGL implementers (Apple, Google, Mozilla) support adding this feature and will implement it.<br />
<br />
== Proposed Solutions ==<br />
<br />
==== Processing Model ====<br />
<br />
Compared to constructing an ImageBitmap from an HTMLImageElement, this alternative construction path downloads and decodes the image data as part of resolving the promise returned by createImageBitmap.<br />
<br />
==== Limitations ==== <br />
<br />
Only supports single images. There's no provision for downloading other types of media from URLs.<br />
<br />
==== Implementation ==== <br />
<br />
Browser vendors would trigger their usual image downloading and decoding code paths upon creating an ImageBitmap from a URL. Browsers would implement this new API in order to provide more efficient image handling for 2D and WebGL canvas applications.<br />
<br />
==== Adoption ==== <br />
<br />
WebGL application authors in particular would readily update their code in order to use this new API because it will provide more efficient uploading of images into textures. It's straightforward to add conditional code which prefers this new, more efficient, mechanism, and which uses the older texture upload paths if it's not available.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=ImageBitmap_Options&diff=9743ImageBitmap Options2014-11-05T17:04:00Z<p>Kenrussell: Added link to currently non-existent ImageBitmap Constructor taking URL page.</p>
<hr />
<div><br />
This page is a Work in Progress<br />
<br />
<br />
'''Problem to be solved''': Loading and preparing image resources for rendering to 2D canvas contexts and WebGL contexts is often inefficient using existing APIs.<br />
<br />
== Use Case Description ==<br />
:WebGL applications often use image media for transferring non-image data (illumination maps, vector maps, etc.) For such applications the image data must usually be decoded without colorspace conversions and without alpha premultiplications.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
:Currently, there are several ways to apply arbitrary transformations on the client side, many of which involve intermediate copies of the image data. Some solutions involve DataURLs, temporary canvases, or direct manipulation of image data arrays, which are inefficient. Preparing the resources asynchronously also presents an additional challenge to the web developer.<br />
<br />
:WebGL applications can request that colorspace conversions be skipped at texture upload time by specifying pixel storage parameters, but this currently causes a synchronous image decode, because it is not known in advance that the image will need to be decoded in a special way. This latency is a significant problem for applications that upload many images to the GPU, such as Google Maps.<br />
<br />
:ImageBitmap offers an opportunity to provide exactly the desired image decoding options when the image is fetched, so that when the ImageBitmap is ready, it is in exactly the desired form for upload to the GPU.<br />
<br />
=== Benefits ===<br />
<br />
:With the right API support, image resources can be prepared asynchronously and/or in advance to avoid rendering latency issues. Furthermore, the preparation steps could be performed by native code, and in many case they can be integrated directly into the image decoding step without any additional memory consumption and little or no additional computational burden.<br />
<br />
=== Requests for this Feature ===<br />
<br />
* <cite>[http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-June/039793.html June 2013 whatwg thread]</cite><br />
* <cite>[http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-July/039983.html July 2013 whatwg thread]</cite><br />
<br />
== Proposed Solutions ==<br />
<br />
=== ImageBitmap Construction Options ===<br />
<br />
:Add an optional ''options'' argument to createImageBitmap to specify image treatments to be applied when creating the ImageBitmap object.<br />
<br />
==== Strongly desired options ====<br />
<br />
* enum imageOrientation, default = 'topLeft' <blockquote><p>'none': Do not change orientation.</p><p>'flipY': Flip image vertically, disregarding image orientation meta data (EXIF or other), if any.</p><p>'topLeft': identify visual orientation from image media meta data and re-order data so top-down, left-to-right processing for display results in correct visual orientation. If the image source is an ImageBitmap, the image data will be flipped vertically if and only if the source was created with bottomLeft orientation. If the image source holds no orientation information, the image orientation is not changed.</p><p>'bottomLeft': identify visual orientation from image media meta data and re-order data so bottom-up, left-to-right processing for display results in correct visual orientation. If the image source is an ImageBitmap, the image data will be flipped vertically if and only if the source was created with topLeft orientation. If the image source holds no orientation information, the image is flipped vertically.</p></blockquote><br />
* boolean colorspaceConversion, default = true <blockquote><p>If set to false, the ImageBitmap must be created by decoding the source image media without performing any colorspace conversions. This means that the image decoding algorithm must ignore color profile meta-data embedded in the image source as well as the display device color profile. colorspaceConversion only applies when creating ImageBitmaps from bitmap media data (i.e. blobs and image elements with non-vector images)</p></blockquote><br />
* boolean premultiplyAlpha, default = true <blockquote><p>If set to false, image sources that are premultiplied by alpha will have their color components divided by alpha and image sources that are not premultiplied will be left untouched.</p><p>If set to true, image sources that are not premultiplied by alpha will have their color components multiplied by alpha and image sources that are premultiplied will be left untouched.</p><p>When premultiplyAlpha is set to false, the division by alpha may result in loss of precision, which must be avoided whenever possible. For example, if the image source is a blob or an image element, the decoded image may already be available in premultipled form. Rather than divide the already decoded data, the user agent must re-decode the image without premultiplying, thus ensuring that the ImageBitmap is delivered with the highest quality data.</p><p>Image sources that do not have an alpha channel are not affected by this option.</p><p>Caution: Canvases are assumed to be premultiplied image sources, which may not always be accurate.</p><br />
</blockquote><br />
<br />
==== Other potentially useful options ====<br />
<br />
* resizedWidth, resizedHeight: dimension to which the image should be resized. Could also be expressed using scale factors<br />
* resizeFilter: filtering method used for scaling the image (pixelated, low quality, high quality, etc.)<br />
* frameIndex: for creating an image bitmap from multi-frame image media<br />
* flipX: like flipY but horizontal<br />
* colorspace: to decode images to a colorspace other than the device colorspace (sRGB, Adobe RGB, YUV, YCbCr, etc.)<br />
* custom color tranforms: color matrix, 3D Look up tables, per component look up tables, bias, gain, ... <br />
<br />
==== Processing Model ====<br />
: TBD ''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
: TBD ''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
: TBD ''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
* Easier to develop asychronous image resource loading and preparation<br />
* Better performance that what can be implemented using current APIs<br />
* Applications can rely on the browser resource management facilities for caching decoded image data with the desired options baked-in and for evicting pre-decoded resources in low memory conditions to avoid out-of-memory crashes.<br />
<br />
==== See Also ====<br />
<br />
Related is the [[ImageBitmap Constructor taking URL]].<br />
<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=WorkerCanvas&diff=9347WorkerCanvas2013-10-23T23:53:46Z<p>Kenrussell: added Category:Proposals using HotCat</p>
<hr />
<div>== WebIDL ==<br />
<br />
[Constructor(unsigned long width, unsigned long height)]<br />
interface WorkerCanvas {<br />
attribute unsigned long width;<br />
attribute unsigned long height;<br />
RenderingContext? getContext(DOMString contextId, any... arguments); <br />
void toBlob(FileCallback? _callback, optional DOMString type, any... arguments);<br />
ImageBitmap transferToImageBitmap();<br />
};<br />
<br />
partial interface RenderingContext {<br />
void commit();<br />
};<br />
<br />
WorkerCanvas implements Transferable;<br />
ImageBitmap implements Transferable;<br />
<br />
partial interface HTMLCanvasElement {<br />
WorkerCanvas transferControlToWorker();<br />
};<br />
<br />
partial interface ImageBitmap {<br />
void transferToImage(HTMLImageElement image);<br />
};<br />
<br />
== Spec changes ==<br />
<br />
Transferring of ImageBitmaps has to be defined. It should neuter the ImageBitmap in the sending thread. Neutering sets the ImageBitmap's width and height to 0.<br />
<br />
HTMLCanvasElement.transferControlToWorker behaves like transferControlToProxy in the current WHATWG spec. WorkerCanvas is Transferable, but transfer fails if transferred other than from the main thread to a worker. All its methods throw if not called on a worker, or if it's neutered.<br />
<br />
ImageBitmap.transferToImage removes the image element's "src" attribute and makes the image display the contents of the ImageBitmap (until the next transferToImage to that image, or until the image's "src" attribute is set). The ImageBitmap is neutered.<br />
<br />
[[Category:Proposals]]</div>Kenrussellhttps://wiki.whatwg.org/index.php?title=ImageBitmap_Options&diff=9251ImageBitmap Options2013-07-17T23:12:26Z<p>Kenrussell: /* Current Usage and Workarounds */</p>
<hr />
<div><br />
This page is a Work in Progress<br />
<br />
<br />
'''Problem to be solved''': Loading and preparing image resources for rendering to 2D canvas contexts and WebGL contexts is often inefficient using existing APIs.<br />
<br />
== Use Case Description ==<br />
:WebGL applications often use image media for transferring non-image data (illumination maps, vector maps, etc.) For such applications the image data must usually be decoded without colorspace conversions and without alpha premultiplications.<br />
<br />
=== Current Usage and Workarounds ===<br />
<br />
:Currently, there are several ways to apply arbitrary transformations on the client side, many of which involve intermediate copies of the image data. Some solutions involve DataURLs, temporary canvases, or direct manipulation of image data arrays, which are inefficient. Preparing the resources asynchronously also presents an additional challenge to the web developer.<br />
<br />
:WebGL applications can request that colorspace conversions be skipped at texture upload time by specifying pixel storage parameters, but this currently causes a synchronous image decode, because it is not known in advance that the image will need to be decoded in a special way. This latency is a significant problem for applications that upload many images to the GPU, such as Google Maps.<br />
<br />
:ImageBitmap offers an opportunity to provide exactly the desired image decoding options when the image is fetched, so that when the ImageBitmap is ready, it is in exactly the desired form for upload to the GPU.<br />
<br />
=== Benefits ===<br />
<br />
:With the right API support, image resources can be prepared asynchronously and/or in advance to avoid rendering latency issues. Furthermore, the preparation steps could be performed by native code, and in many case they can be integrated directly into the image decoding step without any additional memory consumption and little or no additional computational burden.<br />
<br />
=== Requests for this Feature ===<br />
<br />
* <cite>[http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-June/039793.html June 2013 whatwg thread]</cite><br />
* <cite>[http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2013-July/039983.html July 2013 whatwg thread]</cite><br />
<br />
== Proposed Solutions ==<br />
<br />
=== ImageBitmap Construction Options ===<br />
<br />
:Add an optional ''options'' argument to createImageBitmap to specify image treatments to be applied when creating the ImageBitmap object.<br />
<br />
==== Strongly desired options ====<br />
<br />
* boolean flipY, default = false <blockquote><p>If set to true, the image is flipped vertically</p></blockquote><br />
* boolean colorspaceConversion, default = true <blockquote><p>If set to false, the ImageBitmap must be created by decoding the source image media without performing any colorspace conversions. This means that the image decoding algorithm must ignore color profile meta-data embedded in the image source as well as the display device color profile. colorspaceConversion only applies when creating ImageBitmaps from bitmap media data (i.e. Blobs and Image elements)</p></blockquote><br />
* boolean premultiplyAlpha, default = true <blockquote><p>If set to false, the image is decoded without multiplying color components by the alpha component. Only applies when creating ImageBitmaps from bitmap media data. Alpha multiplication is never performed on other types of image sources </p></blockquote><br />
<br />
==== Other potentially useful options ====<br />
<br />
* frameIndex: for creating an image bitmap from multi-frame image media<br />
* flipX: like flipY but horizontal<br />
* colorspace: to decode images to a colorspace other than the device colorspace (sRGB, Adobe RGB, YUV, YCbCr, etc.)<br />
* custom color tranforms: color matrix, 3D Look up tables, per component look up tables, bias, gain, ... <br />
<br />
==== Processing Model ====<br />
: TBD ''Explanation of the changes introduced by this solution. It explains how the document is processed, and how errors are handled. This should be very clear, including things such as event timing if the solution involves events, how to create graphs representing the data in the case of semantic proposals, etc.''<br />
<br />
==== Limitations ==== <br />
: TBD ''Cases not covered by this solution in relation to the problem description; other problems with this solution, if any.''<br />
<br />
==== Implementation ==== <br />
: TBD ''Description of how and why browser vendors would take advantage of this feature.''<br />
<br />
==== Adoption ==== <br />
* Easier to develop asychronous image resource loading and preparation<br />
* Better performance that what can be implemented using current APIs<br />
* Applications can rely on the browser resource management facilities for caching decoded image data with the desired options baked-in and for evicting pre-decoded resources in low memory conditions to avoid out-of-memory crashes.<br />
<br />
[[Category:Proposals]]</div>Kenrussell