OffscreenCanvas.requestAnimationFrame - Revision history

Kenrussell: Added note indicating this is obsoleted

2019-05-10T17:26:48Z

Added note indicating this is obsoleted

← Older revision		Revision as of 17:26, 10 May 2019
Line 1:		Line 1:
	:''This proposal aims to provide a reliable mechanism for driving animations using OffscreenCanvas in a Worker''		:''This proposal aims to provide a reliable mechanism for driving animations using OffscreenCanvas in a Worker''


			== NOTE: OBSOLETED ==

			The proposal below was [https://html.spec.whatwg.org/multipage/imagebitmap-and-animations.html#animation-frames refined and specified] as requestAnimationFrame on the DedicatedWorkerGlobalScope. This page is maintained only to maintain the history of the discussion.


	== Use Case Description ==		== Use Case Description ==

Junov: Adding the commit solution to the proposal

2017-01-18T17:11:34Z

Adding the commit solution to the proposal

@@ Line 27: / Line 27: @@
 </p></blockquote>
-== Proposed Solution ==
+== Proposed Solutions ==
 === OffscreenCanvas.requestAnimationFrame ===
@@ Line 65: / Line 65: @@
 Should it be possible to commit() the contents of other canvases from within a rAF callback?
-==== Adoption ====
+=== OffscreenCanvas.commit() to return a promise ===
-:''There is no other way to guarantee optimal animation smoothness and low latency while not wasting CPU cycles on overdraw, so this API should be the method of choice for driving OffscreenCanvas.commit()''
 [[Category:Proposals]]

Junov: Created page with ":''This proposal aims to provide a reliable mechanism for driving animations using OffscreenCanvas in a Worker'' == Use Case Description == An OffscreenCanvas is used in a wo..."

2016-12-02T18:58:14Z

Created page with ":''This proposal aims to provide a reliable mechanism for driving animations using OffscreenCanvas in a Worker'' == Use Case Description == An OffscreenCanvas is used in a wo..."

New page

:''This proposal aims to provide a reliable mechanism for driving animations using OffscreenCanvas in a Worker''

== Use Case Description ==
An OffscreenCanvas is used in a worker to produce a sequence of frames that constitute an animation. The frames need to be produced at regular intervals that match the frame rate of the display

=== Current Limitations ===

=== Current Usage and Workarounds ===
Currently, it is possible to use setTimeout or setInterval in a worker to invoke an animation callback at a regular interval. However, since this mechanism is not driven by the display, the following issues arise:
* The display device's refresh rate needs to be guessed.
* Even if the correct rate is guessed, drift in the timing will cause dropped frames and skipped frames.
* In GPU-accelerated use cases, it is possible for the rendering script to run at a rate that the GPU cannot keep-up with, which may cause the accumulation of a multi-frame rendering backlog, resulting in high latency and eventually OOM crashes. A possible solution to this is for the user agent to implement a throttling mechanism, in which case the worker's event loop may be periodically de-scheduled while the GPU catches up. Such de-scheduling is bad because it prevents the worker from doing other work.

Another solution is to have a requestAnimationFrame loop in the browsing context's event loop that posts a message to the worker at each animation iteration.
* This mechanism may add undue latency to the signal, especially when the browsing context's event loop is busy, which completely destroys one of the key advantages of using OffscreenCanvas in a worker.
* The the frame rate in the browsing context's event loop may be higher than the worker can keep up which which will require a throttling mechanism to be implemented in script
* As with setTimeout/setInterval, it is possible for the rendering script to run at a rate that the GPU cannot keep-up with.

=== Benefits ===

New API to drive OffscreenCanvas animations can ensure optimal frame rate, minimize animation jank, prevent overdraw, and minimize display latency.

=== Requests for this Feature ===

* <cite>[https://github.com/w3ctag/spec-reviews/issues/141 W3C TAG review for ImageBitmapRenderingContext]</cite><blockquote><p>(...) there seems to be a good bit of concern about lack of things like requestAnimationFrame (...) -- L. David Barron, Mozilla</p></blockquote>
* <cite>[https://lists.w3.org/Archives/Public/public-whatwg-archive/2015Aug/0019.html whatwg mailing list thread "Worker requestAnimationFrame"]</cite> <blockquote><p>For OffscreenCanvas we need a way for a Worker to draw once per composited frame. --Robert O'Callahan, Mozilla
</p></blockquote>

== Proposed Solution ==

=== OffscreenCanvas.requestAnimationFrame ===

Works much like window.requestAnimationFrame, except that the scheduling of callbacks is independent of the browsing context event loop, and therefore is not necessarily synchronized with graphics updates from the browsing context.

==== Processing Model ====

The requestAnimationFrame() and cancelAnimationFrame() methods shall be spec'ed almost identically to their Window interface counterparts, except that the callback list would be stored in the OffscreenCanvas object.

The main difference with respect to the Window.requestAnimationFrame processing model is in how the callbacks are scheduled. In a browsing context, the animation callbacks are coordinated with the graphics update in the [https://html.spec.whatwg.org/multipage/webappapis.html#event-loop-processing-model event loop processing model]. Such coordination shall not exist in workers.

For OffscreenCanvases, the user agent will schedule an "animation task" to run all of the OffscreenCanvas's pending animation callbacks. in a way the respects the following constraints:
* No overdraw: An animation frame committed from an animation task shall not replace an animation frame from a previous animation task until that previous frame has been rendered to the display device.
* animation tasks shall be scheduled at the highest possible rate that can be maintained without going into overdraw and without accumulating a backlog of more than one pending frame.

===== Special Cases =====

* When rAF is invoked on an OffscreenCanvas that does not have a placeholder canvas and is not linked to a VRLayer, throw an InvalidStateError. The OffscreenCanvas content must be composited for the rAF processing model to make sense.
* Attempting to transfer an OffscreenCanvas object with a non-empty animation callback list throws an InvalidStateError.
* Attempting to construct a VRLayer using an OffscreenCanvas object with a non-empty animation callback list throws an InvalidStateError.
* When the OffscreenCanvas is associated with a VRLayer, all calls to {request|cancel}AnimationFrame must be forwarded to the VRLayer's VRDisplay's {request|cancel}AnimationFrame methods. This implies that when the OffscreenCanvas simultaneously is visible through a placeholder canvas and a VR device, the animation loop is driven by the VR device.
* The animations tasks for different OffscreenCanvas objects that live in the same event loop are not necessarily synchronized.

==== Open issues ====
Calling commit() on a given OffscreenCanvas multiple times in the same animation frame is problematic. Possible way of handling the situation:
* Drop all commits but the last one (or the first one?)
* Queue multiple frames and wait for all of them to have be displayed before scheduling the next animation task
* Throw an exception

What to do if commit() is not called from within the animation callback? This is problematic because
* Do an implicit commit()
* Repeat the previous frame
* Schedule the next animation frame immediately.
* Prevent this from ever happening: Let OffscreenCanvas object have a needsCommit flag that is initially false. Set needsCommit to true at the beginning of an animation task. Set needsCommit to false when commit is called. When requestAnimationFrame is called, throw an exception if needsCommit is true.

Should it be possible to commit() the contents of other canvases from within a rAF callback?

==== Adoption ====
:''There is no other way to guarantee optimal animation smoothness and low latency while not wasting CPU cycles on overdraw, so this API should be the method of choice for driving OffscreenCanvas.commit()''

[[Category:Proposals]]