Canvas Context Loss and Restoration

2D Canvas Rendering contexts are currently required to have persistent backing stores. This proposal aims to relax that requirement by introducing an API to allows canvases to be discarded by the browser and re-drawn by the web application on demand.

Use Case Description

The 2d canvas backing store persistence requirement often leads to large amounts of RAM (or GPU memory) being consumed by canvas elements that are not actively used because they are off screen or in background tabs or occluded windows. Other types of RAM-greedy HTML elements can release resources in such cases.

The expectation of canvas content persistence also makes it very difficult--if not impossible--for many web apps to recover from a GPU context reset in web browsers that store 2D canvas contents in GPU memory.

Current Limitations

In theory, web apps do have the capability of discarding canvas backing stores (set canvas size to zero) and regenerating canvas content. However, web apps are not and should not be expected to be responsible for resolving resource contention issues. The browser is responsible for monitoring resource usage and availability and is expected to take all necessary and reasonable measures to avoid crashes, hangs, and catastrophic performance degradations that may be caused by resource contention. Under the current specification, browsers have no options for evicting resources held by 2D canvases because there are no means of guaranteeing that the application will redraw the contents when needed.

Current Usage and Workarounds

Web apps can track events to detect when the page is no longer visible (http://www.w3.org/TR/page-visibility/) and deallocate backing stores at that time by setting the size of the canvas element to 0. Conversely, they can detect when the page is visible again and reinitialize at that time

Web apps can track events that are often associated with GPU context losses (e.g. waking-up from hibernation), and conservatively reinitialize the 2D canvas by resetting the context (set canvas width/height) and redrawing, just in case.

Benefits

• Empower the browser to monitor resources to decide whether to drop canvas backing stores and in what order (LRU backgroung tabs?) in order to achieve better performance and stability. If web apps must handle resource eviction themselves, they may often free resources when not necessary, which may lead to unnecessary tab switching lag.
• Make recovery from GPU context losses more robust.
• Allow GPU-accelerated 2D canvases on platforms that are known to drop graphics contexts often or unpredictably.

Requests for this Feature

• E-mail thread on Chromium graphics-dev mailing list

  "Is there any reason why we don't add a similar optional callback to the 2D context? (in reference to WebGL context loss API)" --Rik Cabanier, Adobe

Proposed Solutions

The following solutions were pondered in the discussion thread cited above:

• Generalize the WebGL context lost / context recovered API, so that it applies to all types of canvases.
• Add a redraw callback on the canvas element

Retained Solution : Upstream the context lost/recovered API form the WebGL specification into the parent canvas specification.

General Concept:

• a renderingContextLost event is fired after the context is lost.
• a renderingContextRestored event is fired immediately after a previously lost canvas context is brought back to a usable state. The canvas context is returned to its initial state and the canvas's backing store is blank (transparent black) when restored.

Processing Model

Rendering context losses may be intended by the user agent (to resolve resource contention), or may be forced by external factors (e.g. a graphics driver reset).

For convenience, the lost state should be accessible. To do so, the isContextLost method that is defined in the WebGLRenderingContext API should also exist in the CanvasRenderingContext2D API.

Losing contexts (applies to 2d contexts, some aspects different for WebGL):

• Intentional losses are only allowed if the contextRestored event is handled on the canvas element associated with the context.
• The UA is free to use any set of rules to decide which contexts are dropped when and in what order.
• The return value of isContextLost() may transition from false to true before the contextLost event is dispatched (like webGL).
• All objects that depend on the content of the canvas (e.g. patterns, imageBitmaps) are neutered when the context is lost. The neutering propagates through creation dependency chains, so a Pattern created from an ImageBitmap created from and ImageBitmap created from a canvas will be neutered if the canvas's context is lost. This rule makes it safe for implementations to optimize their memory consumption by sharing pixel buffers between objects when possible.

Restoring contexts (applies to 2d contexts, some aspects different for WebGL):

• Between the time a context is recovered and invoking the listener for the contextRestored event, no other user code can be executed.
• The app is responsible for re-creating the objects that were neutered when the context was lost.
• There can only be one contextRestored event pending at a time. When there are multiple canvases to be restored, the next canvas to be restored can only be restored after any pending contextRestored events--from previously restored canvases--have been handled.
• Context restoration can only be initiated by the user agent (can not by triggered by a script action).
• A lost context can only be restored after its context lost event has been dispatched. This avoids synchronization inconsistencies with isContextLost().
• The return value of isContextLost() transitions from true to false at the time that contextRestored event is dispatched (like webGL). So the contextRestored event listener is always the first task to be execute after the transition.

Behavior when using a lost context (applies to 2d contexts, WebGL may behave differently):

• All draw calls exit without drawing.
• All rendering context API calls will throw the same exceptions as they would if called on a valid context.
• All calls that read back canvas pixels from either the canvas element or the canvas rendering context (getImageData, toDataURL, toBlob, createPattern, creatImageBitmap, drawImage with lost canvas as source) will behave as they would if the canvas context were valid and blank (all transparent black pixels).
• Using a Patterns or an ImageBitmap that was neutered because the context of its source canvas was lost will behave as if the Pattern or ImageBitmap were valid and blank (all transparent black pixels).

Behaviors specific to GPU-accelerated implementation of 2d canvas (non-normative?)

• If the context was lost due to a GPU-related failure, and the browser is actively restoring GPU functionality, and expects to restore in a timely manner, then context restoration should wait until the browser is ready to resume GPU functionality and the restored canvas should continue to be GPU-accelerated. Conversely, if GPU functionality is permanently disable or if it is unknow whether or how long it may take to resume GPU operation, then the canvas should be restored immediately without GPU-acceleration
• If an accelerated canvas is resized while the GPU is temporarily unavailable, the creation of the new canvas buffer should be postponed until the GPU functionality is restored.
• When getContext() is called while GPU functionality is temporarily unavailable:

a) If the canvas does not have an associated context, create an new unaccelerated context.

b) If the canvas already has an associate context, return the existing context even if it is in a lost state.

Limitations

Web browsers will not reap the stability and performance rewards associated with this API with Web apps that do not provide at least a handler for the context recovered event. In order for this feature to improve the state of the web, apps will need to opt in to this new API, which unfortunately needs to be optional for backwards compatibility reasons.

Implementation

Browser vendors should be highly motivated to implement this API in order to improve platform resilience and performance, especially on mobile platforms where RAM contention is often a critical issue.

Adoption

Web App developers should be motivated to adopt this API in order to improve the stability of their products:

• Avoid triggering out-of-memory crashes in the browser.
• Avoid browser performance issues associated with having a given app in a background tab
• Robustly recover from GPU failures.

Top tier apps (particularly from developers that track usage metrics) would be expected to enthusiastically adopt this API.

Specification

WebGL would continue to behave as currently specified (http://www.khronos.org/webgl/wiki/HandlingContextLost), but the wording of the WebGL specification could be modified to refer to the parent canvas specification and would only specify differences in behavior with respect to 2d canvas.

Open Issues

• Should there be a base CanvasRenderingContext interface that defines isContextLost and is inherited by both WebGLRenderingContext and CanvasRenderingContext2D?