Canvas Batch drawImage

Batching drawImage calls to achieve near-native performance for sprite/image based animations and games.

Use Case Description

Many web applications and games use 2D Canvases and the drawImage method to bring sprites to screen. Often, a large number of calls to drawImage occur for each frame presented to screen.

Current Limitations

When calling drawImage hundreds or thousands of times per animation frame, API bindings overhead and internal bookkeeping costs associated with individual draw calls become a significant performance bottleneck, which prevents web application from achieving near-native performance levels.

Current Workaround

With WebGL, batching sprite draws is possible thanks to vertex buffers. WebGL is not as broadly supported as 2D canvas, and is a considerably more complex solution to a problem that could otherwise be solved with a 2D canvas.

Benefits

Rendering Performance.

Requests for this Feature

• [1]

  On Nexus 7 device, it can reach 60fps for 1000 sprites drawing at the same time [referring to a native canvas implementation]. However, the fps is very poor (about 9fps) if the demo is running in Chromium 36.

Proposed Solution

Batch versions of drawImage

Additional overloads of drawImage that accept array arguments.

Processing Model

We would have new variants of the existing drawImage overloads where all the arguments are arrays (Float32Array for numeric arguments), and other variants where all the arguments except for the image are arrays. The case where the image is not an array would be equivalent to having an array where each element is the same image (convenient for sprite sheets, and more efficient than using an array)

An INDEX_SIZE_ERR DOM exception would be thrown if not all the array arguments have the same size.

Suggested IDL

interface Canvas2DRenderingContext {
    ...
    // existing variants
    void drawImage(CanvasImageSource image, unrestricted double dx, unrestricted double dy);
    void drawImage(CanvasImageSource image, unrestricted double dx, unrestricted double dy, unrestricted double dw, unrestricted double dh);
    void drawImage(CanvasImageSource image, unrestricted double sx, unrestricted double sy, unrestricted double sw, unrestricted double sh, unrestricted double dx, unrestricted double dy, unrestricted double dw, unrestricted double dh);
    // batch variants
    void drawImage(sequence<CanvasImageSource> image, Float32Array dx, Float32Array dy);
    void drawImage(sequence<CanvasImageSource> image, Float32Array dx, Float32Array dy, Float32Array dw, Float32Array dh);
    void drawImage(sequence<CanvasImageSource> image, Float32Array sx, Float32Array sy, Float32Array sw, Float32Array sh, Float32Array dx, Float32Array dy, Float32Array dw, Float32Array dh);
    // single image batch variants
    void drawImage(CanvasImageSource image, Float32Array dx, Float32Array dy);
    void drawImage(CanvasImageSource image, Float32Array dx, Float32Array dy, Float32Array dw, Float32Array dh);
    void drawImage(CanvasImageSource image, Float32Array sx, Float32Array sy, Float32Array sw, Float32Array sh, Float32Array dx, Float32Array dy, Float32Array dw, Float32Array dh);
}

Feedback Anne: Perhaps rather than overloading we should introduce new methods? IDL overloading is somewhat costly and not loved much by the JS community. Also, please float this by [email protected] at some point.

Limitations

Use cases where per-sprite transforms are *required* are not covered. For example, if sprites are individually rotated. If appropriate, this could be resolved by adding more variants to the proposal (a matrix argument?)

Implementation

• The most naive implementation, which would consist in expanding the batch drawImage call into multiple internal drawImage calls would already increase performance by reducing API bindings overhead.
• The use of typed arrays will dramatically reduce the argument type checking burden in the bindings.
• More advanced implementations would carry the batching down to a lower level in the graphics stack. For example, a GPU-accelerated implementation of 2D canvas could use OpenGL vertex buffer objects, or the DirectX sprite interface.
• Some existing implementations of drawImage already detect batching opportunities and will batch consecutive drawImage calls at a lower lever of the graphics stack (for example skia's drawBitmap used in Blink). This auto-detection does improve rasterization performance, but it does not eliminate the bindings overhead and it involves additional overhead to determine whether consecutive calls to drawImage can be grouped together to form a batch. With explicitly batched calls to drawImage, that overhead can be eliminated because the individuals sprite draws would be known to use identical rendering context state, and the calls down to the graphics platform implementation layer would only occur once for the entire batch.

Adoption

Game/app devs looking for high frame rates for sprite blitting are likely to adopt enthusiastically.