Canvas Batch drawImage

Batching drawImage calls to achieve near-native performance for sprite/image based animations and games.

Use Case Description

Many web applications and games use 2D Canvases and the drawImage method to bring sprites to screen. Often, a large number of calls to drawImage occur for each frame presented to screen.

Current Limitations

When calling drawImage hundreds or thousands of times per animation frame, API bindings overhead and internal bookkeeping costs associated with individual draw calls become a significant performance bottleneck, which prevents web applications from achieving near-native performance levels.

Current Workaround

With WebGL, batching sprite draws is possible thanks to vertex buffers. WebGL is not as broadly supported as 2D canvas, and is a considerably more complex solution to a problem that could otherwise be solved with a 2D canvas.

Benefits

Rendering Performance.

Requests for this Feature

• [1]

  On Nexus 7 device, it can reach 60fps for 1000 sprites drawing at the same time [referring to a native canvas implementation]. However, the fps is very poor (about 9fps) if the demo is running in Chromium 36.

Proposed Solution

Batch versions of drawImage

New batch variants of drawImage that accept array arguments.

Processing Model

We would have new variants of the existing drawImage method where the numerical arguments are are packed into a Float32Array. The image argument may or may not be an array. If it is not an array, the same source image is used for for each draw. When rendering from a single sprite sheet, it would be preferable to not specify the image argument as an array in order to minimize bindings overhead and redundant parameter validation.

Suggested IDL

enum CanvasDrawImageParameterFormat { position, destination-rectangle, source-and-destination-rectangles, source-rectangle-and-transform};

interface CanvasRenderingContext2D {
    ...
    void drawImageBatch(sequence<CanvasImageSource> image, CanvasDrawImageParameterFormat parameterFormat, Float32Array drawParameters);
    void drawImageBatch(CanvasImageSource image, ParameterFormat parameterFormat, Float32Array drawParameters);
}

The drawParameters argument is to be interpreted as a table in row-major order. Where each row represents a single draw and each column represents an individual parameter. The mapping of the columns to draw parameters depends on the parameterFormat argument.

• position: dx, dy
• destination-rectangle: dx, dy, dw, dh
• source-and-destination-rectangles: sx, sy, sw, sh, dx, dy, dw, dh
• source-rectangle-and-transform: sx, sy, sw, sh, a, b, c, d, e, f

The parameters sx, sy, sw, sh, dx, dy, dw, and dh have the same meaning as with drawImage()

The parameters a, b, c, d, e, f have the same meaning as with transform()

With 'source-rectangle-and-transform' the destination rectangle is a unit square defined by the vertices (0, 0), (1, 0), (1, 1), and (0, 1). The destination rectangle is transformed by the transform defined by a, b, c, d, e and f, and the by the canvas's current transform.

Feedback Anne: Perhaps rather than overloading we should introduce new methods? IDL overloading is somewhat costly and not loved much by the JS community. Also, please float this by [email protected] at some point.

Feedback Jonas: Yeah, something like drawImagePositionBatch/drawImageDestRectBatch/drawImageSrcAndDestRectBatch/etc will make feature detection of future expansions easier. See [2]

Exceptions

An INDEX_SIZE_ERR DOM exception is thrown if the size of drawParameters is not a multiple of the number of numeric parameters corresponding to 'parameterFormat'

An INDEX_SIZE_ERR DOM exception is thrown if 'image' is a sequence and the size of drawParameters is not equal to the number of elements in 'image' multiplied by the number of parameters corresponding to 'parameterFormat'.

All the same exceptions that apply for calls to drawImage also apply to drawImageBatch. If any individual draw results in an exception being thrown, the entire call to drawImageBatch must abort without drawing anything to the canvas.

Limitations

The use of Float32Array is not very programmer friendly. This compromise on usability is justified by performance considerations, and is necessary to achieve near-native or WebGL-like performance.

Implementation

• The most naive implementation, which would consist in expanding the batch drawImage call into multiple internal drawImage calls would already increase performance by reducing API bindings overhead.
• The use of typed arrays will dramatically reduce the argument type checking burden in the bindings.
• More advanced implementations would carry the batching down to a lower level in the graphics stack. For example, a GPU-accelerated implementation of 2D canvas could use OpenGL vertex buffer objects, or the DirectX sprite interface.
• Some existing implementations of drawImage already detect batching opportunities and will batch consecutive drawImage calls at a lower lever of the graphics stack (for example skia's drawBitmap used in Blink). This auto-detection does improve rasterization performance, but it does not eliminate the bindings overhead and it involves additional overhead to determine whether consecutive calls to drawImage can be grouped together to form a batch. With explicitly batched calls to drawImage, that overhead can be eliminated because the individuals sprite draws would be known to use identical rendering context state, and the calls down to the graphics platform implementation layer would only occur once for the entire batch.

Adoption

Game/app devs looking for high frame rates for sprite blitting are likely to adopt enthusiastically.