[patch][ccla]Support pixel access api's on Canvas renderer

[Chris Mitchell]

Support CanvasPixelArray? and ImageData? in new Canvas extensions module

[Patrick Ruzand]

My concern is how to translate this ticket in term of gfx api (how to support pixel access api).

Basically, as I see it, the ImageData? API allows the dev to do some post-processing on the current drawing, either offscreen (capture, etc.) or onscreen (kind of "filter" effects). In these scenarii, it means the dev needs to invoke the ImageData? api (context.getImageData, context.putImageData, -and context.createImageData to some extend) once the surface has been drawn (or even maybe when it's about to be drawn), and have access to the context. Moreover, keep in mind that the canvas renderer draws the surface asynchronously. That is, a change in the gfx scene marks the surface as dirty, and sets a timer to trigger a render().

So, with that in mind, a possible solution would be to provide a Surface.getImageData() (maybe putImageData as well) method. This method would returns the result of context.getImageData(), and would ensure that the drawing is up to date (if there's a pending render request, it should make sure to create the image data once the pending request has been processed).

Another solution would be to provide 2 generic extension points Surface.beforeRender(context)/Surface.afterRender(context). It would allow the developer to do his pre/post processing on the current drawing (in sync with the rendering), having the context2D in parameter. The benefit of this solution is it's generic, and not limited to the imagedata api. The drawback is if the developer wants to access the canvas context outside of the rendering loop, he still can't.

To raise this limitation, we could think to add a Surface.getContext2D() method that the dev could invoke whenever he wants, free to him to call it when the surface is rendered (that is, ensure the drawing has been done). But if this method is available, I tend to think the previous solution (that is beforeRender/afterRender) may not be needed anymore, as dojo/aspect allow to do that with a "standard" dojo pattern.

Any thoughts, other ideas ?

[Patrick Ruzand]

Some comments after discussing this face to face with Christophe. Because of the delayed rendering implmentation and the fact that ctx.putImageData and ctx.getImageData are quite 2 differents use cases (putImageData needs to be done each time a rendering is done while getImageData must be done on demand in sync with the rendering loop), a good solution would be a mix of the above solutions:

• add a public Surface.render(context) method. This method takes the context as parameter, and document clearly that a aspect.before/after could be done in case the dev need to plug in the rendering loop some pre/post-precessing tasks (as putImageData() for ex).
• modify accordingly the private _render() method so that it delegates to the new render(ctx) method passing the context it has created internally.
• add a public Surface.getImageData() that would return a Deferred. getImageData() would trigger a rendering, and call the Deferred callback when rendering is done.

[Patrick Ruzand]

patch by pruzand (IBM,CCLA)

[Patrick Ruzand]

[summary of offline discussions with Eugene] The problem with this solution is it introduces asynchronous operations in the API to cover up an implementation details. Another solution to deal with this async impl. of the renderer would be to flush the pending render queues before calling the ctxt.getImageData() method. Pros are it exposes a cleaner (and a naming consistent with what does this method) api. The drawback is that it does not handle directly the case where there are pending images downloads. Indeed, in this case, forcing a _render will still miss the images. But a workaround could be to do a 1-time connect to the render() loop to be invoked when every shapes are there.

[Patrick Ruzand]

update patch to match latest discussion

[cjolif]

In [28377]:

refs #13951. #13951 will introduce a render method on canvas surface. The chart was looking for a render method to see if it is here and can call it. Remove that call because it will now fail with the new method coming in. Chart is working perfectly without this in canvas (obviously as this was dead code). This piece of code might be inherited from a time were such a call was needed.

[Patrick Ruzand]

In [28378]:

add pixel buffer access (getImageData()), fixes #13951

[Patrick Ruzand]

In [28379]:

shows gfx clipping capabilities, initial revision, refs #13951