Download
1 / 77
E N D
1. Deep Dive into HTML5 Canvas Jatinder Mann
Program Manager
Internet Explorer
2. Who am I…
3. Have you heard of Canvas?
4. Have you coded with Canvas?
5. Have You Deployed a Canvas Site?
6. Deep Dive
7. What is Canvas?
8. What is Canvas? HTML5 element that allows for dynamic, scriptable rendering of 2D shapes and bitmaps
Immediate mode rendering
Simple API: 45 methods, 21 attributes
9. state
void save();
void restore();
transformations
void scale(…);
void rotate(…);
void translate(…);
void transform(…);
void setTransform(…);
compositing
globalAlpha;
globalCompositeOperation;
colors and styles
strokeStyle;
fillStyle;
CanvasGradient createLinearGradient(…);
CanvasGradient createRadialGradient(…);
CanvasPattern createPattern(…);
Line caps/joins
attribute double lineWidth;
attribute DOMString lineCap;
attribute DOMString lineJoin;
attribute double miterLimit;
Shadows
attribute double shadowOffsetX;
attribute double shadowOffsetY;
attribute double shadowBlur;
attribute DOMString shadowColor;
Rects
void clearRect(…);
void fillRect(…);
void strokeRect(…);
path API
void beginPath();
void closePath();
void moveTo(…);
void lineTo(…);
void quadraticCurveTo(…);
void bezierCurveTo(…);
void arcTo(…);
void rect(…);
void arc(…);
void fill();
void stroke();
void clip();
boolean isPointInPath(…);
focus management
boolean drawFocusRing(…);
drawing images
void drawImage(…);
text
attribute DOMString font;
attribute DOMString textAlign;
attribute DOMString textBaseline;
void fillText(…);
void strokeText(…);
TextMetrics measureText(…);
pixel manipulation
ImageData createImageData(…);
ImageData createImageData(…);
ImageData getImageData(…);
void putImageData(…);
interface CanvasGradient {
void addColorStop(…); };
interface CanvasPattern {};
interface TextMetrics {
readonly attribute double width; };
interface ImageData {
readonly attribute unsigned long width;
readonly attribute unsigned long height;
readonly attribute CanvasPixelArray data; };
interface CanvasPixelArray {
readonly attribute unsigned long length;
getter octet (…);
setter void (…); }; Here is the entire API!
10. What is Canvas? Remember: Canvas is stupid.
It does not remember what you drew last.
11. demo
13. Drawing Model
14.
Let’s create a yellow smiley face using bezier curves, give it a shadow, 50% transparency, circle clip and source-over composition (default) Example: Yellow Smiley Drawing Model
15. Example: Yellow Smiley Drawing Model – Specification description
16. Example: Yellow Smiley Drawing Model – IE9 Implementation
17.
LESSON LEARNED: Not all operations were created equal. Some are more expensive than others.
Shadows, clipping and composition operations are more expensive as they require an additional intermediate.
DON’T: Create thousands of small objects with clipping, shadows or compositions effects.
Tip #1: Shadows, Clipping and Compositions Drawing Model
18.
Canvas is like a kid with a crayon.
Canvas defines 11 composition operations.
Based on Compositing Digital Images, by Porter-Duff (1984) What are Composition Operations?
19. Composition Operations demo
20.
Firefox, Opera and Internet Explorer follow the drawing model as described in the spec.
Safari and Chrome have a slightly different drawing model behavior.
DO: Check if you’re composition is impacted by this interoperability issue.
Tip #2: Interoperable Compositions Drawing Model
22.
All Canvas attributes are global – they effect all operations drawn next.
Saving and Restoring Drawing Model ctx.shadowOffsetX = 10;
ctx.shadowOffsetY = 10;
ctx.shadowBlur = 10;
ctx.shadowColor = "rgba(0, 0, 0, 1)";
ctx.fillText(“Winning!”, 10, 25);
// All future operations will also have a shadow
ctx.strokeRect(0, 0, 115, 40);
23. Saving and Restoring Drawing Model You can save and restore:
The current transformation matrix.
The current clipping region.
The current values of the following attributes: strokeStyle, fillStyle, globalAlpha, lineWidth, lineCap, lineJoin, miterLimit, shadowOffsetX, shadowOffsetY, shadowBlur, shadowColor, globalCompositeOperation, font, textAlign, textBaseline.
24.
Use save() and restore() to scope the attribute
Saving and Restoring Drawing Model ctx.save();
ctx.shadowOffsetX = 10;
ctx.shadowOffsetY = 10;
ctx.shadowBlur = 10;
ctx.shadowColor = "rgba(0, 0, 0, 1)";
ctx.fillText(“Winning!”, 10, 25);
ctx.restore();
// All future operations will NOT have a shadow
ctx.strokeRect(0, 0, 115, 40);
25. Interoperability
26. Interoperability Good news: There is high level of Canvas support on all modern browsers!
27. Philip Taylor’s Test Suite Based on Philip Taylor’s Canvas 2D Context test suite:
Reference: http://philip.html5.org/tests/canvas/suite/tests/
28. Interoperability
29. Security Model
30. Security Model Canvas allows reading and writing pixels to the screen
You can also save the Canvas to a file via toDataURL()
31. Security Model Canvas also allows drawing cross-domain images/videos
However, it has built-in security protection against third-parties saving cross-domain pixels
32. Security Model Canvas protects against Information Leakage attacks
33. Scenario: My Bank Security Model
34. Scenario: My Bank Security Model
35. Scenario: My Bank Security Model
36. Scenario: My Bank Security Model
37. Security Model Canvas has a concept of origin-clean flag.
origin-clean is set false if a cross-domain IMG or VIDEO is used.
A SECURITY_ERR exception is raised, if one tries to save the Canvas pixels if the Canvas origin-clean is false:
toDataURL()
getImageData()
38. Example: SECURITY_ERR Exception Security Model
var image = new Image();
image.onload = function() {
ctx.drawImage(image, 0, 0);
// At this point, image would be drawn
// origin-clean is now false
// SECURITY_ERR exception raise
var data = canvas.toDataURL();
};
image.src = “http://other-domain/picture.png”;
39. Accessibility
40. Accessibility Some context
When we started developing IE9, there was no standard for making the canvas element accessible.
The IE team contributed to a proposal in the W3C for canvas accessibility
IE9 Beta was the first browser to add support for the underlying canvas DOM.
41. Fallback Content Focus (spec stable)
When the Canvas has embedded content, it can be keyboard-focusable via accessibility tools.
IE9 is only browser currently supporting this feature
Focus Management (spec unstable)
A focus ring can be drawn on the Canvas corresponding to embedded content in the Canvas fallback.
No browser currently supports this feature
Canvas Accessibility Features
42. Fallback Content Focus
Access to the DOM tree within <canvas></canvas> tags
Recall, elements within the <canvas></canvas> tags are considered Fallback content and are only displayed on screen when Canvas is not supported.
This same Fallback area can be used for Accessibility description of what is on the Canvas
Canvas Accessibility Features
43. Accessibility demo
44. Best Practices with Canvas
45. Tip #3: Feature detection Best practices with Canvas DO: Canvas feature detection code
DON’T: Browser detection using User Agent string
DON’T: Conditional comments
46. Tip #3: Feature detection Best practices with Canvas var canvas = document.createElement(“canvas”);
if (canvas && canvas.getContext &&
canvas.getContext(“2d”))
{
// Code requiring canvas support
} else {
// Canvas isn’t available.
// Put non-canvas fallback here
}
47. Tip 4: Always add a DOCTYPE Best practices with Canvas DO: Add a DOCTYPE
Add W3C HTML5 DOCTYPE:
<!DOCTYPE HTML>
Or add common strict DOCTYPE:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
48. Tip #4: Always add a DOCTYPE Best practices with Canvas Canvas is only supported in IE9 mode
IE Dev Tools (F12) will show your Doc mode:
49. Performance Best Practices
50. Hardware Accelerated HTML5
51. Tip #5: Make Fewer Calls to Video Memory Best practices with Canvas Reading and writing to video memory is slow.
DON’T: Repeatedly call getImageData() or putImageData() for small amounts of data.
DO: Request larger area of data with fewer calls.
52. Tip #5: Make Fewer Calls to Video Memory Best practices with Canvas for (var y = 0; y < height; y++) {
for (var x = 0; x < width; x++) {
// Don’t make multiple calls for
// small amounts of data
var input = ctx.getImageData(x, y, 1, 1);
var inputData = input.data;
var r = inputData[i+0];
var g = inputData[i+1];
var b = inputData[i+2];
…
53. Tip #5: Make Fewer Calls to Video Memory Best practices with Canvas // Do make fewer calls with larger amounts of data
var input = ctx.getImageData(0, 0, width, height);
var inputData = input.data;
for (var y = 0; y < height; y++) {
for (var x = 0; x < width; x++) {
var r = inputData[i+0];
var g = inputData[i+1];
var b = inputData[i+2];
54. Tip #6: Cache Pixel Array Data Best practices with Canvas Accessing ImageData.data[i] requires DOM access to fetch and index the object
DON’T: Repeatedly access ImageData.data[i] for data.
DO: Cache the ImageData.data in a Javascript object.
55. Tip #6: Cache Pixel Array Data Best practices with Canvas var imageData = ctx.getImageData(0, 0, width, height);
for (var y = 0; y < height; y++) {
for (var x = 0; x < width; x++) {
// Do not repeatedly access ImageData.data[i]
var r = imageData.data[i+0];
var g = imageData.data[i+1];
var b = imageData.data[i+2];
56. Tip #6: Cache Pixel Array Data Best practices with Canvas var imageData = ctx.getImageData(0, 0, width, height);
// Cache the CanvasPixelArray in a JS Object
var inputData = imageData.data;
for (var y = 0; y < height; y++) {
for (var x = 0; x < width; x++) {
var r = inputData[i+0];
var g = inputData[i+1];
var b = inputData[i+2];
57. Tip #7: Canvas objects as D3D textures Best practices with Canvas Canvas objects and patterns are implemented as D3D textures.
DON’T: Create new Canvas objects/patterns every frame.
DO: Re-use existing Canvas objects/patterns.
58. Tip #8: Canvas objects as D3D textures Best practices with Canvas DON’T: Clear Canvas by setting dimensions every frame.
DO: Call clearRect() to clear the Canvas.
59. Tip #9: Drawing the Canvas to itself Best practices with Canvas Re-using a Canvas object is faster than creating an intermediate texture.
DON’T: drawImage() a Canvas to itself, as this requires creating intermediate texture.
DO: Use multiple Canvas objects instead.
60. Tip #10: Avoid clearRect() with Transforms Best practices with Canvas AVOID: Setting a geometric clip, rotation or skew transform when calling clearRect() – it will be much slower.
AVOID: Using clearRect() when clearing paths on non-whole pixel boundaries.
61. Tip #11: Avoid changing attribute settings Best practices with Canvas Avoid setting attributes unnecessarily
While costs are usually small, avoiding all unnecessary DOM calls will help in high performance applications.
Some attributes are slower to set than others.
62. Tip #12: Memory Management Best practices with Canvas DON’T: Hold references to objects that cannot be garbage collected
Will cause associated resources to be held in memory
DON’T: Unintentionally Cache graphics objects in globals or expandos
Objects such as canvas, pattern, image data, and gradients
63. Tip #13: SetTimeOut() and SetInterval() Best practices with Canvas Monitors typically display at 60Hz or 16.7ms periods.
For graphics timers, no point in using a higher resolution
64. Tip #13: SetTimeOut() and SetInterval() Best practices with Canvas DO: Use a 17ms interval for setTimeOut() and SetInterval(). They take integer values – round up 16.7ms.
DON’T: Use a 1ms interval for timers
Using a smaller interval than 17ms has no benefit: it results in choppy animation and reduced battery life
65. When to use Canvas?
66. HTML5 Graphics: SVG and Canvas When to use Canvas?
HTML5 specifies two graphics models: SVG and Canvas.
Almost any vector graphic can be drawn with either technology, however, with varying degrees of effort.
67. HTML5 Graphics: SVG and Canvas When to use Canvas? Canvas can draw SVG, images, videos and other Canvases on Canvas.
You can use Canvas and SVG in the same application complimenting each other.
68. SVG Basics When to use Canvas?
SVG describes “Scalable Vector Graphics”.
Retained mode rendering.
Like HTML, SVG is built into the document using elements, attributes and styles.
69. High level differences: SVG and Canvas When to use Canvas?
70. Scenarios: Canvas and SVG When to use Canvas?
71. Call to Action
72. Call to Action CALL #1: You’ve built applications in HTML and Flash, try building applications with Canvas.
CALL #2: The future is Hardware Accelerated HTML5 Canvas. Start developing Canvas sites on IE9 to take advantage of GPU.
73. Questions
74. Fill Out Evaluation Forms
75. Internet Explorer MIXtery MEGA Box!
