Non-Photorealistic Rendering CS 446: Real-Time Rendering & Game Technology

1. Non-Photorealistic Rendering CS 446: Real-Time Rendering & Game Technology David Luebke University of Virginia

2. Demo • Today: Sean Arietta, Super Mario Sunshine • Thursday: Greg Tylka Real-Time Rendering

3. Assignments • Assn 4 description due tonight at midnight • Remember to post these on forum, not send by e-mail • Assn 4 due Mar 31, integrated demo Mar 23 Real-Time Rendering

4. Non-Photorealistic Rendering “Using a term like ‘nonlinear science’ is like referring to the bulk of zoology as ‘the study of nonelephant animals’” • Photorealism: age-old goal of graphics • Non-photorealistic rendering • Broadly, techniques for rendering that don’t strive for realism, but style, expressiveness, abstraction, uncertainty, etc • A terrible term that we may be stuck with • Better terms: stylized rendering, artistic rendering, abstract rendering, comprehensible rendering, etc. Real-Time Rendering

5. Stylized Rendering • NPR usually used to refer to graphics techniques that emulate a particular artistic style or medium • Impressionistic painting • Pen-and-ink (cross hatching, outlining, etc) • Colored-pencil, copperplate engraving, you name it • Cartoon drawing Real-Time Rendering

6. Toon shading • Toon shading is the term that we use to refer to cartoon-like rendering effects • Tendencies in cartoon/comic style: • Simple, flat shading (cel shading) • Two-tone (light/shadow) or three-tone (light/shadow/highlight) • Edge highlighting • Boundary (border edge) • Crease (hard edge) • Material edge • Silhouette edge Real-Time Rendering

7. Toon Shading • Cel shading • Threshold the lighting equation • Ex: the Cg program we looked at • Diffuse below 0.5 ? dark color : light color • Specular above 0.5 ? hilight color : diffuse color • If no pixel shaders, can use N ● L to look up a 1-D texture containing the shaded colors • In general, can think of these procedures as remapping the traditional lighting equation elements (tone mapping) Real-Time Rendering

8. Silhouette Rendering • Boundary (border edge) • Not shared by two polygons • E.g. the edge of a sheet of paper • Solid models usually have no boundaries • Crease (hard edge) • Shared by two polygons at a dihedral angle greater than some threshold (often 60°) • Or a vertex with two normals/two colocated vertices • Ridge or valley edges Real-Time Rendering

9. Silhouette Rendering • Material edge • Triangles sharing edge have different materials/texture maps/etc • Or just an edge that the artist wants to emphasize • Silhouette edge • Triangles sharing edge face different directions (towards/away from viewer) • Lots of techniques to find at runtime! Real-Time Rendering

10. Finding/Rendering Silhouette Edges • Surface angle silhouetting • Again, like the Cg program we looked at • Calculate N●V, if below threshold  draw black • Problem: depends on surface curvature (see Fig 7.5) Real-Time Rendering

11. Procedural Geometry Silhouetting • Idea: render the geometry in such a way that the silhouettes “fall out” • Ex: • Draw frontfacing polygons • Draw backfacing polygons • But draw them in thick (2-pixel) wireframe • Or draw them in wireframe, z-biased forward a bit • Or “fatten” them • Or displace them along their normals (“halo” effect) Real-Time Rendering

12. Results Wireframe Translation Fattening 40 fps 50 fps 11.5 fps Original Venus model : 5672 triangles, 66 fps Real-Time Rendering

13. Image Processing Silhouetting • Idea: analyze the image after it’s rendered, and extract silhouettes (like edge detection) • Can help by rendering e.g. depth image, object-ID image, normal image • Perfect for fragment program! Real-Time Rendering

14. Silhouette Edge Detection • Idea: find silhouette edges geometrically and render them explicitly • Hierarchy of normal cones • Lots of other interesting geometric approaches • Most work, but gives the most flexibility in how silhouettes are drawn Real-Time Rendering

15. Recap: Edge Highlighting • Toon shading (and other NPR techniques based on drawing) requires drawing/highlighting some edges: • Silhouette edges • Mesh boundaries (always on silhouette) • Creases (ridge and valley) • Material boundaries • Find first at run-time, precalculate the others (unless object is deformable) Real-Time Rendering

16. Recap: Silhouette Edges • Surface angle silhouetting • Calc N●V, if below threshold  draw black • Best as a per-pixel routine • The Cg program we looked at • Options: spheremap, or use a mip-map with top-level textures dark • Pros: • Uses the texture hardware  fast • Can antialias the resulting lines • Cons: • Line width depends on curvature • Doesn’t work for some models (e.g., a cube) Real-Time Rendering

17. Recap: Silhouette Edges • Procedural Geometry Silhouetting • Idea: render geometry such that the silhouettes “fall out”, e.g.: • Draw frontfacing polygons • Draw backfacing polygons • But draw them in (possibly thick) wireframe • Or draw them z-biased forward a bit • Or “fatten” them • Or displace them along their normals (“halo” effect) • Flip normals • Amount of displacement varies w/ distance (why?) • Perfect task for vertex shader! • Pros: relatively robust, doesn’t need connectivity info • Cons: Wastes some fill & some polys, needs antialiasing

18. Recap: Silhouette Edges • Silhouette Edge Detection • Idea: find silhouette edges geometrically on the CPU and render them explicitly • Brute force: test every edge to see if its adjoining polygons face opposite directions in eye space • Can speed this up with hierarchy or randomized coherent search • Most work, but gives most flexibility in how silhouettes drawn • GPU variant: • Draw degenerate quadrilateral at each edge • “Fatten” quad into a “fin” when edge is on silhouette • Fin thickness based on distance to eyepoint • Vertex shader & replicated vertices today  geometry shader!

19. Highlighting Ridge Edges • Clever related technique by Raskar: • Add “fins” to every edge at dihedral angle • Size fins according to distance to viewer • Again, perfect for vertex shader, geometry shader • Similar but more complicated technique for highlighting valley edges Real-Time Rendering

20. Drawing Lines:Outlining Polygons • Surprisingly hard to draw polys as filled outlines • Problem: depth buffer values of edge & polys same • 2-pass technique: draw polys, then draw edges • Z-bias edges forward or polygons back (glPolygonOffset) • Works okay, but has occasional problems • 3-pass technique: • Render filled polygon • Disable depth buffer writes (leave depth test on) • Enable color buffer writes • Render polygon edges polygon • Normal depth & color buffering • Render filled polygon again • Enable depth buffer writes • Disable color buffer writes Real-Time Rendering

21. Drawing Lines:Hidden-Line Rendering • Hidden-line vs. obscured line vs halos • Hidden-line • Draw polygons to depth buffer (not color buffer) • Draw edges using previous technique • Obscured (light, dotted, dashed) line • Draw all edges in obscured style • Draw polygons to depth buffer (not color buffer) • Draw edges using previous technique • Haloed line • Draw all edges as thick background-color lines • Draw edges using biasing, foreground-color Real-Time Rendering

22. Other Styles • Impressionistic or “painterly” rendering: • Sprinkle particles on object surface • Draw particles as brushstrokes • Can render images to encode normals, surface curvature, depth, color/tone info Real-Time Rendering

23. Painterly Rendering • More info if time permits… Real-Time Rendering

24. Other Styles • Hatching: • Store different cross-hatch patterns representing different tones as textures • Clever ways to use texture hardware to blend between tones at run-time • More info if time permits… Real-Time Rendering

25. Other Styles • “Graftals” are a general term used for strokes, decals, little bits of geometry • Dynamic placement of graftals to achieve certain effects/styles:

26. Other Styles • Technical Illustration Real-Time Rendering

27. Generalizing NPR • Recent work by researchers in NPR attempts to generalize many of these concepts: Coherent Stylized SilhouettesKalnins, Davidson, Markosian, Finkelstein

28. Generalizing NPR • Recent work by researchers in NPR attempts to generalize many of these concepts: Suggestive ContoursDeCarlo, Finkelstein, Reinkiewicz, Santella

29. Gallery of NPR images • Technical Illustration • Coherent Stylized Silhouettes • Suggestive Contours • SIGGRAPH papers… • NPAR