Understanding Graphics Rasterization: Vertex to Pixel Pipeline

1. Rasterization Kurt Akeley CS248 Lecture 5 9 October 2007 http://graphics.stanford.edu/courses/cs248-07/

2. The vertex pipeline Application struct { float x,y,z,w; float r,g,b,a;} vertex; Vertex assembly Vertex operations struct { vertex v0,v1,v2 } triangle; Primitive assembly Primitive operations struct { short int x,y; float depth; float r,g,b,a;} fragment; Screen coordinates Rasterization Fragment operations struct { int depth; byte r,g,b,a;} pixel; Frame buffer Display

3. Screen coordinates Ideal screen coordinates are continuous • Not integer pixel locations! Implementations always use discrete math • Fixed-point or floating-point • Always with substantial sub-pixel precision • Fixed-point illustrated in the pre-filter antialiasing lecture A pixel is a ‘big’ thing • Spatial resolution can approach # of pixels on screen • Data resolution can be large too • Multiple copies of pixel data structure • SGI RealityEngine frame buffer was deeper than wide or tall struct { float x,y,z; float r,g,b,a;} vertex;

4. Key facts about perspective projection Straight lines project to straight lines (on a plane) • Only vertexes need to be transformed • That’s why we’re interested in lines and polygons Parameterizations (e.g., distance) are warped: More on projection in later lectures …

5. Two fundamental operations Fragment selection • Identify pixels for which fragments are to be generated • <x,y> ‘attributes’ are special • Must be • Exact, for aliased rendering • Superset, for antialiased rendering • Should be efficient, for performance Attribute assignment • Assign attribute values to each fragment • E.g., color, depth, … struct { short int x,y; float depth; float r,g,b,a;} fragment;

6. Fragment selection Generate one fragment for each pixel that is intersected (or covered) by the primitive Intersected could mean that the primitive’s area intersects the pixel’s: • Center point, or • Square region, or • Filter-function (in area-sampling terms) Some examples …

7. Point-sampled fragment selection Generate fragment if pixel center is inside triangle Implements point-sampled aliased rasterization

8. Point-sampled fragment selection Pixels along shared edges should have exactly one fragment selected for them • Must handle on-edge/on-vertex sample points consistently

9. Tiled fragment selection Generate fragment if unit square intersects triangle Implements multisample and tiled rasterizations

10. Tiled fragment selection Multisample rasterization • 4x4 samples per pixel

11. Tiled fragment selection Tiled rasterization • 4x4 pixels per tile

12. Antialiased fragment selection Generate fragment if filter function intersects triangle Implements pre-filtered antialiasing

13. Fragment selection (continued) What if the primitive doesn’t have a geometric area? • Delta-function points and lines don’t Three choices: • Rule-based approach • n x n pixel point • Bresenham line (details later in this lecture) • Pre-filter • Bandlimited  infinite spatial extent • Assign a screen-space geometry • Circle for point • Rectangle for line

14. Geometry-based attribute assignment (Assumes vertex-specified geometry, i.e., polygons) Two steps • Parameterize the attribute • Fit a function (surface) to the vertex values • Point-sample this parameterization Which parameterization? • Constant (aka flat shading) • No continuity at shared edges

15. Geometry-based attribute assignment (Assumes vertex-specified geometry, i.e., polygons) Two steps • Parameterize the attribute • Fit a function (surface) to the vertex values • Point-sample this parameterization Which parameterization? • Constant (aka flat shading) • No continuity at shared edges • Bilinear (planar surface) • Value continuity at shared edges

16. Bilinear (planar) parameterization red y x

17. Geometry-based attribute assignment (Assumes vertex-specified geometry, i.e., polygons) Two steps • Parameterize the attribute • Fit a function (surface) to the vertex values • Point-sample this parameterization Which parameterization? • Constant (aka flat shading) • No continuity at shared edges • Bilinear (planar surface) • Value continuity at shared edges • Cubic (non-planar surface) • Slope continuity at shared edges

18. Mach banding – value discontinuities Flat shaded, but appeared ‘scalloped’

19. Mach banding – slope discontinuities Same peak intensities

20. Geometry-based attribute assignment (Assumes vertex-specified geometry, i.e., polygons) Two steps • Parameterize the attribute • Fit a function (surface) to the vertex values • Point-sample this parameterization Which parameterization? • Constant (aka flat shading) • No continuity at shared edges • Bilinear (planar surface) • Value continuity at shared edges • Cubic (non-planar surface) • Slope continuity at shared edges • Gouraud (hybrid)

21. Fragment selection Walk (iterate along) edges Change edges at vertexes Attribute assignment Loop in a loop algorithm: Iterate linearly along edges Iterate linearly edge-to-edge Outer loop is complex E.g., either 2 or 3 regions Parameterization is a function of Screen orientation Choice of spans Gouraud shaded quad

22. “All” projected quadrilaterals are non-planar Due to discrete coordinate precision What if quadrilateral is concave? Concave is complex (split spans -- see example) Non-planar  concave for some view What if quadrilateral intersects itself? A real mess (no vertex to signal change –- see example) Non-planar  “bowtie” for some view Problems with quads / polygons

23. All polygons are triangles (or should be) Triangle is always convex • Regardless of arithmetic precision • Simplifies rasterization—no special cases Three points define a plane • All triangles are planar • All parameterizations are (or can be) planar Modern GPUs decompose polygons to triangles • SGI switched in 1990 with the VGX product • OpenGL is designed to allow triangulation • Optimized quadrilateral decomposition developed

24. Complex polygons There are algorithms to rasterize • Self-intersecting polygons • Polygons with holes • … These polygons have applications in 2-D rendering But they are not useful for 3-D rendering • Too slow to render • Don’t have meaningful attribute parameterizations So we will ignore them

25. Normal-based quad decomposition Compute A•C and B•D Connect vertex pair with the greater dot product Avoid connecting the ‘stirrups’ Bottom line: decomposition matters! A C B D

26. Direct evaluation: Iteration vs. direct evaluation Iteration: Between adjacent pixels Along edges

27. Iteration vs. direct evaluation Iteration • Is less numerically intensive (no multiplication) Direct evaluation • Is more precise (no accumulated error) • Parallelizes better (no sequence presumption)

28. DDA iteration Digital Differential Analyzer (DDA) Implements iteration in fixed-point representation • E.g., iiiiiiii.ffff (8.4) or siiiiiii.ffff (s7.4) • Repeatedly adds delta value to accumulated value Loses ½ LSB precision per iteration step • Require log2(n) fraction bits for n steps • To reach the correct extreme values • Dimensions of rendering space determine maximum number of steps • May differ from size of frame buffer • 2-D iteration requires an extra bit

29. Triangle rasterization examples Gouraud shaded (GTX) Edge walk, planar parameterization (VGX) Barycentric direct evaluation (InfiniteReality) Small tiles (Bali – proposed) Per-pixel evaluation (Pixel Planes 4)

30. Algorithm properties Setup and execution costs • Setup: constant per triangle • Execution: relative to triangle’s projected area Ability to parallelize Ability to cull to a rectangular screen region • To support tiling • To support “scissoring” Triangle to berasterized Scissor region

31. dadx Gouraud shaded (GTX) Two-stage algorithm • DDA edge walk • fragment selection • attribute assignment • DDA scan-line walk • attribute assignment only Requires expensive scan-line setup • Location of first sample is non-unit distance from edge Parallelizes in two stages (e.g., GTX) Cannot scissor efficiently Works on quadrilaterals

32. Edge walk, planar evaluation (VGX) dady dadx

33. Edge walk, planar evaluation (VGX) Hybrid algorithm • Edge DDA walk for fragment selection • Efficient generation of conservative fragment set • Pixel-center DDA walk for attribute assignment • Never step off sample grid, so • Sub-pixel adjustment is made just once, • Rather than for each scan-line Scissor cull possible • Adds complexity to edge walk • Easy for attribute evaluation Parallelizes similarly to Gouraud

34. Interpolation outside the triangle

35. DDA can operate out-of-range MSBs beyond desired range don’t influence result • Carry chain flows up, not down • Can handle arbitrarily large slopes • Can iterate outside the triangle’s area Must not clamp (range limit) intermediate results! Doesn’t work for floating point! Accum Delta +

36. Wrapping Problem: overflow or underflow of iterated value • Integer arithmetic “wraps” • Maximum value overflows to zero • Zero underflows to maximum value • Minor iteration error  huge value error Overflow Underflow

37. Guard bits Solution: extend range to detect “wrapped” values • Add one or two “guard” MSBs • Non-zero guard bit(s)  out-of-range value • ‘Clamp’ out-of-range values to the nearer of zero or max

38. Guard-bit example 2-bit value, 1 guard bit Overflow Underflow

39. Guard-bit implementation (n-bit) Inguard Outn-1 Inn-1 Out1 In1 Out0 In0

40. DDA bit-assignment examples Edge walk in 4k x 4k rendering space (35 bits) Pixel (12) Subpixel (10) Iteration (12) Guard (1) Depth walk in 4k x 4k rendering space (39 bits) Depth (24) Iteration (13) Guard (2) (one extra for diagonal)

41. Barycentric (InfiniteReality) Hybrid algorithm • Approximate edge walk for fragment selection • Pineda edge functions used to generate AA masks • Direct barycentric evaluation for attribute assignment • Minimizes setup cost • Additional computational complexity accepted • Handles small triangles well Scissor cull implemented

42. a2 a0 a1 Barycentric attribute evaluation (x1, y1, v1) (x, y, v) (x0, y0, v0) (x2, y2, v2)

43. Small tiles (Bali – proposed) Frame buffer tiled into nxn (16x16) regions • Each tile is owned by one of k separate engines Two-level rasterization: • Tile selection (avoid broadcast, conservative) • Fragment selection and attribute assignment Parallelizes well Handles small triangles well Scissors well • At tile selection stage

44. Tiled fragment selection Tiled rasterization • 4x4 pixels per tile

45. Engine per pixel (Pixel Planes 4) Image courtesy of Anselmo Lastra, University of North Carolina at Chapel Hill

46. Engine-per-pixel (Pixel Planes 4) Individual direct-evaluation engine at every pixel ! • Solves edge equations to determine inclusion • Solves attribute equations to determine values Setup involves computation of plane and edge slopes Execution is in constant-time • Clever evaluation tree makes this possible • Extremely fast for large triangles, but • Extremely inefficient for small triangles • Effectively generates a fragment for every pixel • Scissor culling is a non-issue

47. Pixel Planes 4 fragment selection Image courtesy of Anselmo Lastra, University of North Carolina at Chapel Hill

48. Pixel Planes 4 fragment selection Image courtesy of Anselmo Lastra, University of North Carolina at Chapel Hill

49. Pixel Planes 4 fragment selection Image courtesy of Anselmo Lastra, University of North Carolina at Chapel Hill

50. Pixel Planes 4 attribute evaluation Image courtesy of Anselmo Lastra, University of North Carolina at Chapel Hill