Announcements

1. Announcements • Be sure to register for the class! • Keep doing a good job!

2. Lecture 2 Silicon Art! Quake II – Software Renderer The GPU Pipeline Minecraft1 Weeklies Tips for Minecraft 2 Rendering and OpenGL

3. Software Rendering • Rendering all based on the CPU • Time before hardware acceleration via GPUs was common • Optimizations via software • Developers needed to write more elegant code

4. Quake II Engine • id Tech 2 – used in Quake II • Shipped with software and OpenGL renderer • We will talk more on OpenGL later • Extremely popular engine • Over 30 games have licensed the engine • Marked 3D engine departure from software/8 bit to hardware/24 bit rendering system • Occurred around 1997

5. Quake II Engine • Engine and game written in C • Software renderer used x86 assembly • Beautiful piece of software Code Review http://fabiensanglard.net/quake2/index.php

6. Quake II Rendering • Typical frame had 600-900 polygons • Compared to the millions in modern games • We will compare the two renderers

9. Quake II Software renderer • Main differences from OpenGL: • Absence of colored lighting • Absence of bilinear filtering • Used a color palette of 256 colors to fake 64 gradients of 256 colors • Pentium processors were good at floating point calculations; not good at reading/writing pixels

10. Quake II Software renderer • Used in engines before hardware acceleration • Before graphics cards were widespread • OpenGL 1.1 released in 1997… • Largest part of the Quake II source • Last of its kind • Quake III Arena didn’t ship with software renderer

11. Color Palette

12. Rearranged Palette

13. Rendering Order • Limitations are set via the palette • Rendering focuses on achieving ZERO overdraw • Map (BSP traversal) • Entities (Using Z-buffer to determine visibility) • Transparent textures • Particles • Post processing • Outdated and archaic compared to modern rendering techniques • For more info: http://fabiensanglard.net/quake2/quake2_software_renderer.php

14. Using the Rearranged Palette • First, select color between [0, 255] based on texture coordinate • Add x*256, where x is within [0,63] to get a darkening version of the color initially selected • Basically allowed for 64 gradients of 256 colors with 256 colors • Pretty freaking awesome

16. Quake II OpenGL renderer • 24bit RGB true color palette system • Rendering order similar to the software rendering • 50% less code with 30% increase in frame rate at higher resolutions • Allows players to have: • Bilinear filtering • Color lighting

17. Legacy OpenGL • No glGenTexture – Quake II engine generated its own texture ids • 0 – 1024 – regular textures • 1024 – dynamic lightmap texture • >1024 – static lightmap texture • No use of GL_LIGHTING • CPU did all lighting calculations

19. What is OpenGL? • OpenGL is a standard, like HTML and JPEG • OpenGL ES for mobile • WebGL for browsers • GLU (OpenGL Utility Library) • Part of the OpenGL standard • Higher level: mesh primitives, tessellation • GLUT (OpenGL Utility Toolkit) • Platform-independent windowing API for OpenGL • Not officially part of OpenGL • Platform-dependent APIs for initialization • GLX (Linux), WGL (Windows), CGL (OS X)

20. What is a GPU? • Specialized hardware for 2D and 3D graphics • Special memory for textures and z-buffers • Massively parallel • Hundreds of "cores" • Hardware threading support

21. Lecture 2 Bus, do your stuff! Rendering and OpenGL Minecraft1 Weeklies Tips for Minecraft 2 The GPU Pipeline

22. Stage 1: Uploading vertices • Vertices specified by application • Also per-vertex data (colors, texture coordinates) • Ideally stored in optimized GPU memory • OpenGL has several bad ways: • glBegin() / glEnd() • Slowest method • Start from scratch every frame • Display lists • Pre-compiled OpenGL commands • Not any faster on some drivers • “Vertex arrays" • Data isn't stored on the GPU • Marginally better than glBegin() / glEnd() • Best way: Vertex Buffer Objects (VBOs)

23. Vertex Buffer Objects (VBOs) • Handle to GPU memory for vertices • Leaving geometry info on the GPU avoids wasting the bandwidth that re-uploading it every frame uses • Contents may be modified after creation • But don’t need to be updated from scratch • Can store any attributes, more than just vertex position • Color • Normal • Texture coordinates

24. VBO storage formats • Array-of-structs • Interleaving improves static object speed (cache locality) • Struct-of-arrays • Repeatedly modifying only some attributes ... Vertex 1 Color 1 Vertex 2 Color 2 One interleaved buffer ... Vertex 1 Vertex 2 ... Color 1 Color 2 Two separate buffers

25. VBO example code: initalization // Initialization (interleaved data format) float data[] = { // Position Color 1,0,0, 1,0,0, // Vertex 1 0,1,0, 1,0,0, // Vertex 2 0,0,1, 1,1,0// Vertex 3 }; unsigned intid; glGenBuffers(1, &id); // Generate a new id glBindBuffer(GL_ARRAY_BUFFER, id); // Bind the buffer // Upload the data. GL_STATIC_DRAW is a hint that the buffer // contents will be modified once but used many times. glBufferData(GL_ARRAY_BUFFER, sizeof(data), data, GL_STATIC_DRAW); glBindBuffer(GL_ARRAY_BUFFER, 0); // Unbind the buffer

26. VBO example code: rendering // Rendering (interleaved data format) glBindBuffer(GL_ARRAY_BUFFER, id); glEnableClientState(GL_VERTEX_ARRAY); glEnableClientState(GL_COLOR_ARRAY); unsigned intstride = sizeof(float) * (3 + 3); // Spacing between vertices glVertexPointer(3, GL_FLOAT, stride, (char *) 0); glColorPointer(3, GL_FLOAT, stride, (char *) (3 * sizeof(float))); glDrawArrays(GL_TRIANGLES, 0, 3); // Draw 3 vertices (1 triangle) glDisableClientState(GL_COLOR_ARRAY); glDisableClientState(GL_VERTEX_ARRAY); glBindBuffer(GL_ARRAY_BUFFER, 0);

27. VBO tips • Never initialize VBOs in the draw loop! • Negates all performance benefits • Possibly worse than glBegin()/glEnd() • Qt provides QGLBuffer • http://qt-project.org/doc/qt-4.8/qglbuffer.html • More references • http://www.opengl.org/wiki/Vertex_Buffer_Object • http://www.opengl.org/wiki/VBO_-_more

28. Stage 2: The vertex shader Vertex • Transforms vertices • Object space to clip space (modelview, projection) • Clip space to screen space automatic (perspective division, viewport transformation) • Can perform other computations • Vertex-based fog • Per-vertex lighting • Passes data to fragment shader • E.g. normals for Phong shading • Interpolated between vertices for each fragment Object space Modelview matrix Vertex shader Camera space Projection matrix Clip space Perspective division Fixed function Normalized device space Viewport transformation Screen space Interpolation…

29. Stage 2: The vertex shader Vertex • Inputs • Uniform values (matrices, lights) • Vertex attributes (color, texcoords) • Outputs • Clip-space vertex position (gl_Position) • Varying values (color, texcoords) • Will be inputs to fragment shader • Interpolated across triangles using perspective-correct linear interpolation Object space Modelview matrix Vertex shader Camera space Projection matrix Clip space Perspective division Fixed function Normalized device space Viewport transformation Screen space Interpolation…

30. A vertex shader (GLSL) // Input from C++, global per shader uniformfloat scale; // Output for the fragment shader, input for // fragment shader (interpolated across the triangle) varyingvec4 vertex; void main() { vertex = gl_Vertex; // Input from C++, per vertex vertex.xyz *= scale; gl_Position = gl_ModelViewProjectionMatrix * vertex; }

31. Stage 3: Primitive assembly • Create primitives using index buffer • Index buffer contains offsets into vertex buffer • Used to share vertices between triangles • Also specifies rendering order 2 3 0 1 Vertex Buffer Index Buffer 0: (x, y, z) 0, 1, 2 1: (x, y, z) 1, 3, 2 2: (x, y, z) 3: (x, y, z)

32. OpenGL geometry modes GL_POINTS GL_LINES GL_LINE_LOOP 5 2 3 5 6 2 5 4 6 1 1 3 4 3 1 4 2 GL_LINE_STRIP GL_QUADS GL_TRIANGLES 8 5 2 1 7 6 3 4 6 6 5 3 7 4 3 2 4 GL_POLYGON 2 1 1 GL_TRIANGLE_FAN 5 GL_TRIANGLE_STRIP 4 7 8 5 GL_QUAD_STRIP 6 8 4 5 7 3 3 6 3 4 1 6 2 5 1 2 2 1

33. Stage 4: Clipping and rasterization • Clipping: Determine which (parts of) triangles are visible • Modify geometry to stay within the view volume • Done in clip space where plane equations are simple (e.g. ‐1 ≤ x ≤ 1) • Cull back-facing polygons • Does not include occlusion culling • Culling objects hidden behind other objects • Occlusion culling must be done by the application

34. Rasterization on the GPU • Turn triangles into pixels • Not done using scanlines on GPUs • Usually done in tiles (maybe 8x8 pixels) • Tiles have better cache performance and shaders need to be run in 2x2 blocks, we'll see why later • Use coarse rasterizer first (maybe 32x32 pixels) • Very thin triangles are a worst case

35. Stage 5: Fragment shader • Set the color for each fragment (each pixel) • Inputs • Uniform values (lights, textures) • Varying values (normals, texture coordinates) • Outputs • Fragment color: gl_FragColor (and optionally depth) • Fragment shaders must run in parallel in 2x2 blocks • Need screen-space partial derivatives of texture coordinates for hardware texturing (mipmapping) • Compute finite differences with neighboring shaders • Can compute partial derivatives of any varying value this way

36. A fragment shader (GLSL) // Input from the vertex shader, // interpolated across the triangle varying vec4 vertex; void main() { // Compute the normal using the cross product of the x // and y screen-space derivatives of the vertex position vec3pos = vertex.xyz; vec3 normal = normalize(cross(dFdx(pos), dFdy(pos))); // Visualize the normal by converting to RGB color gl_FragColor = vec4(normal * 0.5 + 0.5, 1.0); }

37. OpenGL texture mapping • Big arrays of data with hardware filtering support • Dimensions: 1D, 2D, 3D • Texture formats: GL_RGB, GL_RGBA, ... • Data types: GL_UNSIGNED_BYTE, GL_FLOAT, ... • Texture wrap modes: Texture GL_CLAMP_TO_EDGE GL_REPEAT

38. OpenGL texture filtering • Each texture has two filters • Magnification • When the texture is scaled up (texture is up close) • Minification • When the texture is scaled down (texture is far away) • Two types of filtering supported • Nearest-neighbor: GL_NEAREST • Bilinear interpolation: GL_LINEAR

39. Mipmapping • Pre-filtered textures used to avoid aliasing • Uses smaller versions of textures on distant polygons • Screen-space partial derivative of texture coordinate used to compute mipmap level • Can also filter between two closest mipmap levels (trilinear filtering) Without mipmapping With mipmapping

40. Mipmapping in OpenGL • Repeatedly halve image size until 1x1, store all levels • Done automatically with gluBuild2DMipmaps() • Ensure minification filter is GL_LINEAR_MIPMAP_LINEAR

41. Stage 6: Depth testing and blending • Depth testing • Reject pixel if behind existing objects • Different modes: GL_LESS, GL_GREATER, ... • Early Z-culling • Move depth testing before fragment shader • Avoids some fragment shader evaluations • Only used if fragment shader doesn't modify depth • Drawing front-to-back is much faster than drawing back-to-front in the presence of overdraw

42. Hierarchical z-buffer culling • Within tile, blue primitive can be culled because it is completely occluded Minimum primitive depth inside tile Maximum depth of all fragments in tile 32x32 pixel tile 1 Green primitive is in front of blue primitive z 0

43. Hierarchical z-buffer culling • Hierarchical z-buffer (maybe 32x32 tiles) • Early z-culling at tile resolution • Occurs before fine rasterization and early z-culling • Tiles store maximum depth of their fragments • Reject all fragments for a primitive within a tile if minimum fragment depth > maximum tile depth • Changing depth testing mode mid-frame disables this optimization, so avoid doing so!

44. Blending • Blend source and destination RGBA colors • Destination (dst): current framebuffer color • Source (src): color of fragment being blended • Final color = source • sfactor + destination • dfactor • Specify factors using glBlendFunc(sfactor, dfactor)

45. GL_ZERO GL_ONE GL_DST_COLOR GL_ONE_MINUS_DST_COLOR GL_SRC_ALPHA GL_ONE_MINUS_SRC_ALPHA Blending diagram sfactor Destination image (current framebuffer contents) GL_ZERO GL_ONE GL_SRC_COLOR GL_ONE_MINUS_SRC_COLOR GL_SRC_ALPHA GL_ONE_MINUS_SRC_ALPHA Source image (fragments being blended in front) dfactor

46. Blending • Good combinations for games: • glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);Regular alpha blending, order dependent • glBlendFunc(GL_ONE, GL_ONE);Additive blending, order independent • glBlendFunc(GL_ONE_MINUS_DST_COLOR, GL_ONE);Additive blending with saturation, also order independent but looks much better than additive!

47. Stage 7: The framebuffer • A collection of 2D arrays used by OpenGL • Color buffer • Stores RGBA pixel values (32 bits) • Depth buffer • Stores a single integer depth value (commonly 24 bits) • Stencil buffer • Linked with depth buffer, stores a bitmask that can be used to limit visibility (commonly 8 bits) • Accumulation buffer • Used to store intermediate results • Outdated and historically slow

48. The stencil buffer • Per-pixel test, similar to depth buffer • Test each fragment against value from stencil buffer, reject fragment if stencil test fails • Distinct cases allow for different behavior when • Stencil test fails • Stencil test passes but depth test fails • Stencil and depth tests pass • OpenGL stencil functions: • glEnable(GL_STENCIL_TEST) • glStencilFunc(function, reference, bitmask) • glStencilOp(stencil_fail, depth_fail, depth_pass) • glStencilMask(bitmask)

49. Stencil buffer example: reflections • glScalef(1,-1,1) draws upside-down, but need to restrict rendering to inside reflecting poly • First render reflective poly, setting stencil to 1 wherever it is without rendering to color or depth buffers • Draw reflected object, restricted to where stencil buffer is 1 • Draw normal object ignoring stencil buffer • Draw reflective surface on top of reflection with blending, using alpha to control reflectivity (higher alpha = less reflective)

50. Stencil buffer rendering tricks Stenciled shadow volumes Mirrors and portals