Real-Time Shading Using Programmable Graphics Hardware

1. Real-Time Shading Using Programmable Graphics Hardware Introduction, Setup and Examples Wan-Chun Ma National Taiwan University

2. Course Infomation • Instructor • Wan-Chun Ma, Alex • Dept. of Computer Science and Information Engineering, National Taiwan University • http://graphics.csie.ntu.edu.tw/~firebird • Course • 4/28, 5/5, 5/12, 5/14 • Suggested readings • R. Fernando and M. J. Kilgard. The Cg Tutorial: The Definitive Guide to Programmable Real-Time Graphics, Addison-Wesley, 2003 (beginners only!) • R. J. Rost. OpenGL Shading Language, Addison-Wesley, 2004 • http://graphics.csie.ntu.edu.tw/~firebird/dokuwiki/doku.php?id=tech:courses:dci_rts:home

3. The Student... • The student should be familiar with • C/C++ programming • Graphics basics • Transformations in 3D (translation, rotation, modelview, projection) • Rasterization • Texturing • OpenGL • GLUT, GLUI • Use texturing in OpenGL

4. Today’s Schedule • Introduction • Setup of Programming Environment • Real-Time Shading Examples

5. Introduction

6. Evolution of GPUs

7. The 5 Generations of GPU • 1stgeneration (up to 1998) • NVIDIA TNT2, ATI Rage, 3dfx Voodoo3 • Lack of transform vertices of 3D objects. Vertex transformation are done by CPU • Limited math operations for combining textures to compute the color of pixels • 2nd generation (1999-2000) • NVIDIA GeForce 256, GeForce 2, ATI Radeon 7500 • GPU has the ability to do transformation and lighting. Both OpenGL and DirectX 7 support vertex transformation by hardware • Configurable (in driver level) but not programmable

8. The 5 Generations of GPU • 3rd generation (2001) • NVIDIA GeForce 3, GeForce 4 Ti, Xbox, ATI Radeon 8500 • Vertex programmability: DirectX 8 vertex shader and OpenGL ARB vertex program • Pixel-level configurable • 4th generation (2002) • NVIDIA GeForce FX, ATI Radeon 9700 • Vertex and pixel programmability • High-level shading language (NVIDIA Cg, Microsoft HLSL, OpenGL GLSL) • 5th generation (2004) • NVIDIA GeForce 6, ATI Radeon X • Infinite length shader program • Dynamic flow control

9. GPU Model (Old) • Fixed function pipeline

10. GPU Model (Current) • Programmability!

11. GPU Process Vertex Processing Fragment Processing

12. Programming GPU • However, programming in assembly is painful Oh my god!

13. Programming GPU • The need of high level shading language Compile

14. Cg: A Shading Language • Cg is a high level language from NVIDIA for programming GPUs, developed in close collaboration with Microsoft • Cg stands for “C for Graphics” • Cg enables a dramatic productivity increase for graphics development developers of: • Games • CAD tools • Scientific visualizations

15. Cg: A C-like Language • Syntax, operators, functions from C • Conditionals and flow control (for, if) • Particularly suitable for GPUs: • Express data flow of pipeline/stream architecture of GPUs (e.g. vertex-to-pixel) • Vector and matrix operations • Support hardware data types for maximum performance • Exposes GPU functions for convenience and speed: • Intrinsic: (mul, dot, sqrt, exp, pow) • Built-in: extremely useful and GPU optimized math, utility and geometric functions (noise, ddx, ddy, reflect) • Compiler uses hardware profiles to subset Cg as required for particular hardware feature sets

16. Cg Workflow • Architecture

17. Cg Workflow

18. What Cg can do? • Real-time visual effects

19. What Cg can do? • Lots of effects…

20. Coffee Break • Next section: Setup of Programming Environment

21. Setup of Programming Environment

22. Requirement • Hardware • The computer should be equipped with programmable graphics hardware • NVIDIA FX, NVIDIA 6, ATI 9x00, ATI X series • Software • Microsoft Visual Studio .NET 2003 • GLUT, GLUI...

23. Installation • Cg Toolkit 1.3 (10MB) • http://developer.nvidia.com/object/cg_toolkit.html • Check the “Cg Installer for Windows” • NVIDIA SDK 9.0 (340MB, not required) • http://developer.nvidia.com/object/sdk_home.html • FX Composer 1.6 (60MB, not required) • http://developer.nvidia.com/object/fx_composer_home.html • Check the “FX Composer 1.6 Installer”

24. Installation • If default installation locations are used, all the packages are installed in the folder of C:\Program Files\NVIDIA Corporation\ • C:\Program Files\NVIDIA Corporation\ • Cg\ • NVIDIA FX Composer\ • SDK 9.0\

25. My Stuff • Several useful codes I collect • http://graphics.csie.ntu.edu.tw/~firebird/download/dci_rts/class.zip • Download it and unpack it into a folder, say • D:\My Projects\Class\ • Any folder is ok, but remember where you put it

26. VC++ Directories • Execute visual studio • Tools, Options, Projects, VC++ Directories • Show the directories for: • Include files • D:\My Project\Class (remember My Stuff?) • C:\Program Files\NVIDIA Corporation\Cg\include • Library files • C:\Program Files\NVIDIA Corporation\Cg\lib

27. Ready to Go • A small engine • http://graphics.csie.ntu.edu.tw/~firebird/download/dci_rts/env.zip • I will use this engine for shader development during these courses • The first example • http://graphics.csie.ntu.edu.tw/~firebird/download/dci_rts/ex01.zip

28. Compilation • cgc –profile profiles filename • profiles: graphics hardware profiles • Vertex: arbvp1, vp20, vp30, vp40... • Fragment: arbfp1, fp20, fp30, fp40... • filename: filename of the shader • Examples • cgc –profile vp30 test_vtx.cxx • cgc –profile fp30 test_frg.cxx

29. Debugging • Debugging is very hard (it is GPU, not CPU) • However, you may still use intermediate visualization to debug your program • Output intermediate data (e.g. position, normal, textures…) as color

30. Coffee Break • Next section: Real-time Shading Examples

31. Real-Time Shading Examples

32. Progression • Games push hardware, hardware advances games

33. Effects in Games Reflection Level of detail Smoke Shading Shadows

34. Effects in Games Per-pixel lighting Multi-texturing Light mapping Bump mapping

35. Multi-pass Rendering • The rendering pass is not fixed anymore. A single rendering pass may consists of many functional programs

36. Multi-pass Rendering • Each different program (effect) is handled individually, and finally summed up to become rendering result

37. Cg Samples • Check out the effect samples in NVIDIA SDK Browser

38. The First Cg Example • A Phong model shader with color texture • Shaders • Vertex: ex1_vtx.cxx • Fragment: ex1_frg.cxx • Texture • Diffuse: wood.bmp

39. Vertex Shader • Vertex-to-fragment data structure struct v2f { float4 P2D : POSITION; // projected 2D position float4 C : COLOR0; // color float4 T : TEXCOORD0; // texture coord float3 P3D : TEXCOORD1; // vertex 3D position float3 N : TEXCOORD2; // normal float3 G : TEXCOORD3; // tangent float3 B : TEXCOORD4; // binormal };

40. Vertex Shader • Main (application-to-vertex) arguments v2f main( float4 C : COLOR, float4 P : POSITION, float4 N : NORMAL, float4 T : TEXCOORD0, uniform float4x4 ModelViewProj, uniform float4x4 ModelView, uniform float4x4 ModelViewIT)

41. Vertex Shader • Main body { v2f OUT; OUT.P2D = mul(ModelViewProj, P); OUT.P3D = P.xyz; OUT.T = T; OUT.N = normalize(N.xyz); // normal OUT.G = normalize(2.0*C.xyz - 1.0); // tangent OUT.B = normalize(cross(OUT.G, OUT.N)); return OUT; }

42. Fragment Shader • Fragment-to-screen data structure struct f2s { float4 C : COLOR0; };

43. Fragment Shader • Main (vertex-to-fragment) arguments f2s main( v2f IN, uniform sampler2D tex01, // texture 01 uniform float3 L, uniform float3 V)

44. Fragment Shader • Main body { f2s OUT; OUT.rgb = 0; L = normalize(L); V = normalize(V); float3 H = normalize(L+V); float diff = dot(normalize(IN.N), L); if(diff > 0) { float spec = 2*pow(dot(IN.N, H), 128); OUT.C.rgb = diff*tex2D(tex01, IN.T.xy) + spec; } return OUT; }

45. Result

46. Try This... • Output red color for all fragments { f2s OUT; OUT.rgb = 0; // L = normalize(L); V = normalize(V); // float3 H = normalize(L+V); // float diff = dot(normalize(IN.N), L); // if(diff > 0) // { // float spec = 2*pow(dot(IN.N, H), 128); OUT.C.rgb = float3(1.0, 0.0, 0.0); // } return OUT; }

47. Try This... • Visualize normal vectors { f2s OUT; OUT.rgb = 0; // L = normalize(L); V = normalize(V); // float3 H = normalize(L+V); // float diff = dot(normalize(IN.N), L); // if(diff > 0) // { // float spec = 2*pow(dot(IN.N, H), 128); OUT.C.rgb = (IN.N+1)/2; // } return OUT; }

48. End