Graphics

1. Graphics CNS 3540

2. Graphics in games • None • ASCII art (so called...) • Images • Bitmaps • Vector • 2D animation • Video • 3D animation

3. 3D Graphics Commonly used libraries: • OpenGL www.opengl.org • Microsoft DirectX http://msdn.microsoft.com/directx Many other libraries, wrappers, and game engines are built on one or both of these libraries.

4. Open standard Provides interface to hardware acceleration Portable Widely used in commercial and non-commercial games No native image or 3D model file format GLUT Proprietary Provides interface to hardware acceleration Only on Microsoft platforms Widely used in commercial and non-commercial games Native .x 3D model format, loaders for common bitmap formats Windows API Direct Play, Direct X Audio, etc. OpenGL vs. DirectX "OpenGL is the only truly open, vendor-neutral, multiplatform graphics standard." http://www.opengl.org/about/overview/

5. Fun with 3D libraries... Both OpenGL and DirectX are complicated libraries and learning how to use them well is not easy. DirectX, in particular, has gone through major changes that obsolete many earlier books and web sites. Microsoft's documentation can also change in confusing ways.

6. Making a game with3D graphics Suppose that I want to make a 3D display for UV Bots (which I do...) How do I do that? We'll look at how do that with some references to the book's game engine and Irrlicht (irrlicht.sourceforge.net)

7. Nine easy steps Nine moderately difficult steps • Find or make 3D models • Initialize software and hardware • Load models • Add models to world • Lights • Camera • Action • Interaction • This, that, and the other Nine frustrating steps that will make you wish you were using Klingon pain sticks instead and will make you want to go into something simple like neurosurgery...

8. Finding models Google (what else?) Call in favors from your artistic friends (you have lots of those, right?) Borrow them from other games (but only if it's ethical) Making models Blender 3D blender.org Milkshape 3D Wings 3D www.wings3d.com Various commercial packages, most of which are very expensive 1. Find or make 3D models

9. 3D model formats • Bad news: There are no "standard" formats for 3D model files that are comparable to JPEG or PNG. • Worse news: Converting between 3D file formats is a pain and the model will probably not look the same after conversion.

10. 3D model formats • Text or binary? • What documentation is available? • Is a file format specification available? • How complicated is the file format? • What software can read/write the format? • Modeling software • Conversion software • Loaders and game engines • Does it include animation?

11. 3D model formats • .3ds - Original 3DS max format • .max - Newer 3DS max format • .x - Direct X • .dxf - created for AutoCAD by AutoDesk • .obj - Wavefront (an ASCII format) • .mdl - Quake plus many more fileformat.info

12. .x DirectX • Both text and binary formats • DirectX SDK has loaders • Includes animation • Specification is available but is not for the faint-hearted • Various importers, exporters, converters, but many are flaky

13. .obj Wavefront • Wavefront (which merged with Alias, which was bought by AutoDesk) • Text format • Relatively simple • Blender can read/write, Irrlicht can load • No animation

14. .ms3d Milkshape • Binary format • Includes animation • Reading and writing: • Chumbalum's web site (www.swissquake.ch/chumbalum-soft) has C++ code to load .ms3d files • Irrlicht can load • Blender cannot read or write

15. 2. Initialize software and hardwareDirectX • Our book has eight fun-filled pages on how to initialize DirectX, MSDN has a lot more than that... • Many options, choosing the wrong options can leave you with a blank screen • Beware of COM • Make sure you deallocate, close, destroy everything you allocate, open, or create if you want to avoid memory leaks

16. 2. Initialize software and hardwareOpenGL • Setup is relatively easy if you use GLU and GLUT, but can still be tricky

17. Encapsulate! • Set-up code can be messy, so hide the details. • Encapsulation makes it easier to port to another OS or another graphics library.

18. 3. Load models Choices: • Write your own loader • Not easy since 3D file formats are complicated and often not well documented • Use someone else's source code • Hard to say how well it will work • Use a game engine • Only an option if the engine supports your format

19. Components of 3D models • Vertices - points in 3D space • Edges - connect vertices • Faces/Polygons • often triangles, sometimes quads • Meshes • Normals - Which way is up? (or out?) • Used for shading

20. More components of models • Hierarchies - parent/child • Colors • Textures • Combinations of other materials • Images - often stored in separate file • Must be mapped to mesh • Bones • Animation - key frames

21. 4. Add model to the world • Left-handed or Right-handed? • Location, size, and orientation • Parent-child hierarchy

22. Left-handed vs. Right-handed y y left-handed x z z x right-handed "The right-handed system is universally accepted as the standard in mathematics and the physical sciences." wikipedia but... 3DS max vs. DirectX

23. Lost in Space(s) • Model space (or local coordinates) • Coordinates are relative to local origin • Used to create model • World space • Coordinates are relative to global origin • Used to place models within the world • Screen space • Projected into 2D plane

24. y y y x x z z x z Local vs. global coordinates The front turrets have the same local coordinates for both instances of the model. The front turrets will have different global coordinates for each instance of the model. How do the x, y, and z global coordinates compare for these two instances of the bot model?

25. Parent/Child hierarchy When the bot turns, the turret should turn also, but sometimes the turret should turn when the bot doesn't. • Frames • Scene graph • Submeshes

26. Lights! Camera! Action! LIGHTING Some material in this section comes from Introduction to Game Programming, edited by Steve Rabin

27. Questions • How much light is incident on a surface? • From which direction does the light come? • How is the light absorbed, reemitted, and reflected? • Which outgoing rays reach the eye of the viewer?

28. Answers(or at least ways to find the answers) • Forward tracing • trace every photon emitted by a light source and figure out where it goes • photon mapping: high quality but extremely slow • Backward tracing • trace a hypothetical photon that hit the eye to see which object it came from • raytracing: practical, but not for real time rendering

29. Answers(or at least ways to find the answers) • "Middle-out" evaluation • "Given this bit of surface, how much light came from these sources of light and ended up hitting the eye?" * • Looks at triangle currently being drawn • Real time rendering has to compromise between speed and quality • p. 471, Introduction to Game Programming, • edited by Steve Rabin

30. Kinds of lighting We'll look at how different kinds of lighting affect this scene. Note: These pictures were made in Blender, not with real-time rendering.

31. Ambient light • Light that is not attributed to any particular source • Can be a reasonable approximation for some multi-light situations, especially indoors What happens if your scene has too much ambient light?

32. Too much ambient light

33. Diffuse lighting

34. Specular lighting

35. Hemisphere lighting Sun + sky + ground Useful approximation for some outdoor scenes direct sunlight diffracted sunlight reflected sunlight

36. Environment map • Approximates reflections • Cube map: • Put cube representing environment around the object being rendered. • Trace vector from eye to object, reflect from object, and see where it strikes the cube. • Same idea as hemisphere lighting but can contain images or more varied colors

37. Shadows • Can be done with ray-tracing. • Some games have shadows. How?

38. Spot Types of lights Point Directional

39. Properties of lights • Type: point, spot, directional • Position: as for any object • Color: color of light • Range: how far the light shines

40. Properties of spotlights Same as point lights, plus: • Direction • Falloff • Attenuation • Radius of inner and outer cones

41. Lights! Camera! Action! POINT OF VIEW and RENDERING

42. From local to output coordinates Untransformed Vertex World Transformation Matrix local coordinates View Transformation Matrix place object in world Projection Matrix choose point of view Output Transformed Vertex viewing frustum Programming Role-Playing Games with DirectX, by Jim Adams

43. The Matrix(actually, lots of matrices) Matrices are used for transformations: • Translate • move from one place to another • Rotate • center of rotation is important • Scale • center of scaling is important

44. x' y' z' x y z 2 0 0 0 2 0 0 0 2 = Using a matrix to transform a vertex • Vertex is represented as three coordinates • vector from origin to location of vertex • 1x3 matrix • Multiply vertex matrix by transformation matrix to get new vertex matrix What transformation does this represent?

45. Combining transformations • Transformations can be combined by multiplying matrices together • Combine matrices, then multiply each vertex in object by combined matrix. • More efficient than multiplying each vertex by each of the transformation matrices. Note: In order to represent translation with matrix multiplication you have to add an extra coordinate. You will sometimes see vertices represented by 4-element vectors, and 4x4 transformation matrices.

46. From local to world coordinates Untransformed Vertex World Transformation Matrix local coordinates View Transformation Matrix place object in world Projection Matrix Output Transformed Vertex

47. Placing an object in the world • Models usually have their center at the origin (0, 0, 0) in local coordinates. • To place a model in the world, the coordinates of all vertices in the model have to be transformed (translated, rotated, and/or scaled). • Usually the world transformation is kept in the (C++) object for the model and then applied each time the model is rendered.

48. Animation using a model's world transformation • By changing the world transformation each frame, you can animate a model: move, rotate, scale, or some combination • Since the world transformation applies to the whole model, this kind of animation affects the whole model also.

49. DirectX • Call SetTransform before drawing each model • DirectX will apply pMatrix to each vertex in the model • pMatrix will usually be some combination of translation, rotation and scaling • Game engines (e.g. Irrlicht) will provide easier ways of setting the location, orientation, and size of models. HRESULT IDirect3DDevice8::SetTransform( D3DTRANSFORMSTATETYPE State, // D3DTSWORLD CONST D3DMATRIX *pMatrix) // World matrix to set Programming Role-Playing Games with DirectX, by Jim Adams

50. From world to view coordinates Untransformed Vertex World Transformation Matrix View Transformation Matrix place object in world Projection Matrix choose point of view Output Transformed Vertex