Graphics

1. Graphics CS 3540

2. Graphics in games • None • ASCII art (so called...) • Images • Bitmaps • Vector • 2D animation • Video • 3D animation

3. 3D Graphics Commonly used libraries: • OpenGL www.opengl.org • Microsoft DirectX http://msdn.microsoft.com/directx Many other libraries, wrappers, and game engines are built on one or both of these libraries.

4. Open standard Provides interface to hardware acceleration Portable Widely used in commercial and non-commercial games No native image or 3D model file format GLUT Proprietary Provides interface to hardware acceleration Only on Microsoft platforms Widely used in commercial and non-commercial games Native .x 3D model format, loaders for common bitmap formats Windows API Direct Play, Direct X Audio, etc. OpenGL vs. DirectX "OpenGL is the only truly open, vendor-neutral, multiplatform graphics standard." http://www.opengl.org/about/overview/

5. Fun with 3D libraries... Both OpenGL and DirectX are complicated libraries and learning how to use them well is not easy. DirectX, in particular, has gone through major changes that obsolete many earlier books and web sites. Microsoft's documentation can also change in confusing ways.

6. Making a game with3D graphics Suppose that I want to make a 3D display for UV Bots (which I did...) How do I do that? We'll look at how do that with some references to Unity and other game engines and libraries.

7. Nine easy steps Nine moderately difficult steps • Find or make 3D models • Initialize software and hardware • Load models • Add models to world • Lights • Camera • Action • Interaction • This, that, and the other Nine frustrating steps that will make you wish you were using Klingon pain sticks instead and will make you want to go into something simple like neurosurgery... Unity makes these steps much easier, but it’s good to know at least a little bit about what’s happening behind the scenes.

8. Finding models Google (what else?) Call in favors from your artistic friends (you have lots of those, right?) Borrow them from other games (but only if it's ethical) Making models Blender 3D blender.org Milkshape 3D Wings 3D www.wings3d.com Various commercial packages, most of which are very expensive 1. Find or make 3D models

9. 3D model formats • Bad news: There are no "standard" formats for 3D model files that are comparable to JPEG or PNG. • Worse news: Converting between 3D file formats is a pain and the model will probably not look the same after conversion.

10. 3D model formats • Text or binary? • What documentation is available? • Is a file format specification available? • How complicated is the file format? • What software can read/write the format? • Modeling software • Conversion software • Loaders and game engines • Does it include animation?

11. 3D model formats • .3ds, .max - 3DS max format • .x - Direct X • .dxf - created for AutoCAD by AutoDesk • .obj - Wavefront (an ASCII format) • .mdl – Quake • .fbx – Used by AutoDesk and others • .dae – COLLADA www.khronos.org/collada/ plus many more fileformat.info

12. .x DirectX • Both text and binary formats • DirectX SDK has loaders • Includes animation • Specification is available but is not for the faint-hearted • Various importers, exporters, converters, but many are flaky Note: It’s been a long time since I’ve used or read about .x files.

13. .obj Wavefront • Wavefront (which merged with Alias, which was bought by AutoDesk) • Text format • Relatively simple • No animation • Blender can read/write, Unity can load

14. .ms3d Milkshape • Binary format • Includes animation • Reading and writing: • Chumbalum's web site (http://chumbalum.swissquake.ch/) has C++ code to load .ms3d files • Blender cannot read or write

15. COLLADA From the Khronos Group web site: www.khronos.org/collada/ • COLLADA transports 3D assets between applications • Enables binding of diverse DCC and 3D processing tools into a production pipeline • COLLADA is a XML database schema for 3D assets • COLLADA is an intermediate language - not a delivery format and not a scene graph • COLLADA can be lossless • Retains all information - even multiple versions of the same asset • COLLADA has comprehensive encoding of visual scenes • Including geometry, animation, shaders and physics • COLLADA is an open, archive-grade format that retains meta information • When your DCC tool upgrades, you keep your assets

16. File formats in Unity • Can read .FBX, .dae, .3DS, .dxf, and .obj files • Natively imports Maya files • “Natively” imports Blender files (via FBX) http://unity3d.com/support/documentation/Manual/HOWTO-importObject.html

17. 2. Initialize software and hardwareDirectX • One book has eight fun-filled pages on how to initialize DirectX, MSDN has a lot more than that... • Many options, choosing the wrong options can leave you with a blank screen • Beware of COM • Make sure you deallocate, close, destroy everything you allocate, open, or create if you want to avoid memory leaks

18. 2. Initialize software and hardwareOpenGL • Setup is relatively easy if you use GLU and GLUT, but can still be tricky

19. Encapsulate! • Set-up code can be messy, so hide the details. • Encapsulation makes it easier to port to another OS or another graphics library.

20. 3. Load models Choices: • Write your own loader • Not easy since 3D file formats are complicated and often not well documented • Use someone else's source code • Hard to say how well it will work • Use a game engine • Only an option if the engine supports your format

21. Components of 3D models • Vertices - points in 3D space • Edges - connect vertices • Faces/Polygons • often triangles, sometimes quads • Meshes • Normals - Which way is up? (or out?) • Used for shading

22. More components of models • Hierarchies - parent/child • Colors • Textures • Combinations of other materials • Images - often stored in separate file • Must be mapped to mesh • Bones • Animation - key frames

23. 4. Add model to the world • Left-handed or Right-handed? • Location, size, and orientation • Parent-child hierarchy

24. Left-handed vs. Right-handed y y left-handed x z z x right-handed "The right-handed system is universally accepted as the standard in mathematics and the physical sciences." wikipedia but... 3DS max vs. DirectX Unity is left-handed

25. Lost in Space(s) • Model space (or local coordinates) • Coordinates are relative to local origin • Used to create model • World space • Coordinates are relative to global origin • Used to place models within the world • Screen space • Projected into 2D plane

26. y y y x x z z x z Local vs. global coordinates The front turrets have the same local coordinates for both instances of the model. The front turrets will have different global coordinates for each instance of the model. How do the x, y, and z global coordinates compare for these two instances of the bot model?

27. Parent/Child hierarchy When the bot turns, the turret should turn also, but sometimes the turret should turn when the bot doesn't. • Frames • Scene graph • Submeshes

28. Adding models in Unity Some links to the Unity User Guide • Asset Import and Creation • Prefabs

29. Lights! Camera! Action! LIGHTING Some material in this section comes from Introduction to Game Programming, edited by Steve Rabin

30. Questions • How much light is incident on a surface? • From which direction does the light come? • How is the light absorbed, reemitted, and reflected? • Which outgoing rays reach the eye of the viewer?

31. Answers(or at least ways to find the answers) • Forward tracing • trace every photon emitted by a light source and figure out where it goes • photon mapping: high quality but extremely slow • Backward tracing • trace a hypothetical photon that hit the eye to see which object it came from • raytracing: practical, but not for real time rendering http://www.povray.org

32. Answers(or at least ways to find the answers) • "Middle-out" evaluation • "Given this bit of surface, how much light came from these sources of light and ended up hitting the eye?" * • Looks at triangle currently being drawn • Real time rendering has to compromise between speed and quality • p. 471, Introduction to Game Programming, • edited by Steve Rabin

33. Kinds of lighting We'll look at how different kinds of lighting affect this scene. Note: These pictures were made in Blender, not with real-time rendering.

34. Ambient light • Light that is not attributed to any particular source • Can be a reasonable approximation for some multi-light situations, especially indoors • Unity: Edit->Render Settings What happens if your scene has too much ambient light?

35. Too much ambient light

36. Diffuse lighting

37. Specular lighting

38. Hemisphere lighting Sun + sky + ground Useful approximation for some outdoor scenes direct sunlight diffracted sunlight reflected sunlight

39. Environment map • Approximates reflections • Cube map: • Put cube representing environment around the object being rendered. • Trace vector from eye to object, reflect from object, and see where it strikes the cube. • Same idea as hemisphere lighting but can contain images or more varied colors

40. Shadows • Can be done with ray-tracing. • Some games have shadows.

41. Spot Types of lights Point Directional

42. Properties of lights • Type: point, spot, directional • Position: as for any object • Color: color of light • Range: how far the light shines

43. Properties of spotlights Same as point lights, plus: • Direction • Falloff • Attenuation • Radius of inner and outer cones

44. Lights in Unity • Point, directional, spot • Shadows (only in the Pro version) • Ambient light: Edit->Render Settings http://unity3d.com/support/documentation/Components/class-Light.html

45. Lights! Camera! Action! POINT OF VIEW and RENDERING

46. From local to output coordinates Untransformed Vertex World Transformation Matrix local coordinates View Transformation Matrix place object in world Projection Matrix choose point of view Output Transformed Vertex viewing frustum Programming Role-Playing Games with DirectX, by Jim Adams

47. The Matrix(actually, lots of matrices) Matrices are used for transformations: • Translate • move from one place to another • Rotate • center of rotation is important • Scale • center of scaling is important

48. Using a matrix to transform a vertex • Vertex is represented as three coordinates • vector from origin to location of vertex • 1x3 matrix • Multiply vertex matrix by transformation matrix to get new vertex matrix x' y' z' 2 0 0 0 2 0 0 0 2 x y z What transformation does this represent? =

49. Combining transformations • Transformations can be combined by multiplying matrices together • Combine matrices, then multiply each vertex in object by combined matrix. • More efficient than multiplying each vertex by each of the transformation matrices. Note: In order to represent translation with matrix multiplication you have to add an extra coordinate. You will sometimes see vertices represented by 4-element vectors, and 4x4 transformation matrices.

50. From local to world coordinates Untransformed Vertex World Transformation Matrix local coordinates View Transformation Matrix place object in world Projection Matrix Output Transformed Vertex