Gaming Technologies

1. Gaming Technologies Craig Peeper Software Architect Windows Graphics & Gaming Technologies Microsoft Corporation

2. Overview • Games Yesterday & Today • Game Components • PC Platform & WGF 2.0 • Game Trends • Big Challenges in Game Creation • Summary

3. Games Yesterday & Today Quake 2 - 1997 Prince of Persia - 1989 Doom - 1993 Max Payne - 2001 Far Cry - 2004 Age of Empires III

4. Game Components Content + Engine Platform

5. Content Components sound effects scripting • as well as … • user interface story lighting Content modeling level design animation texturing / surfacing visual technologies

6. Content Creation • Tool driven • Combination of tools forms “Art Pipeline” • Character Pipeline • 3D Model/Sculpting • Animation/Rigging • Skinning • Texturing • Lighting/Shading • An artist often specializes on one part of pipeline

7. Game Engine Components AI Graphics Physics Engine • as well as … • installation • patching • resource management • multithreading • disk i/o • state save/restore Networking + I/O Audio

8. Game Engines • Studio chooses to build or buy • Quake Source, Unreal engines • Renderware, Gamebryo middleware • Often come with great authoring tools (level editors, etc.) • Componentized software • May buy specialized components • How many people can write a physics engine? • Video codecs, etc • Engine may be optimized for game genre/style • Engine bounds range of content • Global illumination, terrain, indoor scenes

9. Game Platform Components CPU + Memory Graphics Storage SW + HW Platform Networking + Input Audio

10. Game Platforms • Console • 10M – 100M installed base • “fixed platform” • Moderately powerful hardware • Standard & High definition video resolution (1080i, 720p) • Controller, wheel (& keyboard, mouse) input • Refreshed every 4-5 years • 10’ experience on TV

11. Game Platforms (cont.) • PC • 600M installed base (running Microsoft Windows) • Non-uniform platform (video cards, memory, CPU) • Most powerful hardware • Least powerful too • Very high resolution (1600x1200, 1920x1080, …) • Keyboard, mouse (& controller) input • 2’ & 10’ (desk vs living room) • Notebook (portable gaming) • Upgraded yearly!

12. Other Platforms • Hand-held • Dedicated consoles (PSP, Gameboy) • Controller-like input • Mobile phones • 700M phones/year • Non-uniform platform • Keypad input • Modest resolution (320x200 … 640x480) • 1995-class PC graphics • Arcade • Industrial version of home console?

13. PC as a Platform • Large installed base • Least expensive to publish on • No platform royalties • Large hw variation, rapidly evolving • Large number of support calls • Supports rapid innovation • Profit from selling PC hardware! • Games must span variability • Influences content & engine • Reach versus development cost • Challenges • Abstract hardware differences in platform • Support innovation

14. New Direction for PC Platform • Reduce “unnecessary” hardware variation • Programmable hardware causes new problems • Shading programs are part of content not engine • Shading programs more complex • Content must be portable & reusable • Approach: • Specify details that matter • Representation, precision, accuracy • Don’t specify non-observable implementation details • Use virtual machine model

15. Windows Graphics Foundation 2.0 • Maintain consistent feature set • Eliminate capability bits (optional features) • Well defined behavior • Multiple implementations produce same result • E.g., IEEE-754 arithmetic • Migrate legacy fixed functions to shaders • Fixed-function lighting, alpha test, etc • Consistent shader programming model • Vertex and pixel shaders use common instruction set • Obsolete assembly level programming • Add optimizations where large benefit • Scenario-specific formats (high dynamic range, normal maps)

16. Innovation in WGF 2.0 • Integer & floating-point processing • More general data flow • Stream out to memory after vertex processing • Unify textures, render targets, vertex buffers • “Arrayed” resources • Select array element to render to in shader • Enables single-pass render to cube map • New pipeline stage (geometry shader) • Whole-primitive processing

17. Geometry Shader • Shader stage that “sees” whole primitive • 3 vertices of a triangle (or 2 vertices of a line) • As well as adjacency (6 vertices per triangle) • Can compute per-primitive data • Pass to pixel shader • Barycentric parameters • Plane equations • Can amplify data • Extrude edges, expand points, generate shells • Find silhouette edges • Limited amplification (not a general tessellator)

18. WGF 2.0 Pipeline fixed programmable memory Constant Constant Constant Input Assembler Vertex Shader Geometry Shader Setup Rasterizer Pixel Shader Output Merger Sampler Sampler Stream out Sampler Vertex Buffer Index Buffer Texture Texture Stream Buffer Texture Depth Stencil Render Target Memory

19. Game Trends • 2004-2005 Record sales • Visually stunning titles • Half Life 2, Halo 2, Far Cry, Doom 3 • Largest worlds • Most detailed levels of realism • Not just graphics: AI, physics, animation • Longest development times • Largest development costs • Bulk of cost in producing content • Increasing ratio of artists : programmers

20. Trends - Software • Demand for larger worlds • Demand for more detail • Floppies vs CDs vs DVDs → HALO2 4.2GB • HD-DVD/Blueray → 20GB • Rising development cost • Content creation is the bottleneck • $10M content budget • Art Pipeline is not scaling • Amortize cost over multiple platforms

21. Trends – Hardware • CPU performance • Clock speed brick wall • Transition to “multi-core” CPUs • Games are CPU-limited • Games are tuned until they are not GPU-limited • PC & consoles • GPU performance • 2x increase every 2 years

22. HardwareLots of Room To Grow? • Enthusiast GPU today (soon): • 1-2 Teraflops • 110-90nm • ~300-350M transistors • ~80 Watts • Increasing attention on • power • efficiency

23. Game Development Challenges • Creating & Managing Detail • Scalable process for content creation • CPU overload • Multi-core not a panacea • Volume of data • Effective run-time management of data • Tool power & expressiveness • Traditional tools not powerful enough

24. GPU as Solution • How best to use performance increases? • CPU bottlenecks • Offload more processing to the GPU? • Simulations, physics? • Content • Support greater detail • Don’t create more work for artists • Procedural detail?

25. Procedural Detail • Techniques used in film production • Scene Composition & Post Processing • Translucency & Antialiasing • Filter & Blend • Texture Detail • Procedural generation • Modeling & Animation • Tessellation + Displacement/Normal Maps

26. Scene Composition & Post • “Post” effects becoming common place • HDR, bloom, tints, grain, motion blur, depth of field …. • Global atmosphere • Don’t merge into individual assets • Next step, on the fly • Apply to individual scene elements • Composite scene elements together • Use to “break up” the scene – detailing

27. Transparency • Traditionally render back to front with blend • Sort on CPU • Interactions with other algorithms (e.g., shadows) • Different algorithms for • Objects with feathered edges (trees) • Transparent objects (windows) • Volumetric effects (particle systems) • Opacity versus Coverage • Use alpha to represent both • Can we solve with hardware?

28. Unsorted Transparency • Traditionally solved with a-buffer algorithm • Save all of the pixel fragments for each pixel • Resolve at end of frame • Arbitrary number of fragments per pixel  • Lists (linked), dynamic allocation • Look at “good enough” solutions • Save some maximum number of fragments • Fixed-function or programmable solution? • Fixed-function more efficient, but extra hardware

29. Texture Maps • Resolution increasing rapidly • 4Kx4K high-resolution maps • Nearing the 8Kx8K used for film • Too much work to paint this much detail • Don’t paint individual skin pores • Digital camera capture not the answer • 4Kx4K RGBA = 32MB @ 2x compression

30. Texture Maps • GPU virtual memory helps with size, but… • Can we ever really store enough resolution? • A better way? • Use procedural texture techniques • Shaders now expressive enough • Do they have enough performance? • Artists concentrate on form • Algorithmically add detail • “paint” algorithm ids

31. Surfaces and Tessellation • Increasing character complexity • 2K → 5K → 15K triangles/character • Increased skinning and morphing complexity • k weights for each vertex • Hand-tuned for animation • Silhouette edges still look poor • Compared to lighting and shading quality • Alternative – use higher-order surfaces • Beziers, Catmull-Clark subdivision, NURBS, … • Many studios already model with surfaces • Convert to triangles when packaging content

32. Tessellation Advantages • Animate and skin the control mesh • Less computation than per-vertex • Tessellate the resulting control mesh • Combine with displacement mapping • Use for additional geometric detail • Adds detail to silhouette edges • Use normal maps for fine shading detail • Combine with adaptive tessellation?

33. Skinned Control Mesh

34. Summary • Games increasingly more complex • World size, detail • Bulk of cost in producing content • Content creation process not scaling • Amortize cost over multiple platforms • Try to reduce cost of additional detail • Run-time generation of detail • Improved tools for controlling detail • Good areas for additional research

35. Games Tomorrow

36. Questions