Source Multicore

1. 1 November 2006 Source Multicore

2. New Source Features Dynamic Shadow Mapping New Foliage System Cinematic Physics Version 2 of Facial Animation System Dynamic Scripted Sequences Particle System 64-Bit Source Engine Reworked Character Lighting Model Companion AI (Alyx in Episode One) Real-Time statistics gathering Automatic game cache defragmentation Multicore

3. New Features in Source Multiple Approches to multi-core games Source is implementing Hybrid-Threading Close to linear improvements Pretty but dumb era is ending, scalability we've seen in graphics will now apply to rest of the game Hybrid Threading Source will ship before Episode Two You'll get a version of this engine to take home to test with

4. Multicore Most significant development since 3D cards Huge potential Huge challenge

5. Multicore Most significant development since 3D cards Huge potential Huge challenge The decisions faced with multiple cores How we are using multiple cores

6. Multicore Most significant development since 3D cards Huge potential Huge challenge The decisions faced with multiple cores How we are using multiple cores Four cores is more than twice as interesting as two cores

7. Challenges

8. Challenges Games always want 100% CPU utilization

9. Challenges Games always want 100% CPU utilization Games are inherently serial

10. Challenges Games always want 100% CPU utilization Games are inherently serial Decades of experience in single threaded optimization

11. Challenges Games always want 100% CPU utilization Games are inherently serial Decades of experience in single threaded optimization Millions of lines of code written for single threading

12. Strategies Threading model Threading framework Application of cores

13. Theading Models Single threading Coarse threading Fine grained threading Hybrid threading

14. Single threading Easy Obsolete

15. Coarse threading Put whole systems on cores Pretty easy, “multiple single threads” Stay partially serialized, or double buffer

16. Coarse threading: Early experimentation Client User input Rendering Graphics simulation Server AI Physics Game logic

17. Coarse threading: Early experimentation Experiment: run client and server each on own core

18. Coarse threading: Early experimentation Experiment: run client and server each on own core Benefits: forced to confront systems that are not thread safe or not thread efficient

19. Coarse threading: Early experimentation Experiment: run client and server each on own core Benefits: forced to confront systems that are not thread safe or not thread efficient Outcome: Can approach 2x in contrived maps

20. Coarse threading: Early experimentation

21. Coarse threading: Early experimentation

22. Coarse threading: Early experimentation Experiment: run client and server each on own core Benefits: forced to confront systems that are not thread safe or not thread efficient Outcome: Can approach 2x in contrived maps More like 1.2x in real single player Added latency to single player game

23. Coarse threading: Early experimentation Experiment: run client and server each on own core Benefits: forced to confront systems that are not thread safe or not thread efficient Outcome: Can approach 2x in contrived maps More like 1.2x in real single player Added latency to single player game Opened door to improved listen servers

24. Coarse threading Put whole systems on cores Pretty easy, “multiple single threads” Stay partially serialized, or double buffer Scales poorly Partially idle cores Synchronization, or lag Entirely idle cores

25. Fine grained threading Divide many small identical tasks across cores E.g., take a loop that updates state of 1000 objects and perform 1000/N on each core for N cores Moderate difficulty Scales well… Tricky if cost of each unit is variable Memory bandwidth Limited problem domains

26. Fine grained threading Leverage multicore in production tools: VMPI

27. Fine grained threading VVIS – Visibility calculations VRAD – Lighting calculations

28. Fine grained threading

29. Hybrid threading Performance tuned for mid-level work sharing Not splitting sets of very small operations over cores Not putting whole systems onto cores

30. Hybrid threading Use the appropriate tool for the job Some systems on cores (e.g. sound) Some systems split internally similar to coarse Split expensive iterations across cores fine grained Queue some work to run when a core goes idle Most difficult Scales well Maximum core utilization

31. Hybrid threading: Rendering Rough pipeline Build world lists Build object lists Graphical simulation (particles, ropes, sprites) Update animations Compute shadows Draw Once for every “view” Player’s POV TV monitors Water reflections Many times CPU bound

32. Hybrid threading: Rendering Revised pipeline Construct scene rendering lists for multiple scenes in parallel (e.g., the world and its reflection in water) Overlap graphics simulation Compute character bone transformations for all characters in all scenes in parallel Compute shadows for all characters Allow multiple threads to draw in parallel Serialize drawing operations on another core

33. Threading Tools Implementing Hybrid Threading Operating system: pools, synchronization Compiler extensions: OpenMP, fine threading Tailored tools Programmers solve game development problems, not threading problems

34. Operating system Too low level Prone to error Lots of stalling Unpredictable scheduling Unpredictable cost

35. Compiler extensions OpenMP Focused on fine threading Lack of control Implementation interferes

36. Tailored tools: Game Threading Infrastructure Custom work management system Aimed at gaming problems, intuitive for game programmers Focus on keeping cores busy Thread pool: N-1 threads for N cores Support hybrid threading Function threading Array parallelism Multiple work modes Opportunistic core utilization Queued core utilization

37. Tailored tools: Game Threading Infrastructure The simple thing is the worst thing “Lock-free” algorithms Never leave cores idle waiting on other cores Leverages atomic write primitives of the CPU Under the hood of all services and data structures See: http://en.wikipedia.org/wiki/Lock-free_and_wait-free_algorithms Example: the spatial partition

38. Application of cores

39. Application of cores Dual core CPUs: framerate

40. Application of cores Dual core CPUs: framerate Quad core CPUs: new experiences

41. Application of cores Dual core CPUs: framerate Quad core CPUs: new experiences Richer visuals Improved simulation Richer AI

42. Application of cores: Particle Simulation

43. Application of cores: Particle Simulation Not simply a GPU issue Interactivity Presence

44. Application of cores: Particle Simulation Use cores to run multiple particle systems in parallel Individual particle systems using multiple cores

45. Application of cores: Particle Simulation More complicated systems Interactive particle systems Particles with gameplay implications Like rigid body physics, reinforce consistency of world

46. Particle Simulation Benchmark

47. AI Traditionally strict CPU limits Interesting combinations of minimalist algorithms “What could we do if extra CPU were given to AI?”

48. AI Better framerate Run AI in parallel with other systems Parallel agent execution

49. AI Example: Parallel Animation

50. AI Better framerate Increased sophistication Without hitching by asynchronously running on secondary cores