Computer Animation in Games

1. Computer Animation in Games Ryan Anderson Senior Technology Engineer Rainbow Studios For The Phoenix IEEE Society Wednesday, November 4, 2009

2. Tonight’s Presentation • Animation Challenges (or, why we couldn’t play Toy Story) • 2D Animations – Past and Present • 3D Animations – Past, Present, and Future • Making the Game

3. Games vs. MoviesWhy can’t we play what we see? • It’s a question of resources: • Processing Power • RAM • ROM • Human Resources

4. Pixar Toy Story • 1995 • 2 – 15 hours spent animating each frame • 4 hours rendering each scene • Sun SPARCStation (up to 200MHz, 512MB RAM)

5. Console Games – 1995 - MGS • Playstation 1 • 33MHz • 2MB RAM (1MB VRAM) • Games must run at least 30 fps (33ms frames)

6. In The Beginning 2D Animations Work just like a flip book Low processing power “Lots” of RAM –for the time

7. Pacman Frames 3 Frames Don’t forget rotated frames Fortunately for us, frame 3 works in both orientations

8. 5 Frames for Pacman • Most Hardware could flip picture for you, no need to animate both directions • Saves you half the memory! • More savings from palette swapping ghosts.

9. How much is this costing us? Most textures were palette based, cost is determined by number of colors • 256 colors (where available) 1Byte per pixel • 16 colors .5 Bpp • 4 colors .25 Bpp • 2 colors .125 Bpp (or, 1 bit per pixel)

10. How big was Mario? The Map The Palette 0 1 2 3 32x32 Mario 4 colors (2 bits per pixel) 256 Bytes per frame Original Mario was only 16x16 (64 bytes per frame)

11. And Luigi was free! • No need to do two sets of animations, just change the palette. • Very common in early computer games to fight “new” monsters and bosses that were palette swaps of old ones.

12. How is 2D animation still used in 3D games? Billboards – Camera facing quads with 2D or rendered 3D textures • Perspective problems: Might have to render from a lot of angles • Can constrain camera angles to help prevent this

13. One Example of Current Use of 2D Nintendo DS • Limited to 2048 triangles per frame • Pre-rendering characters can save a LOT of polygons • Fast cart access allows swapping in and out: no need to hold them all in RAM

14. Billboarded Particles Internal pictures removed Particles are billboarded rectangles that change in shape and location, and can play frames of animations at the same time to give off the appearance of additional motion.

15. Billboarded Crowd (paper people) 10,000 crowd members can really add up (even if low poly) Render from different angles, then display on camera facing polygons

17. 3D Animations Mesh of triangles Each vertex usually has at least a position, texture coordinate, and a normal Modern games can add multiple texture coordinates, tangent vectors, and more!

18. Morph Target Animations Keep a mesh per frame Blend vertex positions to animate Can start to use a lot of RAM fast Can have linearity issues when blending

19. Morph Target Blending Mesh minute hand at noon Mesh minute hand at 15 after Blend meshes to get 7 after Blended positions cause a shortened hand You can add more frames to minimize this effect, but at additional RAM costs

20. Morphs Advantages Allows artists to give exact positions to vertices. • Great for cloth and facial animations Fast on older processors (1 vector scale, 1 vector madd) – Used with such formats as the Quake 3 .md3

21. Morph Disadvantages Blending artifacts Size gets big fast • Size of vertex * number of vertices in mesh * number of animation frames Sometimes difficult to create – often created in tools using skeletal animations anyway. Difficult (if not impossible) to do IK, ragdoll, etc.

22. Skeletal Animations Pretty much how we do things now Still uses a mesh of triangles Every vertex is associated with a mathematical “bone” Animation now takes place in 2 parts • Animate the bone frame • “Skin” the mesh to the bone

23. Bone Frames Each bone has exactly 1 parent (up to root node) but can have multiple children Creates a hierarchy of bones, each with a local rotation and positions Root Bone 1 Bone 2 Local bone matrices Concatenated world matrix

24. Rotate bone 1 by 90 degrees Bone 2 Root Bone 1 Local Matrices Concatenated World Matrices

25. Skinning to a bone: Associate a vertex with bone 2 V Each vertex must be in its associated bones local space So, apply the inverse of the bone’s world space transform to each vertex =

26. Now, when a new world transform is calculated for the bone, apply it to the local space position for every associated vertex V Root =

27. Weighting In early games, it was not uncommon to associate each vertex with just 1 bone, due to processing cost. Imagine a flexible mesh over our earlier 2 bone manipulator (kind of like a square-ish thumb), made up of 4 vertices. Now, if we associate each vertex with a bone, V1 and V2 pretty obviously go with bone 2, but what about V3 and V4? V3 V1 Bone 1 Bone 2 V2 V4

28. V3 and V4 if associated with Bone 1 V2 V3 V4 This looks painful!

29. V3 and V4 if associated with bone 2 V1 V2 V3 V4 A bit better, but still very distorted around the joint.

30. What we need to do is allow V3 and V4 to be “weighted” to both bones 1 and 2 V1 V2 V3 V4 Where the sum of all the weights is equal to 1

31. A model in its “T-Pose” and the associated bone hierarchy Internal pictures regretfully removed

32. Now we only need 1 copy of the mesh in memory, and can store animations as a series of bone rotations and translations. • This is important, because modern models can be 10,000 – 20,000 polygons with large vertices. • “Skinning” can still be CPU intensive, but is generally done by the GPU on modern hardware • The Xbox360 can transform millions of vertices each frame.

33. Size still matters Computers have more RAM and processing power these day, but that just means game designers want to pack even more into them. Often games will even do their cut scenes as animations rather than video. • Metal Gear Solid 4 for the PS3 had HOURS of animated cut scenes.

34. Animation Compression Store rotations as Quaternions and not Matrices (4 numbers instead of 9) • Not Euler angles! • Makes interpolating rotations much faster Store 16 bit numbers instead of 32 bit numbers for rotations • Quaternions are normalized anyway, so our values don’t get above a 1 • Usually 1/16000 is plenty accurate for animations

35. Key Frame Reduction Data Compression is still not enough. Animations usually sampled at 30 frames per second. We could cut our animation size in half by simply halving our sample rate and interpolating between them, but this isn’t good enough. • Loss of quality • We can do better!

36. Frame reduction is very similar to any other sampling problem • Sample rate of change per bone over frames • Drop frames where a tolerance value is met • Be careful of 180 degree rotations! Other compression methods also exist • Wavelets

37. Issues with skeletal animations Labor intensive to create • Mocap can help, but not always appropriate Limited motions cause blending artifacts • Targeting objects (grab, punch, kick, etc.) • The dreaded foot-slide Some solutions are …

38. Procedural Animations… for today Lots of different kinds • Headlook (simplest kind in our game) • Jersey Flap • IK Hands and feet Spine moving with the bumps • Ragdoll … Crashes are fun! But how would you stand up from this?

39. Procedural Animations… moving forwards Define what a motion is, generate an animation for it on the fly • Actions such as punch, kick, dodge • Standing up from various fallen positions • Define a walking path with start and endpoints to avoid foot-slide

40. Some procedural videos

41. Creating MX vs. ATV Physics driven rather than animation driven • The game state controls the animation state • Animations are blended together according to a tree traversal of physics based inputs

42. Animation Based Event Driven Games Game state based on animation states Events in games cause state traversal Faster content creation, scripting possible, less programmer time Havok Behavior is a recent example of middleware supporting this

44. Questions, comments, or complaints?