Blending & State Machines

1. Blending & State Machines CSE169: Computer Animation Instructor: Steve Rotenberg UCSD, Winter 2004

2. Issues • The midterm is next Thursday • Project 1 sample on the web page • Hopefully, I will have some morph files and texture examples on the web page by tomorrow • Still grading project 1

3. Blending & Sequencing • Now that we understand how a character rig works and how to manipulate animation data, we can edit and play back simple animation • The subject of blending and sequencing encompasses a higher level of animation playback, involving constructing the final pose out of a combination of various inputs • We will limit today’s discussion to encompass only pre-stored animation (channel) data as the ultimate input. Later, we will consider how to mix in procedural animation…

4. Blending & Sequencing • Most areas of computer animation have been pioneered by the research and special effects industries • Blending and sequencing, however, is one area where video games have made a lot of real progress in this area towards achieving interactively controllable and AI characters in complex environments… • The special effects industry is using some game related technology more and more (battle scenes in Lord of the Rings…)

5. Keyframe Channels

6. Keyframe Access class Keyframe { float Value,Time; char RuleIn,RuleOut; float TangentIn,TangentOut; float A,B,C,D; } class Channel { float Evaluate(float time); void Precompute(); }

7. Channel::Precompute() • The two main setup computations a keyframe channel needs to perform are: • Compute tangents from rules • Compute cubic coefficients from tangents & other data • This can be done in two separate passes through the keys or combined into one pass (but keep in mind there is some slightly tricky dependencies on the order that data must be processed if done in one pass)

8. Computing Tangents • Both the incoming and outgoing tangents need to be generated for each keyframe • Tangents are generated from simple ‘rules’ • Flat: tangent = 0 • Fixed: tangent = some constant • Linear: tangent points to next/last key • Smooth: tangent equals slope formed by next & last key • The first two are trivial, the third is simple, and the fourth is pretty simple too… • Note: I use ‘v’ for tangents (velocity) instead of ‘t’ which is used for time

9. Flat Tangents • Flat tangents are particularly useful for making ‘slow in’ and ‘slow out’ motions (acceleration from a stop and deceleration to a stop) • v = 0 • • •

10. Linear Tangents • (p1,t1) v1in v0out • (p0,t0)

11. Smooth Tangents • Keep in mind that this won’t work on the first or last tangent (just use the linear rule) (p0,t0) • v1in (p2,t2) • • (p1,t1) v1out

12. Step Tangent • Occasionally, one comes across the ‘step’ tangent rule • This is a special case that just forces the entire span to a constant • This requires hacking the cubic coefficients (a=b=c=0, d=p0) • It can only be specified on the outgoing tangent and it nullifies whatever rule is on the next incoming tangent • • • •

13. Computing Cubic Coefficients • Once we have the tangents for every keyframe, we can compute the cubic coefficients (a,b,c,d) for each span • Usually, these are just stored with the keyframe at the start of the span

14. Computing Cubic Coefficients • Note: My matrix34 class won’t do this properly! • Tough! Do it yourself! Actually, all of the 1’s and 0’s in the matrix make it pretty easy to multiply it out by hand

15. Channel::Evaluate() • The Channel::Evaluate function needs to be very efficient, as it is called many times while playing back animations • There are two main components to the evaluation: • Find the proper span • Evaluate the cubic equation for the span

16. Finding the Span • Assuming we are doing a ‘random access’ of the channel, we need to first locate where the specified t value is • There are 4 main possibilities • t < tstart (before first key: use extrapolation) • t > tend (after last key: use extrapolation) • t is on a key • t is on a span between keys (most likely)

17. Finding the Span: Binary Search • A very reasonable way to find the key is by a binary search. This allows pretty fast (log N) access time with no additional storage cost (assuming keys are stored in an array (rather than a list)) • Binary search is sometimes called ‘divide and conquer’ or ‘bisection’ • For even faster access, one could use hashing algorithms, but that is probably not necessary, as they require additional storage and most real channel accesses can take advantage of coherence (sequential access)

18. Finding the Span: Linear Search • One can always just loop through the keys from the beginning and look for the proper span • This is an acceptable place to start, as it is important to get things working properly before focusing on optimization • It may also be a reasonable option for interactive editing tools that would require key frames to be stored in a linked list • Of course, a bisection algorithm can probably be written in less than a dozen lines of code…

19. Evaluating the Cubic • Once we have the span, we can access the necessary cubic coefficients • First, however, we need to turn our time t into a 0..1 value for the span (we’ll call this parameter u)

20. High Performance Channels • If 1/(t1-t0) is precomputed, we can evaluate the cubic equation with 4 additions and 4 multiplies • In fact, a,b,c, & d can actually be precomputed to include the correction for 1/(t1-t0) so that the cubic can be directly solved for the original t. This reduces it to 3+ and 3* • In other words, evaluating the cubic is practically instantaneous, while jumping around through memory trying to locate the span is far worse • If we can take advantage of sequential access (which we usually can), we can reduce the span location to a very small number of operations

21. Robustness • The channel should always return some reasonable value regardless of what time t was passed in • If there are no keys in the channel, it should just return 0 • If there is just 1 key, it should return the value of that key • If there are more than 1 key, it should evaluate the curve or use an extrapolation rule if t is outside of the range • At a minimum, the ‘constant’ extrapolation rule should be used, which just returns the value of the first (or last) key if t is before (or after) the keyframe range • When creating new keys or modifying the time of a key, one needs to verify that its time stays between the key before and after it

22. Project 3 • Details are TBD, but the basic idea will be to load an animation and play it back on a character rig • The animation file will contain an array of keyframe channels • You will have to compute the tangents from the rules and be able to evaluate the channels to play back the animation • I might require you to have a channel Draw() function as well that graphs the channel in 2D. It could display the channels of the currently selected joint as an overlay… • As extra credit, perhaps a simple channel editor (insert / delete key, move key, set tangent rule…)

23. References • “3-D Computer Graphics: A Mathematical Introduction with OpenGL”, S. Buss • 4: Averaging and Interpolation • 7: Bezier Curves • 8: B-Splines • 12: Animation and Kinematics (covers quaternions also)

24. Animation Playback

25. Poses • A pose is an array of values that maps to a rig • If the rig contains only simple independent DOFs, the pose can just be an array of floats • If the rig contains quaternions or other complex coupled DOFs, they may require special handling by higher level code • Therefore, for generality, we will assume that a pose contains both an array of M≥0 floats and an additional array of N≥0 quaternions

26. Animation Clip • Remember that the AnimationClip stores an array of channels for a particular animation (or it could store the data as an array of poses…) • This should be treated as constant data, especially in situations where multiple animating characters may simultaneously need to access the animation (at different time values) • For playback, animation is accessed as a pose. Evaluation requires looping through each channel. class AnimationClip { void Evaluate(float time,Pose &p); }

27. Animation Player • We need something that ‘plays’ an animation. We will call it an animation player • At it’s simplest, an animation player would store a AnimationClip*, Rig*, and a float time • As an active component, it would require some sort of Update() function • This update would increment the time, evaluate the animation, and then pose the rig • However, for reasons we will see later, we will leave out the Rig* and just have the player generate and output a Pose

28. Animation Player class AnimationPlayer { float Time; AnimationClip *Anim; Pose P; public: const Pose &GetPose(); void Play(AnimationClip &clip); void Update(); };

29. Animation Player • A simple player just needs to increment the Time and access the new pose once per frame • The first question that comes up though, is what to do when it gets to the end of the animation clip? • Loop back to start • Hold on last frame • Deactivate itself… (return 0 pose?) • Send a message…

30. Animation Player • Some features we may want to add for a more versatile animation player include: • Variable playback rate • Play backwards (& deal with hitting the beginning) • Pause • It’s kinda like a DVD player…

31. Animation Player • The animation player is a basic component of an animation blending & sequencing system • Many of these might ultimately be combined to get the final blended pose. This is why we only want it to output a pose • By the way, remember the issue of sequential access for keyframes? The animation player should ultimately be responsible for tracking the current keyframe array (although the details could be pushed down to a specific class for dealing with that)

32. Animation Player • As we will use players and static poses as basic components in our blending discussion, we will make a notation for them:

33. Animation Blending

34. Blending Overview • We can define blending operations that affect poses • A blend operation takes one or more poses as input and generates one pose as output • In addition, it may take some auxiliary data as input (control parameters, etc.)

35. Generic Blend Operation

36. Cross Dissolve • Perhaps the most common and useful pose blend operation is the ‘cross dissolve’ • Also known as: Lerp (linear interpolation), blend, dissolve… • The cross dissolve blender takes two poses as input and an additional float as the blend factor (0…1)

37. Cross Dissolve • The two poses are basically just interpolated • The DOF values can use Lerp, but the quaternions should use the ‘Slerp’ operation (spherical linear interpolate)

38. Cross Dissolve: Handling Angles • If a DOF represents an angle, we may want to have the interpolation check for crossing the +180 / -180 boundary • Unfortunately, this complicates the concept of a DOF (and a pose) a bit more. Now we must also consider that some DOFs behave in different ways than others

39. Cross Dissolve: Quaternions • Also, for quaternions, we may wish to force the interpolation to go the ‘short way’:

40. Cross Dissolve: Stand to Walk • Consider a situation where we want a character to blend from a stand animation to a walk animation

41. Cross Dissolve: Stand to Walk • We could have two independent animations playing (stand & walk) and then gradually ramp the ‘t’ value from 0 to 1 • If the transition is reasonably quick (say <0.5 second), it might look OK • Note: this is just a simple example of a dissolve and not necessarily the best way to make a character start walking…

42. Cross Dissolve: Walk to Run • Blending from a walk to a run requires some additional consideration…

43. Cross Dissolve: Walk to Run • Lets say that we have a walk and a run animation • Each animation is meant to play as a loop and contains one full gait cycle • They are set up so the character is essentially moving in place, as on a treadmill • Let’s assume that the duration of the walk animation is dwalk seconds and the run is drun seconds • Let’s also assume that the velocity of the walk is vwalk and run is vrun (these represent the speed that the character is supposed to be moving forward, but keep in mind, the animation itself is in place)

44. Cross Dissolve: Walk to Run • We want to make sure that the walk and run are in phase when we blend between them • One could animate them in a consistent way so that the two clips both start at the same phase • But, let’s assume they aren’t in sync… • Instead, we’ll just store an offset for each clip that marks some synchronization point (say at the time when the left foot hits the ground) • We’ll call these offsets owalk and orun

45. Cross Dissolve: Walk to Run • Let’s assume that f is our dissolve factor (0…1) where f=0 implies walking and f=1 implies running • The resulting velocity that the character should move is simply: • v'=Lerp(f,vwalk,vrun) • To get the animations to stay in phase, however, we need to adjust the speeds that they are playing back • This means that when we’re halfway between walk and run, the walk will need to be sped up and the run will need to be slowed down

46. Cross Dissolve: Walk to Run • As we are sure that we want the two to stay in phase, we can just lock them together • For example, we will just say that if twalk is the current time of the walk animation, then trun should be:

47. Cross Dissolve: Walk to Run • To speed up the walk animation appropriately, we will define a rate rwalk that the walk animation plays at (default would be 1.0)

48. Basic Math Blend Operations • We can also define some blenders for basic math operations:

49. Basic Math Blend Operations • Its not always obvious how to define consistent behaviors between independent DOFs and quaternions • Quaternion addition and subtraction don’t really give an expected result • Addition of orientations implies that you start with the first orientation and then you do a rotation from there that corresponds to how the second orientation is rotated from neutral • This behavior is more like quaternion multiplication (although quaternion multiplication is not commutative)

50. Add & Subtract Blenders • A reasonable behavior for an add blender could be: • For subtraction, we could multiply by the conjugate of the quaternion