Behavioral Animation

1. Behavioral Animation 數位內容學院 遊戲開發研究班第一期 3D圖學 沈育德 Edward Shen May 21, 2005

2. Course Information • Date: 5/19, 5/21, 5/26, 5/28 (2005) • Lecturer: Edward Yu-Te Shen 沈育德 • Course Website: http://graphics.csie.ntu.edu.tw/~edwards/dokuwiki/doku.php?id=lecture

3. Lecturer • 沈育德, Edward Shen • PhD Candidate (1st year) • Graphics group, Dept. of CSIE, National Taiwan University • http://graphics.csie.ntu.edu.tw/~edwards/ • edwards@cmlab.csie.ntu.edu.tw

4. Course Overview • This course course introduces several topics related to behavioral animation, including real-time crowd animation, collision detection and bounding boxes, self-generating behaviors through various techniques such as reinforcement learning, genetic algorithms, and so on. • I hope that the class helps the students to gain general knowledge in the related areas as well as to apply such techniques in computer games and animated films development. The four classes will address more about design methodologies than practical programming issues.

5. Schedule • Artificial Lives: Fishes and Evolving Creatures (Saturday, May 21) • Introduction to Artificial Lives • Artificial Fishes and Reinforcement Learned Behavior • Evolving Creatures and Genetic Algorithm • Simulating Human Crowds (Thursday, May 26) • Collision Detection and Bounding Boxes (Saturday, May 28)

6. Artificial Lives: Fishes and Evolving Creatures • Introduction to Artificial Lives • Artificial Fishes and Reinforcement Learned Behavior • Evolving Creatures and Genetic Algorithm

7. Applications Source:http://ligwww.epfl.ch/research/research_index.html http://www.imdb.com/gallery/ss/0266543/FNC-131.jpg http://www.lordoftherings.net/legend/gallery/

8. Overview of Artificial Life / Behavioral Animation • Artificial life modeling and the computer graphic modeling hierarchy Demetri Terzopoulos, Artificial life for computer graphics, Communications of the ACM, v.42 n.8, p.32-42

9. Overview of Artificial Life / Behavioral Animation • Combine geometric models and kinematic models to simplify key-framing • Mid 1980s : Dynamic Simulation • particles • rigid bodies • deformable solids • fluids

10. Overview of Artificial Life / Behavioral Animation • Biomechanical Modeling • Biological tissue • Internal muscles actuators • Behavior Modeling • Self-animating characters/ objects that react to environment stimuli • Cognitive Modeling • Artificial intelligence • Knowledge representation • Reasoning • Planning

11. Examples of Behavioral Animation in the Movies Demetri Terzopoulos, Artificial life for computer graphics, Communications of the ACM, v.42 n.8, p.32-42

12. Examples of Behavioral Animation in the Video Games • Creatures (http://www.creaturs.co.uk) • The Sims(http:// thesims.ea.com) • Demosaurus Rex (an experimental interactive game environment) • etc

13. Artificial Life in Computer Graphics Artificial plants Conference proceedings cover image, SIGGRAPH '94 [R. Mech, P. Prusinkiewicz, B. Wyvill, and A. Glassner]

14. Artificial Evolution Hopping [Sims] • Sims [Sims 91,94] • simulate evolution from genotypes to phenotypes • occasional mutations

15. Steering Behaviors Action Selection: strategy, goals, planning • Wandering • Seek / Flee • Pursuit / Evade • Path Following • Cohesion • Alignment • Separation • Leader Following Steering: path determination Locomotion: animation, articulation

16. Artificial Animals • Terzopoulos [Tu and Terzopoulos 94] • Basic behaviors • Simulation of fish movements using a simple muscle system

17. Artificial Humans HUMANOID-2 • Thalmann [Thalmann 98] • Facial animation • Simulation of sensory perception • Interaction / communication between real humans and virtual ones

18. Design Motion Design Behavior Behavioral Animation • Goal : Model animated characters / objects that interact autonomously within a virtual world • Behavioral Animation shifts the traditional role of the animator • We still probably want external control and direction on an autonomous agent • Behavioral animation can be used as a vehicle for setting up simulations of real world events

19. Behavioral Animation v.s. Key-framing • Key-framing • Animator : Specifies the first and last frames of the animation sequence • Computer : Generates the in-between frames • Behavioral Animation • Animator : Determines a set of rules for the behavior of a character • Computer : Animates the scene based on the rules specified

20. Components of a Behavior Animation System • Framework of a generic Behavioral Animation system [Millar et al. 99]

21. [Hanna et al. 2001]

22. Perception Techniques • The perception techniques are the means by which the animated character perceives its environment • Can be grouped roughly in 3 classifications : • Zonal approach • Sensory approach • Synthetic vision approach

23. Zonal Approach • Objects in the zone are sensed by the character • Usually, perception zones are circular or spherical but it can be otherwise Perception Zone

24. Sensory Approach • There are areas where perception of an object is vague and where it is precise • There might be many perception regions depending on the senses involved (e.g. smell, touch, sight, etc.) • Each object in the environment has a transmissive area • Perception occurs when character’s sensory area intersects with the object’s transmissive area

25. Behavioral Model • Decision making model of the system • Behavior can be • reactive • intelligent response • Form of response is various : • movement vector • change in internal attributes • etc. • Different approaches used to implement the behavioral techniques • Rules • Network • Artificial intelligence • Mathematical

26. Rule-Based Approach • Involves giving each animated character a set of rules defining how they react to their environment • (+) Can produce reasonable behaviors in a dynamic environment • (+) Relatively easy to modify the rules to produce different behaviors • (-) Reduce the chance of modeling behavior which cannot be easily predicted • (-) Number of rules can grow very large in complex environments • (-) Specific to a particular environment

27. Artificial Intelligence Approach • Artificial Intelligence Approach • Uses AI techniques such as reasoning engines and neural networks • (+) Provides the ability to generate behaviors which have not been explicitly programmed • (-) Requires a level of understanding by the animator of the techniques being used

28. Mathematical Approach • The behavior of the character is defined in mathematical terms • Ex : Model of object’s nature & Model of observer’s liking • Nature = (N1, N2) • Liking = (2, -3.7) • Model of function “Attractiveness” • Attractiveness = N . L = 2N1 – 3.7N2 • (+) Provides a means of specifying behavioral responses in a precise manner • (-) Not very intuitive for animators

29. Methods of Motion Generation • Traditional Principles (Keyframing) • Performance Capture (Motion Capture) • Modeling/Simulation (Physics, Behaviors) • Automatic Discovery (High-Level Control)

30. Keyframing (I) • Specify the key positions for the objects to be animated. • Interpolate to determines the position of in-between frames.

31. Keyframing (II) • Advantages • Relatively easy to use • Providing low-level control • Problems • Tedious and slow • Requiring the animator to understand the intimate details about the animated objects and the creativity to express their behavior in key-frames

32. Motion Interpolation • Interpolate using mathematical functions: • Linear • Hermite • Bezier … and many others (see Richard Parent’s online book) • Forward & inverse kinematics for articulation • Specifying & representing deformation

33. Basic Terminologies • Kinematics: study of motion independent of underlying forces • Degrees of freedom (DoF): the number of independent position variables needed to specify motions • State Vector: vector space of all possible configurations of an articulated figure. In general, the dimensions of state vector is equal to the DoF of the articulated figure.

34. Forward vs. Inverse Kinematics • Forward kinematics: motion of all joints is explicitly specified • Inverse kinematics: given the position of the end effector, find the position and orientation of all joints in a hierarchy of linkages; also called “goal-directed motion”.

35. Forward Kinematics • As DoF increases, there are more transformation to control and thus become more complicated to control the motion. • Motion capture can simplify the process for well-defined motions and pre-determined tasks.

36. Inverse Kinematics • As DoF increases, the solution to the problem may become undefined and the system is said to be redundant. By adding more constraints reduces the dimensions of the solution. • It’s simple to use, when it works. But, it gives less control. • Some common problems: • Existence of solutions • Multiple solutions • Methods used

37. Motion Capture (I) • Use special sensors (trackers) to record the motion of a performer • Recorded data is then used to generate motion for an animated character (figure)

38. Motion Capture (II) • Advantages • Ease of generating realistic motions • Problems • Not easy to accurately measure motions • Difficult to “scale” or “adjust” the recorded motions to fit the size of the animated characters • Limited capturing technology & devices • Sensor noise due to magnetic/metal trackers • Restricted motion due to wires & cables • Limited working volume

39. Basic Behavioral Component Loop [ after Hanna et al. 2001] • Execute all the mental triggers for each entity • Ex. : Increase hunger level • Execute all the rules for each entity • Enter associated goals in entity’s goal queue • Ex. : Go to; Wander; Feed; Mate • Select highest priority goal from queue and execute it • Delete all expired goals • Display state of the animation

40. Coffee Break

41. Artificial Lives: Fishes and Evolving Creatures • Introduction to Artificial Lives • Artificial Fishes and Reinforcement Learned Behavior • Introduction • Physics-Based Fish Model & Locomotion • Sensory Perception • Behavioral Modeling & Animation • Intention Generator • Artificial Fish Types • Evolving Creatures and Genetic Algorithm

42. “Go Fish!”

43. Overview Artificial Fish Habits Intention Generator intention Behavior Routines … F O C U S S E R Filtered sensory data … Sensory data Sensors … Control parameters Behavior Perception Motor Controllers … Actuators(Muscles) Underlying Physical Model Motor (Action) 3D Virtual World

44. Spring-Mass Dynamic Fish Model

45. Mechanics • node i have mass mi, position xi(t) = [xi(t), yi(t), zi(t)], velocity vi(t) = dxi/dt, acceleration ai(t) = d2xi/dt2. • spring Sij connect node i to node j. • Denote springs constant as cij & natural, rest length as lij, eij(t) = ||rij|| -lij, where rij = xj(t) – xi(t) • The Lagrange equations of motion of the dynamics fish: i = 0, …, 22, where wi(t) = jNi

46. Muscles & Hydrodynamics • The artificial fish moves as a real fish does, by contracting its muscles. • Characteristic of swinging tail: contracting one side, relaxing the other side. • When the fish’s tail swings, it sets in motion a volume of water. The inertia of the displaced water produces a reaction force normal to the fish’s body and proportional to the volume of water displaced per unit time, which propels the fish forward. • With tail swinging towards positive X axis, reaction force at point niacts along the inward normal. Component resists the lateral movement, while is forward thrust. Aggregate thrust propels fish towards positive Y axis.

47. Motor Controllers • A artificial fish has three MCs, swim-MC, left-turn-MC & right-turn-MC. • The MCs prescribe muscle contractions to the mechanical model. The swim-MC controls the swimming segment muscle, while the turning MCs control the turning segment muscles. • swim-MC(speed) {r1, s1, r2, s2} • Turn-MC(angle) {r1, s1, r2, s2}

48. Pectoral Fins • The pectoral fins control pitching and yawing. • The pectoral fins work by applying reaction force to nodes in the midsection, i.e. nodes 1 ≦i≦ 12. • The pectoral fins are analogous to the wings of an airplane. Pitch, yaw, and roll control stems from changing their orientations relative to the body; i. e., the angle π/4 ≦ γ ≦ π. • A fin has an area A, surface n and the fish in velocity v relative to the water, the fin force

49. Learning How to Swim • Eu: Guiding the muscle actuator control function u(t) by discouraging large, rapid flutuation of u • Ev: Guiding good trajectory of the artificial fish (final distance from a target location, closeness to a specified speed, etc)

50. Learning Autonomously