Nicolas Pronost

1. Definition and realisation of modeling methods and motion computation algorithms for virtual humans Nicolas Pronost Thursday 7 December 2006

2. Where are they found ? • Animation • Video games • Movies • Motion sciences • Biomechanics • Medicine, Health • Sports • Robotics • Bipedal robot • Anthropology • Modern human • Fossilised hominids • Simulation • Motion analysis • Motion synthesis

3. Fondamental aspects • Modeling the human • Symbolical and controlable representation • Kinematical and physical properties • Modeling the motion • Manipulable mathematical representation • Dependant of the editing methods • Editing methods • Based on motions laws • Looking for generiness

4. Outline • Related works • Overview and motivations • Analysis / synthesis loop • Kinematical adaptation • Evaluation of the dynamics • Forward dynamics synthesis • Conclusion and future work

5. Outline • Related works • Overview and motivations • Analysis / synthesis loop • Kinematical adaptation • Evaluation of the dynamics • Forward dynamics synthesis • Conclusion and future works

6. vt1 l_hip vl2 r_hip l_knee vl3 r_knee l_ankle vt4 r_ankle vt5 l_subtalar r_subtalar r_clav l_mid_foot vt6 r_mid_foot l_toe r_toe head_top r_hand_center l_hand_center Modeling a virtual human • Simplification through a hierarchical representation of rigid bodies [H-Anim 06] • Mechanical joints are perfect and the number of limbs is limited • Consensus between the anatomical reality and motion control • Manipulation of the rotational degrees of freedom

7. Biomechanical modeling • The physical properties describe the movement capacity of the limbs • At least the masses and inertias • These data are avalaible from anthropometrical tables and computable from regression laws [Vaughan et al. 99] • Cadaverical data [Dempster 55, Winter 90] • Gamma radiography [Zatsiorsky 90] • Various definitions of limbs [Chandler et al. 75, De Leva 96] • Additionnal data • Articular limits, muscular activations [Liu et al. 05], articular elasticities

8. Modeling the motion • Motion is a sequence of postures • Marey’s work [Marey 1894] on motion decomposition • Kinematical model • Positions, velocities and acceleration • Description of the possibility of motions • Static and quasi-static (kinetical) models • Usefull for slow motions • Balance of internal and external forces • Dynamical model • Use of motor forces at joints

9. Motion editing methods • Methods of kinematics and inverse kinematics • Representation by splines [Zeltzer 82, Bruderlin and Calvert 93] • Local linearisation and secondary tasks [Boulic and Thalmann 92] [Tolani and Badler 96] θ1 θ2 X

10. Motion editing methods • Kinematics and control of the center of mass • Important on quasi-static positions [Phillips 91] • Projection of the center of mass on the sustentation polygon [Boulic et al. 94] • Resolution using inverse kinematics with a priority formulation [Boulic et al. 97] • Conservation of the Zero Moment Point • Point of null result of linear momentum of the limbs [Tak et al. 00] • Dynamics filtering of motions [Yamane and Nakamura 03]

11. Motion editing methods • Methods of dynamics and inverse dynamics • Animation engine using a system of secondary order differential equations [Hodgins 98] • Newton’s laws of motion and fondamental physical laws • Virtual works and Lagrangian formalism [Rémion 00]

12. Manipulation of real movements • Real movements intrinsically have all of the information of the motion • Correct perception of the realism from few positions of caracteristical joints [Johansson 73] • Good realism of animations • Low generiness • Usable database for generating new motions

13. Manipulation of real movements • Adaptation to new characters: retargeting

14. Manipulation of real movements • Adaptation to new characters: retargeting • With different morphologies [Gleicher 98] thanks to spacetime constraints • Use of intermediate skeletons [Monzani et al. 00, Ménardais 03] • Decomposition of articular trajectories into hierarchical splines [Lee and Shin 99] • Take account of muscular forces [Komura et al. 00]

15. Manipulation of real movements • Modification of the motion • Frequential description of the articular trajectories and deformation function [Bruderlin and Williams 95] • displacement map, conservation of the high frequencies • Description by key postures and interpolation on the parameters [Witkin and Popovic 95] • Interpolation of the deformation function (scale and translation)

16. Manipulation of real movements • Combining motions • The motion graphs [Kovar et al. 02] • node = key posture • arc = possible transition • Motion blending • Synchronisation by dynamic time warping [Bruderlin and Williams 95] • Blending by linear combinaison or weighted sum [Guo and Robergé 96, Park et al. 02, Rose et al. 98, Ménardais et al. 04]

17. Manipulation of real movements • Motion database • Efficient interpolation on simple motions [Wiley et Hahn 97] • Behavioral or frequential decomposition • Fourier space [Unuma et al. 95] • Radial basis function [Rose et al. 96] • Hidden Markov chains [Brand and Hertzmann 00] • Static models using PCA [Bowden 00] • Physical simulations [Zordan et al. 05, Arikan et al. 05, Tang et al. 06]

18. Bipedal locomotion • The locomotion is a cyclic movement • Decomposable into phases [Marrey 1894, Plat and Veil 83] • A step is a half cycle • walk = sequence of single and double support phases • run = sequence of single support and flying phases • Specific modeling of locomotion • With cyclic state machines [Multon 98] • With deformed hypertorus from PCA [Martineau 06] • Large number of biomechanical data • Articular trajectories [Alexander 84, Patla 91] • Support phases [Girard 87]

19. Summary • The analysis and the synthesis of virtual human motions [Gibet 02] answer to very differents constraints according to the application • Compromise interactivity / realism / generiness

20. Outline • Related works • Overview and motivations • Analysis / synthesis loop • Kinematical adaptation • Evaluation of the dynamics • Forward dynamics synthesis • Conclusion and future works

21. Overview • Motivation : to study and to realise a process of modeling tools and motion computation • Application to locomotion Database of motions Adaptation algorithm adapted motion synthetised motion Synthesis by forward dynamics Analysis of the dynamics resulting forces

22. Outline • Related works • Overview and motivations • Analysis / synthesis loop • Kinematical adaptation • Evaluation of the dynamics • Forward dynamics synthesis • Conclusion and future works

23. Database of motions Database of motions Database of motions Adaptation algorithm Anatomical data Scale Speed profiles adapted motion synthetised motion synthetised motion Interpolations - transversal - frontal - sagital Synthesis by forward dynamics Synthesis by forward dynamics Kinematical and Dimensionnal Interpolation Footprints Analysis of the dynamics Analysis of the dynamics Rest posture resulting forces resulting forces Time adjustement and Synchronisation Post-treatments Articular limits Adapted motion Kinematical adaptation

24. Modeling the human • Definition of a kinematical chain with 11 dof • Spherical joints at pelvis and hips • Pin joints at knees world 3 rotations pelvis reference frame 3 rotations 3 rotations left femur reference frame right femur reference frame 1 rotation 1 rotation right tibia reference frame left tibia reference frame

25. Why use this model ? • Application field in paleoanthropology • Study of the bipedalism of fossilised hominids • Australopithecus afarensis Lucy (A.L. 288-1) pictures – courtesy of G. Berillon

26. 2 1 1 2 4 3 3 4 Modeling the locomotion • Treatments on the motion • Homogeneous reconstruction [Ménardais 03] • Orientation of the locomotion • Identification of the cycles • Definition of the movement of the articular centers • Computed from real landmarks • Accurate positions of articular centers and virtual points

27. 1 4 2 3 Modeling the locomotion • A parametrical representation of the locomotion: the poulaine • Definition: the Cartesian displacement of the ankle in the root reference frame • Modeled by a cubic curve using 4 characteristic points of the cycle

28. Method of computation • The principle of dimensional interpolation in the database • Definition of the step size on x • Definition of the step shift on y • Definition of the rest posture on z

29. Post-treatments • Adding the temporal dimension • Use of an average profile of speed, normalised by the time cycle and distance on the ground • Representation of the profile by a polynomial function • Integrating the function, computing the curvilinear x-coordinate and the parameters of the cubic curves • Synchronisation of the left and right poulaines • By minimisation of vertical differences • By minimisation of ground sliding

30. Post-treatments • Computation of the postures by an IK solver • Proposition of secondary tasks [Nicolas et al. 04] • (C1) Maximal distance from joint limits • (C2) Minimisation of the kinematical energy of rotation • (C3) Search of the closest posture to the rest posture • Evaluation of these tasks by 3 criteria • The total Jerk, third derivate of the angles • The difference between the final and the initial posture • The internal work

31. Post-treatments • Preparation of the animation • Construction of the foot and the ankle angle • Feet lenghts from anthropometrical tables [De Leva 96] • Trajectories of ankles computed by corrections of the ground penetrations • To go to the global motion • Global displacement minimising the sliding • Upper body movement • Adapted to the morphology and synchronised with the real motion

32. Results of the adaptation • Validation of the interpolation

33. Results of the adaptation • Validation of the adaptation • Comparison between real angular trajectories and adapted trajectories of 7 subjects in the database

34. Results of the adaptation • Simulations

35. Partial summary • The method computes a plausible locomotion from biomechanical knowledge and rules [Pronost et al. 05] • Controled by limbs sizes, bones configuration, physical parameters of the limbs, joints types, footprints, articular limits and the style of motion • Applied to paleoanthropology [Pronost et al. 06] • Future work • Combining the method with a real time adaptation to the environment • To drive the extrapolation by physical properties • A global resolution of the IK to reduce discontinuity of pelvis angles • Increase the size and the diversity of the database

36. Outline • Related works • Overview and motivations • Analysis / synthesis loop • Kinematical adaptation • Evaluation of the dynamics • Forward dynamics synthesis • Conclusion and future works

37. Adapted motion Database of motions Database of motions Mechanical model Mapping Adaptation algorithm Adaptation algorithm angle-based motion dimensional model adapted motion adapted motion synthetised motion synthetised motion Synthesis by forward dynamics Synthesis by forward dynamics Analysis of the dynamics Scaled anthropometrical table Analysis of the dynamics resulting forces Support phases recognition Biomechanical model Resulting forces and torques Evaluation of the dynamics

38. Modeling the human • Creation of a biomechanical model • Description of Denavit-Hartenberg [Hartenberg and Denavit 55] • Parameters of rotation: user • Parameters of translation: auto • Gender and nature of limbs • Gender: user • Nature: auto • Using anthropometrical tables [deLeva 96] and regression laws [Vaughan et al. 99]

39. Modeling the motion • The mapping issue • An iterative method on kinematical chains from the root joint to the effectors • Using a sequencing of the articular systems • Treatment according to the number of degrees of freedom • 1dof => pin joint, minimisation of the error • 3 dof => spherical joint, an infinity of solutions • with constraints => minimisation of the future error • without constraints => minimal rotation

40. Method of computation • In order to solve the inverse dynamics issue, we have to know the external forces applied to the system, for locomotion: • the gravity • constant value for any motion • the aerodynamical forces • supposed negligible • the ground reaction forces • When are they applied ? • Support phase recognition

41. Support phase recognition • Evaluation of 4 methods of ground contacts recognition • hand-labeled, method of reference, accurate at the frequency of the motion capture system • speed, evaluation of a speed threshold for the effectors • height, evaluation of a height threshold for the effectors • configuration, particular configurations of the effectors • By four criteria • the number of failures • the average error • its S.D. • the normalised S.D. of the thresholds

42. Support phase recognition • Results with 12 x 2 (left/right) x 2 (flex/ext) = 48 contacts • Results of the thresholds estimation with heels and toes effectors • Our algorithm chooses the best method according to the configuration of the effectors and the evaluation of the criteria

43. Application of Newton’s law • Application of the FBD principle • Free Body Diagram on each segment • Study of forces and torques applied to the limbs • Application of Newton’s second law of motion • on limbs s

44. The translation form Single support, from the free foot to the support foot Double support, global resolution No support, independent resolution The rotation form Iterative resolution of the equation: Resolution of the equations

45. Validation of the resolution • Forces (in N) at left toe and torques (in N.m) at left knee of 6 adapted locomotions Forces (N) Torques (N.m) cycle % cycle % cycle %

46. support phase support phase support phase Validation of the resolution • Comparison between 3 real ground reaction forces (black plots) and analysed forces (blue plot) from characters with similar biomechanical properties GRF (N)

47. Validation of the resolution • Ground reaction forces of different styles of real locomotions • run • jump • walk

48. Influence of the retargeting • The global scale • Most used parameter • Large influence on the dynamics of the motion • Linear relation (c.c. = 0.87) between the scale and the relative values of the GRF • Experimental validation between [0.7 , 1.2] scales GRF (N) relative GRF cycle % global scale

49. Influence of the retargeting • The femur/tibia ratio • To evaluate errors on articular centers (relative length of limbs) • Experimental validation between [0.8 , 1.2] ratios • The GRF are not compensated by relative lenghts of the limbs RMS error GRF (N) cycle % femur/tibia ratio

50. Influence of the retargeting • The structure of the skeleton • Models with 33 and 21 dof (pin joints at knees, ankles and elbows) • Kinematical influence: 1.4 cm per limbs • Dynamical influence: mostly on fore-aft acceleration • Here, corresponds to 2.5 kg reduction (4.5 % of the mass) GRF (N) RMS error cycle % cycle %