Motion Synthesis for Articulated Human Bodies

1. Motion Synthesis for Articulated Human Bodies

2. Contents • Human motion synthesis: Zoo • Articulated body mechanics • Optimization based motion synthesis • Feedback based balance control

3. Human motion components • Bone (skeleton): ~206 (motion invariant) • Joint (articulation): limitation • Tendon and ligament: elasticity / plasticity • Muscle: ~639, works in group, limitation • uni-articulate and bi-articulate (passive inefficiency) • parallel or perpendicular shape • Neuron-system: reflection speed Neuron  Muscle  Tedon  Bone

5. Human motion model (I) • Pure articulated model • Simplified skeleton (hand / foot / tibia-fibula) • Directly control force and torque on joints • Convenient for high-level control

6. Human motion model (II) • Deformable model • Skin mesh motored by skeleton or captured skin animation • Muscle shape representation

7. Human motion model (III) • Tendons and muscles model • Musculoskeletal models and simulation • Muscle-tendon model (strand) Hand can have similar DoF with coarse whole body!

8. Human motion model (IV) • Visible Human data set (realistic musculature and flesh) [Teran05]

9. Methods for motion synthesis (I) • Data-driven based synthesis • Captured motion skeleton data • Captured deformable data (face / skin) • Motion dynamics and naturalness is trivial • Limited response to environment • Blending, Editing, Style transfer; no guarantee for physics correctness • Captured data is sparse • [pose, time] space is huge • Difficult to obtain data • High dynamic motion • Outdoor environment

10. Methods for motion synthesis (II) • Pure generative-based synthesis • Physics constraint can be guaranteed • Easy to interact with dynamic environment • Hard to be natural • Naturalness is a subset of physics correctness • Dynamics simulator complexity • E.g. stable / dynamic frictions • Computational complexity and stableness • High gain means high stiffness • Controller design • Mutual controller is difficult to design and hard to generalize

11. Methods for motion synthesis (III) • Pure motion planning • Can obtain solution in constrained environment • High-level path planning • Solution has no continuous or physics guarantee • Still time consuming • High dimension of human skeleton

12. Combined methods • Mocap data + physics: better controller • Controller for hand [Nancy05] • Controller for whole body [Yin08] • Mocap data + motion planning: high level motion: • Manipulating [Katsu04] • Tangling [Edmond08] • Dynamics + motion planning (for articulated model) [Russell07]

13. Other related • Robotics [Kris06] • Primitive based motion planning use transition motion to balance natural motion and environment constraints. • Biomechanics • Provides many principles for motion control: balance, Locomotion, neuroscience. [Alexandrov05] [Christine07]

14. Contents • Human motion synthesis: Zoo • Articulated body mechanics • Optimization based motion synthesis • Feedback based balance control

15. Articulated body dynamics • Each joint i is 1 DoF • Ball-and-socket and saddle can be represented by some 1 DoF • Ball-and-socket: 3 DoF • Saddle: 2 DoF • Hinge: 1 Dof

16. A special case of multi-body dynamics Yin’s work gives an example Joint makes impulse and penalty difficult

17. Parameters for articulated body • State of root (position, rotation, linear / angular velocity) • Configuration of joints (q, , ) • Inertia matrix (for articulated body, inertia matrix is an equivalent inertia matrix: give a test force, get an acceleration  inertia) • Lecture in COMP 790-058

18. Inverse dynamics • Known: q, , • Unknown: f, τ • Simple: recursive [Featherstone] • Compute v and a; compute net force on a link, similar to f = ma (downwards) • Compute force and torque on a joint (upwards) • Root’s p, v, a • End effector’s force and torque

19. Forward dynamics • Known: q, , f, τ • Unknown: • More difficult • 3 loops [Evangelos04]

20. Limitation of basic algorithms • Many things are simplified in algorithms • Joint limitation • Unilateral constraint (ground constraint) • Friction (kinetic, static, rolling and spinning) • Non-interpenetrate constraint • Collision response • Modeling contact and constraint is a well-studied problem for rigid body simulation. • See David Baraff’s papers and note.

21. Linear complementarity problem (LCP) [Andreas] • One of the standard methods to handle contact and constraints.

22. Solution to LCP: pivot method • Basic idea: if q is positive or zero, solution is trivial: (w = q, z = 0) • Pivot q and w to make it true • where

23. Solution to LCP: iterative method • LCP can be represented by QP, so can be solved by iterative methods, like Gauss-Seidel, Newton

24. Pros and Cons • Pivot method • Convergence is guaranteed after limited steps • suffer from numerical problem, especially for large-scale and/or ill-conditioned problems • Iterative method • easier to implement and • numerically robust • convergence is proven only for a limited class of M matrix

25. LCP model for constraints • Unilateral constraint (force exists only when contacting) • Joint Limit • LCP guarantees zero virtual work for contact forces • w = Mz + q?

26. Extended LCP model for frictions • Coulomb cone • Dynamic friction Static friction • Solved by extending iterative LCP solver

27. Other possible solutions • Recent there are many new algorithms other than LCP to deal with unilateral constraints and friction • Nonconvex Rigid Bodies with Stacking [Eran03] • Extension to LCP • Staggered Projections for Frictional Contact in Multibody Systems [Danny08] • Velocity based shock propagation[Kenny09] • Implicit Contact Handling for Deformable Objects [Miguel09]

28. Impact constraints • Non-interpenetrate constraint • Collision reaction • Impulse or force modeling • See David Baraff’s papers and note.

29. Simulation loops LCP solver forward dynamics Ode solver Collision detector Contact response

30. Contents • Human motion synthesis: Zoo • Articulated body mechanics • Optimization based motion synthesis • Feedback based balance control

31. Optimization based motion synthesis [Sumit09]

32. System overview

33. Lagrange mechanics Generalized force Lagrange function • Lagrange mechanics for rigid body • Lagrange mechanics for articulated body Gravity, contact force, external force

34. Complete force modeling [Liu05]

35. Actuated and non actuated joint • Root joint: non (passive) actuated • It’s configuration (position and rotation) is actuated by joint constraints. • Other joints: (active) actuated • It’s configuration is actuated by muscle energy Muscle force

36. Passive controller • Muscle controller • Limited torque / force • Limited torque / force change rate • Contact controller • Static friction • Dynamic friction • Non-penetration

37. Optimization with passive controller • Why (minimize q variation)? • Avoid trivial solution, like sliding or break-off • Zero virtual work guarantee • Roll back

38. User controller specification • Balance controller • Climb controller • Swing controller • Multiple controller composition • controller example

39. Controller protocol • Finite State Machine • Each state has its own constraints (in objective form) • State transition happens when feasible solution can not find or contact breaks.

40. Balance controller balance takeStep relaxFoot p: target position qv: slow spline: perpendicular to ground cp: CoM balance cf: support feet position c: friction

41. Climb controller relaxFoot relaxHand allSupport moveFoot moveHand com: change CoM

42. Swing controller trySwing passiveSwing

43. Visual sensor • Reachable objects search

44. Contents • Human motion synthesis: Zoo • Articulated body mechanics • Optimization based motion synthesis • Feedback based balance control

45. Human motion control model

46. Feedback based balance control • Balance control under small perturbation [Yin03] • Balance control under large perturbation [Yin08] • Similar strategy: feed-back + feed-forward, large perturbation  step strategy

47. System description • The dynamics system does not use generalized coordinate • Instead, use full-coordinate constrained form for dynamics • Traditional style (based on Barraf 1996 paper) • Control is a hybrid of generalized-coordinate form and full-coordinate form

48. Full-coordinate constrained matrix form (I) • Constraint i between body a and b • Constraint system (each item in J is 3*3 matrix) Here f is dim-12 vector, every 3 sub-vector is force for body (a, b, c, or d), J is 9*12

49. Full-coordinate constrained matrix form (II) • For torque, similar • row number of H is number of freedom, its column number is 3 * object number. • Each term is joint i’s torque on all objects

50. Balance control under small perturbation • Preprocess: use inverse dynamics to compute force for original motion • In each step of dynamic simulation • Forward dynamics: current state • Feedback control • Feedforward control • Net control