Human Motion Modeling and Synthesis

1. Human Motion Modeling and Synthesis Payam Saisan Alessandro Bissacco UCLA Vision Lab

2. Problem Statement Modeling of human gaits such as walking and running. Formulate a simple model that can capture the dynamics of a human gait with its subtleties. Great deal of information is embedded in the dynamics. Gender Age Mood … Personality?

3. Recognition Space of models Data Model Synthesis t Why model human gaits? Bissacco et al, CVPR 2001

4. Some common techniques Kinematics based Methods : Tools from robotics: inverse kinematics. Physics Based Methods : Build the physical model of the kinematics chain, Generate key frames, calculate torques at joints. Data driven/statistical methods:Sampling from actual data.

5. Trajectories (Time series) Gait Representation Joint Angles 3D Feature Points

6. x(t) A,B,C v(t) y(t) The Model Discrete-time, continuous-state linear dynamical system, Gait information embedded in system parameters. Input to the system Output : Joint angles or marker positions, etc…

7. Linear Dynamical Systems • Simple • Can capture different gait dynamics • Choice of the input Example Data Simplest Autoregressive model Synthesis Model y(t+1)=Ay(t) Y(0)=yo

8. Outline of our approach • Modeling/Representation: dynamical system • Identification/Learning: obtain A, B, C • Gaussian input: simple closed form solutions. • Non-Gaussian input: may be posed as Dynamic ICA • Synthesis: generate inputs from the identified distribution • Gaussian inputs • Non-Gaussian inputs

9. System Identification Given the dynamical system model of the form Find That minimize some cost function.

10. Identification: Gaussian Input Optimality criterion : Problem has been solved in the framework of subspace system ID Also : DeMoor in ECCV 02 tutorial n4sid : Implementation in MATLAB, System ID Toolbox

11. Identification: non-Gaussian input with independent components • Gaussians model second order statistics, ignore higher order structure. • Need to re-pose the identification problem to arrive at non-Gaussian inputs. • Problem is hard : No pretty solution as in the Gaussian case.

12. Connection to Independent Component Analysis • ICA : factor y, into another random vector v and linear mixing matrix C. Find C and Pi such that vi assumed to be independent and non-Gaussian in the estimation process.

13. Formulating Dynamic Independent Component Analysis Goal : Estimate system parameters to yield input v(t) with independent components :

14. Formulating Dynamic ICA Lets re-write the output of the system y at time t :

15. Formulating Dynamic ICA

16. Standard ICA Dynamic ICA Estimate C subject to Block Teoplitz structure

17. Two stage sub-optimal solution • First Estimate A, sub-optimally • Use standard ICA to estimate B and v(t) Stage 1: Estimate A C=I, System reduced to AR model

18. ICA on Residuals Stage 2 : Estimate B,v(t) Compute residuals from data, given A [B,v]=infomaxICA(e);

19. Synthesize with non-Gaussian input model V’(t)=SamplefromPDF(pdf(v));

20. Extracting skeletal model from images 2D image space Joint angle space y(t) Our Observations (Gait Data)

21. Extracting 3D feature points

22. Walking Learn A,B,C,q Gait Data Model A,B,C,q x(0)=xo Synthesis

23. Joint angle data from motion capture Data Synthesis

24. Running Learn A,B,C,q Gait Data Model A,B,C,q x(0)=xo Synthesis

25. Limping Man Synthesis Data

26. Walking (Gaussian vs non-Gaussian) Gaussian Input Original (Data) Non-Gaussian Input

27. Walking (Gaussian vs non-Gaussian) Gaussian Synthesis Gait Data Non-Gaussian Synthesis

28. Remarks • Proposed a method for modeling human gaits • Model : Linear Dynamical Systems. • Learning: Identification of both second order (Gaussian) and higher order statistical input models • Synthesis: We can generate human motion with subtleties of motion using a simple linear model • Future work: optimal solution of identification • More extensive experiments • Extension of Dynamic ICA method to Dynamic textures, moving scenes with stationary properties.

29. Human Motion Modeling and Synthesis Payam Saisan Alessandro Bissacco UCLA Vision Lab

30. Kinematics based Methods Barrowing tools from Robotics (inverse kinematics) Specify 3D trajectories of the end points, and derive trajectories of joints along the chains by solving the inverse Kinematics problem. Key-framing necessary. For long chains need to specify additional constrains, since inverse Kinematics has more than one solution.

31. Body Model : Articulated rigid body, limbs (sticks) connected at joints. Kinematic chains Physics Based Methods Newton-Euler equations at the joints lead to a system of second order nonlinear differential equations for motion. For complex models, numerical solutions only..

32. Non-Gaussian data Gaussian envelope Simulating Physics Based Methods Simulating Motion - Key framing and tracking in between frames Specify motion trajectories for joint angles and positions, have a control system produce the necessary torques to track the specified trajectories. (Petros Faloutsos, 2000)