Application of Bayesian Network in Human Motion Analysis

1. Application of Bayesian Network in Human Motion Analysis Ping Liu & Shizhong Han

2. What is Bayesian Network? • A Bayesian network is a probabilistic graphical model that represents a set of random variables and their conditional dependencies via a directed acyclic graph (DAG). • It is a combination of Graph algorithm and Probabilistic theory In Bayesian Network • The node—random variables • The edge—conditional dependencies • The directed edge & undirected edge

3. A sample of Bayesian Network

4. A sample of Bayesian Network • Three nodes in this example: sprinkler, raining and grass wet. • From the edges, we can conclude: --Rain can lead to grass wet, and also open the sprinkler --Open the sprinkler can lead to grass wet • So The joint probability function is: P(G,S,R) = P(G | S,R)P(S | R)P(R) G = Grass wet, S = Sprinkler, and R = Rain.

5. A sample of Bayesian Network • Here is a question based on the sample: “What is the probability that it is raining, given the grass is wet?” • We can solve this problem by using the conditional probability formula:

6. A sample of Bayesian Network • What does this mean? • How should we use the conclusion?

7. How to apply BN into human motion analysis?

8. How to apply BN into human motion analysis?-- a more complex example

9. How to apply BN into human motion analysis?-- a more complex example • H5: Vertical pose of the outermost leg • H6: Horizontal pose of the outermost leg • V8: Vertical position of maximum curvature point of lowerbody convex hull • V9: Lower-body ellipse aspect ratio of major to minor axes • V10: Lower-body ellipse rotation on image plane • V11: Horizontal position of maximum curvature point of lowerbody convex hull

10. How to apply BN into human motion analysis?-- a more complex example • How we get the dependence probability between each node? • How we get the prior information of corresponding node? • The answer is : Use the information from the “training” data set. From the training data set, we will get the information about the prior probability of node, the conditional dependence probability between nodes.

11. How to apply BN into human motion analysis?--Estimation & Inference • We can predict the action based on the observation -- If Jack holds a baseball bat in his hand, is he playing base ball or swimming? -- If Jack nods his head, does he mean ”yes” or “no”? • We can estimate the “observation” based on the action --If Jack is playing base ball, it is probable that he holds a baseball bat in his hand -- If Jack agrees with my opinion, it is not likely that he shakes his head

12. Arm Pose Estimation • V7: Horizontal position of maximum curvature point of upper-body convex hull. • H3: Vertical pos of the outermost arm • H4: Horizontal pose of the outermost arm • V4: Vertical position of maximum curvature point of upper-body convex hull. • V5:Torso ellipse aspect ratio of major to minor axes • V6: Torso ellipse rotation on image plan

13. Observation for the arm pose estimation. Individual upper body is represented by an ellipse A and B in (a) and the corresponding convex hull in (b). The maximum curvature points C and D in (b) are detected as candidate hand positions

14. Our goal is to estimate the posterior probability probability P(H3:4|V4:7): • P(H3:4|V4:7)=P(V4:7|H3:4)P(H3:4) • The hidden node's states are defined bellow: • H3={'low', 'mid-low', 'mid-high', 'high'} • H4={'withdrawn','intermediate','stretching'}

15. Leg Pose Estimation H5:Vertical pos of the outermost leg H6:Horizontal pose of the outermost leg V8:Vertical position of maximum curvature point of lower-body convex hull. V9:Lower-body ellipse aspect ratio of major to minor axes V10:Lower-body ellipse rotation on image plan • V11: Horizontal position of maximum curvature point of lower-body convex hull.

16. Observation for the leg pose estimation. Individual lower body is represented by an ellipse A and B in (a) and the corresponding convex hull in (b). The maximum curvature points C and D in (b) are detected as candidate leg positions

17. Our goal is to estimate the posterior probability P(H5:6|V8:11): P(H5:6|V8:11)=P(V8:11|H5:6)P(H5:6) The hidden node's states are defined bellow: H5={'low', 'mid-low', 'high'} H6={'withdrawn','intermediate','stretching'}

18. Bayesian network for pose estimation of an interacting person. The BN is composed of 6 hidden nodes H1:6 and 11 visible nodes V1:11 . H1(the torso pose)= {‘front-view’, ‘left-view’, ‘right-view’, ‘rear-view’}. V3:the median width of the torso in an image

19. Our goal is to estimate the posterior probabilities P(H1|H2:6,V3): • P(H1|H2:6,V3 )=P(H2:6 ,V3|H1)P(H1 )

20. Reference Recognition of two-person interactions using a hierarchical Bayesian network --Park, S. and Aggarwal, JK http://en.wikipedia.org/wiki/Bayesian_network Graphical Models: Methods for Data Analysis and Mining -- Borgelt, Christian; Kruse, Rudolf