1 / 36

Fuzzy Systems Lifelog management

Case Study 2 A hierarchical Bayesian network for event recognition of human actions and interactions Sangho Park, J.K. Aggarwal Multimedia Systems, vol. 10, pp. 164-179, 2004. Fuzzy Systems Lifelog management. Outline. Overview Post estimation using a hierarchical Bayesian network

raymond-booker
Download Presentation

Fuzzy Systems Lifelog management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Case Study 2A hierarchical Bayesian network for event recognitionof human actions and interactionsSangho Park, J.K. AggarwalMultimedia Systems, vol. 10, pp. 164-179, 2004 Fuzzy SystemsLifelog management

  2. Outline • Overview • Post estimation using a hierarchical Bayesian network • Recognition by DBN • Relative constraints • Experiment • Summary

  3. Overview • Recognition of human interaction • Applications: video surveillance, video-event annotation, virtual reality, human-computer interaction, and robotics • Difficulty: Ambiguity caused by body articulation, loose clothing, and mutual occlusion between body parts • Previous work • A method to segment and track multiple body parts in two-person interactions • Multilevel processing at pixel level, blob level, and object level

  4. Motivation • A methodology • To estimate body-part pose • To recognize different two-person interactions including pointing, punching, standing hand-in-hand, pushing, and hugging • System component • Bayesian network: estimate body poses • Dynamic Bayesian network: classify a sequence of body poses

  5. Head Pose Estimation • Environmental setup: lighting conditions, reflectance of light from the head • Head pose: head’s 3D rotation angles, visible part

  6. Head Pose Estimation: Example • V1: angle of the vector • V2: ratio of the two ellipses • a, B: fixed • P(V1=C|H2=B) = 0.18

  7. Arm Pose Estimation

  8. Arm Pose Estimation: Example • P(V5=B|H3=C, H4) = 0.34

  9. Leg Pose Estimation

  10. Leg Pose Estimation: Example

  11. Overall Body Pose Estimation (1)

  12. Overall Body Pose Estimation (2)

  13. Body-part Pose Recognition by DBN • DBN hidden states • Q1: set of DBNs for legs {“both legs are together on the ground”, “both legs are spread on the ground”, “and “one foot is moving in the air while the other is on the ground”} • Q2: set of DBNs for the torso {“stationary”, “moving forward”, and “at least one arm gets withdrawn”} • Q3: set of DBNs for arms {“both arms stay down”, “at least one arm stretches out”, and “at least one arm gets withdrawn”}

  14. Interaction Recognition • Whole-body pose: q1, q2, q3 • {“stand still with arms down”, “move forward with arm(s) stretched outward”, “move backward with arm(s) raised up”, “stand stationary while kicking with leg(s) raised up”, etc.} • Two-person interaction • Subject = {torso, arm, leg} • Verb = {raise, lower, stretch, withdraw, stay, move forward, move backward} • Object = {head, upper body, hand, lower body}

  15. Relative Constraints: Spatial • Examples • “standing hand-in-hand”: the torsos of the two persons be side by side and facing in the same direction • “pointing at the opposite person”: the torsos of face one another • Relative position and orientation • Gross level: proximity between two persons • Intermediate level: relative orientations of the torso poses between the two persons • Detailed level: relative configuration of individual body parts

  16. Relative Constraints: Temporal • Interval temporal logic: before, meet, overlap, start, during, and finish • Example • A pushing interaction • Event A: a person moving forward with arms stretched outward toward the second person • Event B: m of the second person

  17. Experimental Results • Human interaction (9):approaching, departing, pointing, standing hand-in-hand, shaking hands, hugging, punching, kicking, and pushing • Image • 320*240 pixels • 15 fps • 6 pairs of different people with various clothing • 56(?) sequences (6*9)

  18. Interaction Examples

  19. BN’s Belief Changes

  20. Performance of DBNs • Leave-one-out-cross-validation • Training: 5 sequences, test: 1 sequence • Accuracy: 78% • Approaching: 100 • Departing: 100 • Pointing: 67  similar interaction • Standing hand-in-hand: 83 • Shaking hands: 100 • Hugging: 50  occlusion • Punching: 67  similar interaction • Kicking: 83 • Pushing: 50  similar interaction

  21. Semantic Interpretation

  22. Summary • Contribution • A hierarchical framework for the recognition of two-person interactions • BN for managing ambiguity in human interaction • A human-friendly vocabulary for high-level event description • Stochastic graphical model • Future works • Extending the method to crowd behavior recognition • Incorporating various camera-view points • Recognizing more diverse interaction patterns

  23. Case Study 3Evolutionary Learning of Dynamic Probabilistic Models with Large Time LagsA. Tucker, X. Liu and A. Ogden-SwiftInternational Journal of Intelligent Systems,Vol. 16, no. 5, pp.621-646, 2001. Fuzzy SystemsLifelog management

  24. Outline • Introduction • Background • Methodology • Algorithm • Evaluation • Conclusions

  25. Introduction • Multivariate time-series (MTS) • A large number of interdependent variables • Large time lags between causes and effects (ex. Oil refinery processing) • Learning dynamic Bayesian networks • Not focused on learning models automatically • Focused on models with small time lags • Challenge task for large datasets with large possible time lags

  26. Background Dynamic Bayesian Networks • Bayesian networks • A set of n nodes {x1,…, xn}, representing the N variables in the domain • Each node, xi has a finite set of ri mutually exclusive states, vi1 to viri. • Each node xi with a set of parents, πi has an associated probability table P(xi|πi). • Dynamic Bayesian networks consist of BNs at differing time slices • Links over different time lags (non-contemporaneous links) and within the same time lag (contemporaneous links)

  27. Background Learning Bayesian Network Structures • K2/K3 algorithms • Use a greedy search which begins with an empty structure with no links • Explores the effect of adding each of the possible links to the current structure • K2 (a log likelihood metric) / K3 (a description length metric) • Branch and Bound technique • Perform an efficient exhaustive search by stopping any further exploration along a search path based on a bound • Evolutionary methods • Larranaga et al. used a genetic algorithm with a repair operator to remove cycles • Wong et al. used evolutionary programming with freeze, defrost and a knowledge guided mutation (KGM) • Sahami used the mutual information • Missing data management: Structural EM algorithm with Dempster’s expectation maximization algorithm

  28. Background Evolutionary Learning Bayesian Network Structures • Learning BNs involves scoring candidate network structures • Log likelihood • Description length metric of a network structure • Description length metric of encoding the dataset given that model • n: number of nodes • ri: possible instantiations of the node • qi: possible instantiations of the parent nodes • Fij = ∑Fijk • Fijk: frequency of occurrences in the dataset that the node xi takes on the value vikand the parent nodes πi take on the instantiation wij

  29. Methodology Representation • Assume that a dynamic network contains no contemporaneous links • n = N (# of variables at a single time slice) + Q (# of variables at previous time slice) • A list of triples represents a possible networks (a,b,l) • a: the parent variable • b: the child variable • l: the time lag

  30. Methodology Useful Heuristics • No contemporaneous links • Finding a good network structure  finding a group of simple tree structures • LagMutation: Each mutation is based on a uniform distribution with mean equal to the present lag • Autoregression links (a,a,1)

  31. Methodology Seeded GA for Search • Seed the entire first population with links found from the single link analysis • Using an approximate method to find a good list of single links rather than scoring the entire set • Exploiting this knowledge in the first population by seeding it entirely with a random selection of good links • EP method is particularly efficient at finding a good selection of links with good correlation • An individual represents a single triples • Self-adapting parameters (SAP)

  32. Algorithm

  33. Evaluation: Efficiency (1) • Adapting static BN search algorithms for DBN search • K2/K3 • The genetic algorithm • The evolutionary program • Knowledge guided mutation (KGM)

  34. Evaluation: Efficiency (2)

  35. Evaluation: Structural Comparison

  36. Conclusion • Problem: Learning dynamic probabilistic models with large time lags • Proposed method: EP-Seeded GA • Future works: Discretisation & parameterisation • Brainstorming • Mutually not-exclusive states  Fuzzy BN • Hybrid of the GA and K2/K3

More Related