1 / 78

Sparse Point-Sets Matching with underlying non-rigidity

ICPR August 23-26, 2004 University of Cambridge, UK. Sparse Point-Sets Matching with underlying non-rigidity. Baihua Li, Qinggiang Meng, Horst Holstein (speaker) bal@aber.ac.uk, qqm@aber.ac.uk, hoh@aber.ac.uk. University of Wales, Aberystwyth, UK. Where we are.

varian
Download Presentation

Sparse Point-Sets Matching with underlying non-rigidity

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ICPR August 23-26, 2004 University of Cambridge, UK Sparse Point-Sets Matching with underlying non-rigidity Baihua Li, Qinggiang Meng, Horst Holstein (speaker) bal@aber.ac.uk, qqm@aber.ac.uk, hoh@aber.ac.uk

  2. University of Wales, Aberystwyth, UK Where we are

  3. Robust one-to-one correspondence through binary space partition trees: An application to interpret real-world human motion capture data H Holstein and B Li (University of Wales, Aberystwyth, UK)Speaker: Horst Holstein Abstract Commercial optical motion capture (MoCap) systems successfully achieve real-time motion capture through simplified image acquisition. They employ reflective markers attached to the subject, and track only these feature points in multiple cameras. There remains, however, the problem of establishing the identity of each feature point, so that motion capture can be displayed as an animated joined-up stick figure. Feature point identification is currently a manual process. In this talk, I describe a successful approach towards automated labelling. We set up a feature-point correspondence between a known generic model and one frame of capture data of the subject in a design pose. Pose differences and scaling make this a problem of robust correspondence generation. In the talk, I will include an overview an optical motion system (the Vicon 512), and of k-d trees. A new form of these leads to an elegant approach to the robust correspondence problem.

  4. MLDs as sparse point sets Moving Light Displays – Gait perception in Psychology: • Activity recognition. [Johansson, 1975] • Friends Recognition. [Cutting et al. 1977] • Gender recognition. [Barclay et al. 1978]

  5. Research with MLDs • Computer vision recognize and classify human activities; lip reading recognition; identify individual subjects. • Biomedical studies, e.g. gait analysis • Motion analysis and synthesis, (game/animation) • Kinematics studies, e.g. sportsscience • Humanoid robot design [Hill et al, 2000]

  6. Example in animation • Facial capture and transformation

  7. Z X Marker-based Optical MoCap Vicon 512 (3D MLD)

  8. Individual Camera Views camera 1 camera 3 camera 4 camera 2

  9. 3D-MLDs data collection from Vicon 512 system

  10. Our Problem -identify 3D sparse feature points 3D Moving Light Displays (3D-MLDs) - An extension of Johansson’s 2D-MLDs [1975]

  11. Current Practice (Vicon) • Capture ONE frame of the subject in a design pose • Using a GUI, manually attach labels to screen marker points

  12. Static Pose Determination

  13. head upper arm Marker Labelling rfh lfh lsh lel

  14. Transfer to Dynamic Marker Labelling/Segment ID 3D Moving Light Displays (3D-MLDs) - An extension of Johansson’s 2D-MLDs [1975]

  15. A common situation Given marker distribution for • Transverse study (different individuals) • Longitudinal study (same individual) Currently • manually identify markers in each case

  16. Remedy ? • Manually identify one “model” • Automate transfer of marker identities from model to each new “subject”

  17. Robust Correspondence Problem: • Two point sets (model and subject) • Same cardinality • Differently scaled • Similar but non-identical poses How do the sets correspond? Can’t use rigid transformation foralignment

  18. Identification of point-sets with underlying non-rigidity

  19. Robust Correspondence Idea: • Bottom to top ordering • Left to right ordering Formalise: • Binary space partition tree • partition strategy

  20. The k-d tree • 1975 J. L. Bentley. Multidimensional binary search trees used for associative searching. Communications of the ACM, 18:509–517. Reference:

  21. The similarity k-d tree • partition (dynamically) at biggest projected gap • gives robustness • record partition cardinality • deals with translation and scaling Focus on two aspects for tree of model:

  22. A 2D walk through • Step 1 • construct the k-d tree for the manually identified model • once only

  23. Z-projection 1 3 2 4 5 6 7 X-projection

  24. Z-projection 1 1 3 2 4 5 6 7 X-projection 1

  25. Z-projection 1 1 3 2 4 2 5 6 7 X-projection 2 1

  26. Z-projection 1 1 3 2 3 4 2 5 6 7 X-projection 2 3 1

  27. Z-projection 1 1 3 2 3 4 2 4 5 6 7 X-projection 2 3 1 4

  28. Z-projection 1 1 3 2 3 4 2 4 5 5 6 7 X-projection 2 3 1 4 5

  29. Z-projection 1 1 1 3 2 3 4 2 4 5 5 6 7 6 X-projection 2 6 3 1 4 5

  30. Z-projection 1 1 1 3 2 3 4 2 4 5 5 6 7 6 7 X-projection 2 6 3 1 7 4 5

  31. {1, 2, 3, 4, 5, 6, 7} (z, 2) Z-projection 1 1 {6, 7} {1, 2, 3, 4, 5} 3 2 3 4 2 4 5 5 6 7 6 7 X-projection 2 6 3 1 7 4 5

  32. (z, 2) (z, 2) Z-projection 1 1 (x, 1) {6, 7} {1, 2, 3, 4, 5} {1, 2, 3, 4, 5} 3 2 3 4 2 4 6 7 5 5 6 7 6 7 X-projection 2 6 3 1 7 4 5

  33. (z, 2) (z, 2) Z-projection 1 1 (x, 1) (x, 1) {1, 2, 3, 4, 5} (x, 1) 3 2 3 4 2 4 6 6 7 7 2 {1, 3, 4, 5} 5 5 6 7 6 7 X-projection 2 6 3 1 7 4 5

  34. (z, 2) (z, 2) Z-projection 1 (x, 1) (x, 1) (x, 1) (x, 1) 3 2 4 6 6 7 7 2 2 {1, 3, 4, 5} (x, 3) 5 {1, 3, 5} 4 6 7 1 3 2 4 5 6 7 X-projection 2 6 3 1 7 4 5

  35. (z, 2) (z, 2) Z-projection 1 1 (x, 1) (x, 1) (x, 1) (x, 1) 3 2 3 4 2 4 6 6 7 7 2 2 (x, 3) (x, 3) 5 5 {1, 3, 5} (z, 1) 4 4 6 7 6 7 5 {1, 3} X-projection 2 6 3 1 7 4 5

  36. (z, 2) (z, 2) Z-projection 1 1 (x, 1) (x, 1) (x, 1) (x, 1) 3 2 3 4 2 4 6 6 7 7 2 2 (x, 3) (x, 3) 5 5 (z, 1) (z, 1) 4 4 6 7 6 7 5 5 {1, 3} (z, 1) X-projection 2 6 3 1 7 4 5 3 1

  37. A 2D walk through • Step 2 • Use the model k-d tree to set up the 1-1 correspondence between the model and a new subject (in a similar pose). • Same cardinality for model and subject

  38. Given a new subject similar pose to model, same marker protocol, what are the labels? Step 2

  39. What is the correspondence between the labels in model and new subject? 1 3 2 4 5 E D 6 7 F C B G A Step 2

  40. The model tree contains “navigation” rules Follow these to partition the subject of a similar pose Assign the labels from terminal partitions (z, 2) (x, 1) (x, 1) 6 7 2 (x, 3) (z, 1) 4 5 (z, 1) 3 1 Step 2

  41. (z, 2) (x, 1) (x, 1) E 6 7 2 (x, 3) D F (z, 1) 4 C B 5 (z, 1) 3 1 6 G 7 A

  42. (z, 2) (x, 1) (x, 1) E 6 7 2 (x, 3) D (z, 1) 4 C B 5 (z, 1) 6 7 3 1 2 F G A

  43. (z, 2) (x, 1) (x, 1) E 6 7 2 (x, 3) D 2 (z, 1) 4 C 5 (z, 1) 6 7 3 1 F 4 B G A

  44. (z, 2) (x, 1) (x, 1) E 6 7 2 (x, 3) D 2 (z, 1) 4 4 5 (z, 1) 6 7 3 1 F 5 C B G

  45. (z, 2) (x, 1) (x, 1) E 6 7 2 (x, 3) 2 (z, 1) 4 5 4 5 (z, 1) 6 7 3 1 1 D 3 F C B G A

  46. (z, 2) (x, 1) (x, 1) 1 E 6 7 2 (x, 3) D 3 2 F (z, 1) 4 5 C 4 B 5 (z, 1) 6 G 7 A 3 1

  47. 1 3 2 4 1 E 5 D 3 2 F 6 7 5 C 4 B 6 G 7 A Robust Correspondence

  48. 100 Points: median

  49. Similarity:

More Related