A Multiple Camera with Real-Time Volume Reconstruction for Articulated Skeleton Pose Tracking

1. A Multiple Camera with Real-Time Volume Reconstruction for Articulated Skeleton Pose Tracking 指導教授：王聖智 教授 學生：謝佳峻 Zheng Zhang, Hock Soon Seah1 CheeKwangQuah,AlexOng, and Khalid Jabbar K.-T. Lee et al. (Eds.): MMM 2011, Part I, LNCS 6523, pp. 182–192, 2011.Springer-VerlagBerlin Heidelberg 2011

2. Outline • Introduction • Multi-camera System • Volume Reconstruction • Skeleton Pose Estimation • Results • Conclusion

3. Outline • Introduction • Multi-camera System • Volume Reconstruction • Skeleton Pose Estimation • Results • Conclusion

4. Introduction • Markerless don’t need markers or special suits. • Multi-view deal better with occlusion and appearance ambiguity problems. 剪出主要物件 還原個體輪廓形狀 偵測動作與行為 建立場景資訊

5. Outline • Introduction • Multi-camera System • Volume Reconstruction • Skeleton Pose Estimation • Results • Conclusion

6. Multi-camera System • System Setup 1.Cameras work synchronously for acquiring multiple image in time. 2. The frame rate of image acquisition should be at least 15 fps. 3. The bandwidth is sufficient for supporting the transfer of multi-video streams. 4.The acquisition room ought to be large. Only one PC !!

7. Outline • Introduction • Multi-camera System • Volume Reconstruction • Skeleton Pose Estimation • Results • Conclusion

8. Volume Reconstruction • Background Subtraction Background modeling constructs a reference image representing the background. Threshold selection determines appropriate threshold values used in the subtraction operation. Subtraction operation or pixel classicationclassies the type of a given pixel, i.e., the pixel is the part of background, or it is a moving object. : 目前影像 : 參考背景 : 為一門檻值

9. Volume Reconstruction • Shape-from-Silhouette and Visual Hulls 1.Each multi-view silhouette contour is firstly obtained. 2.Silhouette polygons are back-projected into their corresponding camera positions. 3. Volume reconstruction method 4.Testing each voxel’s 6-connected neighbors.

10. voxel texture (a) (b) (c) Illustration of volume reconstruction rendered in point clouds (a), voxels without texturing (b) and voxels with texturing (c)

11. Outline • Introduction • Multi-camera System • Volume Reconstruction • Skeleton Pose Estimation • Results • Conclusion

12. Skeleton Pose Estimation • The body model • Barrel model • 10 body segments 29 DOFs (1) (2)

13. Skeleton Pose Estimation • PSO(particle swarm optimization) is the position of the i-th particle at k-th iteration . is the velocity of the i-th particle at k-th iteration . represents a vector of random numbers uniformly distributed in is the history best position found by the i-th particle. is the global best position found by its neighborhood so far. is a constriction coefficient .

14. Outline • Introduction • Multi-camera System • Volume Reconstruction • Skeleton Pose Estimation • Results • Conclusion

15. Results

16. Results

17. Outline • Introduction • Multi-camera System • Volume Reconstruction • Skeleton Pose Estimation • Results • Conclusion

18. Conclusion 1.Real-time volume sequences are reconstructed for articulated pose recovery. 2.Relies on single PC. 3.Different body segments are not allowed to intersect in the space . 4.Different model points should avoid taking the same closest feature point. Future work will concentrate on enhancing the tracking robustness and accurateness.

19. References • Horprasert, T., Harwood, D., Davis, L.S.: A statistical approach for real-time robust background subtraction and shadow detection. • Laurentini, A.: The visual hull concept for silhouette-based image understanding . • Matusik, W., Buehler, C., McMillan, L.: Polyhedral visual hulls for real-time rendering. • Clerc, M., Kennedy, J.: The particle swarm - explosion, stability, and convergence in a multidimensional complex space. • http://www.csie.ntu.edu.tw/~cyy/courses/vfx/05spring/lectures/scribe/12scribe.pdf