1 / 34

Performance Evaluation of Vision-based Real-time Motion Capture

Performance Evaluation of Vision-based Real-time Motion Capture. Naoto Date, Hiromasa Yoshimoto, Daisaku Arita, Satoshi Yonemoto, Rin-ichiro Taniguchi Kyushu University, Japan. Background of Research. Motion Capture System Interaction of human and machine in a virtual space

penn
Download Presentation

Performance Evaluation of Vision-based Real-time Motion Capture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Performance Evaluation of Vision-based Real-time Motion Capture Naoto Date, Hiromasa Yoshimoto, Daisaku Arita, Satoshi Yonemoto, Rin-ichiro Taniguchi Kyushu University, Japan

  2. Background of Research • Motion Capture System • Interaction of human and machine in a virtual space • Remote control of humanoid robots • Creating character actions in 3D animations or video games • Sensor-based Motion Capture System • Using Special Sensors (Magnetic type, Infrared type etc.) • User’s action is restricted by attachment of sensors • Vision-based Motion Capture System • No sensor attachments • Multiple cameras and PC cluster

  3. Key Issue • Available features acquired by vision process is limited. • Head, faces and feet can be detected robustly. • How to estimate human postures from the limited visual features • Three kinds of estimation algorithms • Comparative study of them

  4. CG model 人物 2 System Overview camera PC PC PC camera PC camera PC camera camera PC

  5. CG model 人物 2 System Overview Using 10 cameras for robust motion capture camera PC camera PC camera PC camera camera PC

  6. CG model 人物 2 System Overview 1 top-view camera on the ceiling camera PC camera camera PC camera camera PC

  7. CG model 人物 2 System Overview 9 side-view cameras around the user camera camera camera camera camera PC

  8. CG model 人物 2 System Overview Using PC cluster for real-time feature PC PC PC PC camera PC camera camera PC

  9. CG model 人物 2 System Overview First, take images with each camera camera PC camera PC camera PC camera camera PC

  10. CG model 人物 2 System Overview Extract image-features on the first stage PCs camera PC PC camera PC camera PC camera camera PC

  11. CG model 人物 2 System Overview Reconstruct human CG model by feature parameters in each image PC PC PC PC camera camera PC

  12. CG model 人物 2 System Overview Synchronous IEEE1394 cameras: 15fps camera camera PC camera PC camera camera PC

  13. CG model 人物 2 System Overview CPU : PentiumⅢ700MHz ｘ 2 OS : Linux Network ：Gigabit LAN Myrinet PC PC PC camera PC camera camera PC

  14. Top-view camera process • Background subtraction • Opening operation • Inertia principal axis • Detect body directionand transfer it

  15. Top-view camera process • Background subtraction • Opening operation • Inertia principal axis • Detect body directionand transfer it

  16. Top-view camera process • Background subtraction • Opening operation • Inertia principal axis • Detect body directionand transfer it

  17. Top-view camera process • Background subtraction • Opening operation • Feature extraction • Inertia principal axis • Body direction

  18. Side-view camera process • Background subtraction • Calculate centroids of skin-color blobs

  19. Side-view camera process • Background subtraction • Calculate centroids of skin-color blobs

  20. Side-view camera process • Background subtraction • Calculate centroids of skin-color blobs

  21. Estimate 3D position of skin-color blob • From all the combination of cameras and blob centroids, we select all possible pairs of lines of sight. Then we calculate an intersection point of each line pair. Unless the distance of the two lines is smaller than a threshold, we decide there is no intersection point.

  22. Estimate 3D position of skin-color blob • The calculated points are clustered according to distances from the feature points (head, hands, feet) of the previous frame. • Select points where feature points are dense as the 3D positions of the true feature points.

  23. head V V: V is the vector which intersects perpendicularly with a body axis and with a body direction. L1 right shoulder L2 torso Estimate 3D position of torso ・A method based on simple body model Center point

  24. Performance evaluation of right hand position estimation

  25. Estimate 3D positions of elbows and knees • 3 estimation methods • Inverse Kinematics (IK) • Search by Reverse Projection (SRP) • Estimation with Physical Restrictions (EPR)

  26. Estimate 3D positions of elbows and knees • IK • f3 assumed to be a constant

  27. Estimate 3D positions of elbows and knees • SRP

  28. Estimate 3D positions of elbows and knees • EPR • An arm is assumed to be the connected two spring model. • The both ends of a spring are fixed to the position of the shoulder, and the position of a hand. • The position of an elbow is converged to the position where a spring becomes natural length. (the natural length of springs is the length of the bottom arm and the upper arm which acquired beforehand.)

  29. Accuracy of estimating right elbow position

  30. Accuracy of posture parameters

  31. Visual comparison of 3 methods

  32. Computation time required in each algorithm Top-view camera processing : 50ms Side-view camera processing : 26ms 3D blob calculation : 2ms IK calculation : 9ms SRP calculation : 34ms EPR calculation : 22ms

  33. Online demo movie (EPR)

  34. Conclusions • We have constructed a Vision-based Real-time Motion Capture System and evaluated its performance • Future works • Improvement of posture estimation algorithm • Construction of various applications • Man and machine interaction in a virtual space • Humanoid robot remote control system

More Related