Pose Estimation

1. Pose Estimation 2010. 3. 16. TUE. Kim Kyungkoo Active Grasp

2. Contents • Introduction • Pose Estimation • Object modeling with features • Real-time pose estimation • Demo • Future works

3. Introduction • Importance of object recognition and pose estimation

4. Pose Estimation • Problem Definition • Robot knows • The target object to grasp • The corresponded 3D model • The grasp point on a 3D model • BUT! Do not know • The grasp point in real-environment Orientation matching between an object and a 3D model is needed

5. Pose Estimation • System overview • Object modeling • Automatic pose estimation Tracking Reconstruction Stereo Camera Live video Partial model Model features Transformation Feature matching Pose estimation Stereo Camera Live video 3D model of an object

6. Object Modeling with features • Object Modeling process Merging Accumulated Image Set Captured Image Set SURF Feature Tracking 2D image Disparity 2D image Disparity Image Bi-layer Segmentation 3D depth Image Homogeneous Matrix Calculation Object Segmentation 3D depth Image Merged Foreground Depth image Depth Image Reconstruction Object depth image Merged Image Set Merged Object Depth image 2D image 3D depth Image

7. Object Modeling with features • Object feature list creation during modeling process • Features • Using SURF algorithm to extract features • Each feature consist of a 3D coordinate and a descriptor • Storing features extracted from object region of each frame • As the system extracts features from each image, it accumulates the features with a previous feature list • It stores all features for the first image in image stream Transformed 3D Feature list A 3D Feature SURF Feature Match YES NO Matched? Add feature descriptor into same ID Create new ID for corresponding points Updated 3D Feature list

8. Feature list creation on an object • Example of feature list

9. Real Time Pose Estimation • Feature matching between feature list of an object and features of current image • Using SURF feature extraction and matching algorithm • Each feature consist of a 3D coordinate and a descriptor • Acquisition of 3D corresponding points • Transformation • The 3D model of an object is transformed to fit a current image using 3D corresponding points • Method?

10. Pose Estimation of current view • Transformation of the 3D model for pose estimation • Using threecorresponding points • Calculate the best transformation matrix with three corresponding points using RANSAC algorithm

11. Pose Estimation of current view • Transformation of the 3D model for pose estimation H

12. Pose Estimation of current view • Transformation of the 3D model for pose estimation T1 T2 3쌍의 corresponding points중 random하게 한 쌍을 선택하여 선택된 각 점을 3차원 공간상 0,0,0으로 이동

13. Pose Estimation of current view • Transformation of the 3D model for pose estimation R1 남은 2쌍의 corresponding points 중 한 쌍을 선택하여 그 점이 같은 축 위에 존재 하도록 회전

14. Pose Estimation of current view • Transformation of the 3D model for pose estimation 같은 축 위로 회전된 corresponding point를 기준으로 scaling

15. Pose Estimation of current view • Transformation of the 3D model for pose estimation 마지막 남은 corresponding point를 다른 쪽에 맞도록 회전

16. Pose Estimation of current view • Transformation of the 3D model for pose estimation • Choose three corresponding points randomly • Calculate a transformation matrix • Transform all the corresponding point of model using the transformation matrix • Sum the distance between each corresponding point • Repeat 1st to 4th process • Select the transformation matrix which contains minimum distance summation value • Transform all the point of an object model using the inverse matrix of the selected transformation matrix in 6th process

17. Demo • Modeling process

18. Demo • Pose estimation

19. Future Works • Accuracy • Transformation • Feature list