Learning to Efficiently Detect Repeatable Interest Points in Depth Data

1. Learning to Efficiently Detect RepeatableInterest Points in Depth Data Stefan Holzer, Jamie Shotton, and PushmeetKohli Department of Computer Science, CAMP, Technische University at Munchen (TUM). Microsoft Research Cambridge Presented by RimmaShulman

2. What we are going to talk about.. • Motivation • Related work • Proposed solution * Learning Interest Point Detectors * Designing Optimal Interest Point Detectors • Results • Pros & Cons • Future idea

3. Motivation Estimating good interest points in noisy depth data

4. Jointly find camera motion and reconstruct the surface

6. Challenges: • computational time is very expensive • Interesting points are not reliable • Performing in real time

7. Related Work 3D Interest Point Extraction: * Steder, B., Grisetti, G., Burgard, W.: Robust place recognition for 3D range data based on point features. In: ICRA. (2010)

8. * Steder, B., Rusu, R.B., Konolige, K., Burgard, W.: Point feature extraction on 3D range scans taking into account object boundaries. In: ICRA. (2011)

9. Related Work Learning-based Interesting Points Extraction: * Sochman, J., Matas, J.: Learning a fast emulator of a binary decision process. 2007

10. * Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A. Blake, A.: Real-time human pose recognition in parts from a single depth image. (2011)

11. Proposedsolution Dataset Dataset Data for Training Data for Training Learning Phase Learning Phase Regression tree Regression tree Response map Response map Filter Filter Data for Testing Data for Testing

12. Learning Interest Point Detectors • Dataset Dataset Data for Training Learning Phase Regression tree Response map Filter Data for Testing

13. Learning Interest Point Detectors • Dataset Dataset Data for Training Learning Phase Regression tree Response map Filter Data for Testing Collected From Kinect fusion reconstruction system Collected From Kinect sensor

14. Learning Interest Point Detectors • Decision Tree Dataset Data for Training Learning Phase Regression tree Response map Filter Data for Testing

15. Learning Interest Point Detectors - Mean of responses - Optimal feature Dataset Data for Training Learning Phase Regression tree Response map - Threshold e – example q – index of training sequence r – index of frame in sequence x,y – location c- response value Filter Data for Testing

16. Learning Interest Point Detectors Dataset Data for Training Learning Phase Regression tree Response map Filter Data for Testing

17. Learning Interest Point Detectors • Post-processing 3D image Dataset Data for Training Learning Phase Regression tree Response map Filter Data for Testing Depth map Depth map with corresponding surface curvature map Unfiltered response After filteration

18. Result • Learning Interest Point Detector Responses • curvature –based Interest Point detector on raw Kinect depth map. • curvature –based Interest Point detector on 3D model KinectFusion system. The results of using regression forest on raw depth data and reconstructed depth data.

19. Can we design better interest points detector to train the model?

21. Designing Optimal Interest Point Detectors • Optimality Criteria – Sparseness: there should be only a small number of points in the scene. – Repeatability : the points should be detected in all views of the scene.

22. Designing Optimal Interest Point Detectors – Distinctiveness : the area around an interest point should be unique. – Efficiency: points could be estimated efficiently.

23. Designing Optimal Interest Point Detectors

24. Designing Optimal Interest Point Detectors

25. Designing Optimal Interest Point Detectors Gaussian response

26. Curvature vs. designed features

27. Designed features are better for training than original features!!

28. Comparing to previous work

29. Varying the depth of trees

30. Varying the number of trees

31. Processing Time

32. Pros & Cons • Pros : • Fast • Efficient • Simple

33. Pros & Cons • Cons: • Small scenes. no larger than 4 m • Depending on curvature information • No code available

34. Future idea • Enlarge scenes • Reduce learning time