DEVELOPMENT OF A COMPUTER PLATFORM FOR OBJECT 3D RECONSTRUCTION USING COMPUTER VISION TECHNIQUES

1. DEVELOPMENT OF A COMPUTER PLATFORM FOR OBJECT 3D RECONSTRUCTION USING COMPUTER VISION TECHNIQUES Teresa C. S. Azevedo João Manuel R. S. Tavares Mário A. P. Vaz

2. Contents • Introduction to Computer Vision; • Computer Platform presentation; • Experimental results; • Conclusions; • Future work. Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz

3. Computer Vision • Computer Vision is continuously trying to develop theories and methods for automatic extraction of useful information from images, as similar as possible to the complex human visual system. Introduction Platform • Some applications: • Medicine - 3D reconstruction / modelling, surgery planning; • Identification and navigation systems; • Virtual reality; • … Results Conclusions Future Work Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz

4. Goals and Methodology • Contactless techniques to recover the 3D geometry of an object are usually divided in two classes: Introduction • active techniques - require some kind of energy projection or the camera’s (or object’s) movement to obtain 3D information about the shape; Platform Results • passive techniques - only use ambient light and so, usually, the extraction of 3D information becomes more difficult. Conclusions Future Work • Our goal was to obtain 3D models of objects using an active vision technique called Structure From Motion (SFM). Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz

5. Ported to C using MATLAB Compiler toolbox Computer Platform • Integration of functions for 3D reconstruction, available from five software programs and one computational library, all open source: • OpenCV; • Peter’s Matlab Functions; • Torr’s Matlab Toolkit; • KLT; • Projective Rectification without Epipolar Geometry; • Depth Discontinuities by Pixel-to-Pixel Stereo. Introduction Platform Results Conclusions Future Work • Modular structure; • User’s graphical interface; • Computer language: C++; • Developing tool: Microsoft Visual Studio, using MFC libraries (Microsoft Foundation Classes); • Operational system: Microsoft Windows. Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz

6. Computer Platform • The functions integrated enclose several Computer Vision techniques: Introduction • feature points detection; Platform • feature points matching between two images; Results • epipolar geometry determination; Conclusions • rectification; Future Work • dense matching. • For each technique, the user can easily choose the algorithm to use, as well as conveniently define its parameters. Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz

7. available algorithms for feature points detection OpenCV KLT Feature Points detection Introduction Platform Results • Reflect the relevant discrepancies between their intensity values and those of their neighbours; Conclusions • Usually represent vertices of objects, and their detection allows posterior matching between the images of the sequences. Future Work Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz

8. available algorithms for feature points matching matching points coordinates on 2nd image fundamental matrix 1st image feature points coordinates Feature Points matching Introduction Platform Results Conclusions Future Work • Image 2D points association between sequential images, which are the projection of the same 3D object point; • A short set of matching points is enough to determine the epipolar geometry between two images (the fundamental matrix). Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz

9. Feature Points matching • Some results: Introduction Platform Results Conclusions Future Work Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz

10. algorithms for epipolar geometry determination algorithms for epipolar lines determination Epipolar Geometry determination Introduction Platform Results Conclusions Future Work • Corresponds to the geometrical structure between two stereo images and its expressed mathematically by the fundamental matrix; • Also allows the elimination of some previous wrong matches (outliers), as well as make easier the determination of new matching points (dense matching). Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz

11. Epipolar line Inlier Epipolar Geometry determination • Some results: Introduction Platform Results Conclusions Future Work Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz

12. available algorithm for rectification Rectification Introduction Platform Results • Method that changes two stereo images, in order to make them coplanar; • Performing this step makes dense matching easier to obtain; Conclusions • The quality of the results is proportional to the quality of the epipolar geometry determination. Future Work Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz

13. available algorithms for dense matching Dense matching Introduction Platform Results Conclusions • Disparity map - codifies the distance between the object and the camera(s): closer points will have maximal disparity and farther points will get minimum disparity; Future Work • A disparity map gives some perception of discontinuity in terms of depth; • One of the algorithms also returns a discontinuity map – defines the pixels who border the changing between at least two levels of disparity. Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz

14. Original images Disparity map Discontinuity map Dense matching • Some results: Introduction Platform Results Conclusions Future Work Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz

15. Conclusions Introduction • The functions, already integrated in the computer platform, give good results when applied to objects with strong characteristics; Platform Results • From the experimental results, it is possible to conclude that low quality results are strongly correlated with few (strong) feature points detection and wrong matching; Conclusions Future Work • This weakness is higher as the object shape variation is smooth. Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz

16. Future work • The next steps of this work will focus on improving the results obtained when the objects have smooth and continuous surfaces: Introduction Platform • inclusion of space carving techniques for object reconstruction; Results • the feature points to use in the 3D space object definition will be detected with the use of a reduced number of markers added on the object; Conclusions • inclusion of a camera calibration technique, as well as pose and motion estimation algorithms; Future Work • Finally, the computer platform will be used in 3D reconstruction and characterization of 3D external human shapes. Teresa Azevedo, João Manuel R. S. Tavares, Mário A. P. Vaz

17. DEVELOPMENT OF A COMPUTER PLATFORM FOR OBJECT 3D RECONSTRUCTION USING COMPUTER VISION TECHNIQUES Acknowledgments This work was partially done in the scope of the project “Segmentation, Tracking and Motion Analysis of Deformable (2D/3D) Objects using Physical Principles”, reference POSC/EEA-SRI/55386/2004, financially supported by FCT - Fundação para a Ciência e a Tecnologia in Portugal. Teresa C. S. Azevedo João Manuel R. S. Tavares Mário A. P. Vaz Thank you!