Stereo Vision: An Introductory Approach

1. Stereo Vision: An Introductory Approach Binsan Khadka binsan_khadka@yahoo.com Digdarshan Lal Dhonju dhonjudigdarshan@hotmail.com Amol Shrestha amolshrestha@yahoo.com Department of Computer Science and Engineering Kathmandu University

2. Objective • Images are projections of 3-Dimensional world onto a 2-Dimensional plane. • We can recover information about the 3-Dimensional world from images.   • A single image does not contain sufficient information  • Ambiguity: a given feature could correspond to a large distant object or a small nearby object.   • Can be resolved using multiple views of the scene.   • One specific technique: only a pair of image is taken for the creation of 3D model, called “Stereo Vision”.

3. Background (Human Vision, Computer Vision) • "stereo" (from the Greek word "stereos" ,which means firm or solid). • With Stereo Vision we see an object as solid in three spatial dimensions: width, height and depth or x, y and z. • probably evolved as a means of survival.

4. Human Vision Figure: Human eye using a pair of images to interpret correct vision. (Ref: http://www.vision3d.com/stereo.html )

5. Human Vision (cont..) • Humans come equipped with two eyes located side-by-side • The different perspectives of our two eyes lead to slight relative displacements of objects (disparities ) in the two monocular views of scene.   • The human visual system is able to • Use these disparities for depth-estimation. • Merge both monocular views into a fused cyclopean view of the scene (3D). • we can see where objects are in relation to our own bodies with much greater precision

6. Human Vision (cont..) • examples. of occupations that depend heavily on Stereo Vision: • Baseball player, Waitress, Driver, Architect, Surgeon, Dentist. • examples of general actions that depend heavily on Stereo Vision: • Throwing, catching or hitting a ball. • Driving and parking a car. • Planning and building a three-dimensional object. • Threading a needle and sewing. • Reaching out to shake someone's hand. • Pouring into a container. • Stepping off a step.

7. Computer Vision • Deals with the visual perception of the scene by machine through the use of cameras  • Previously identified as Digital Image processing. • Techniques for the manipulation, correction and enhancement of digital images have been in practical use for over forty years – an early application being the removal of defects from images obtained by NASA’s unmanned lunar probes.

8. Computer Vision(Cont…) • Many different areas of human endeavor, ranging from small-scale activities such as desktop publishing and healthcare, through to activity on the largest scales imaginable: the search for natural resources on Earth, or the study of other planets, stars and galaxies in our universe.  • Our concern: use stereo-vision in machine for better applications

9. Stereo Vision: Introduction • The ability to infer information on • the 3D Structures and the distances of a scene • from at least two images (Left and right), • taken from different viewpoints. • to recover depth information, • which is also known as Binocular Stereo Fusion. • In both humans and machines, • the problem reduces to a matching of the two views, • in order to find the displacement (disparity) of corresponding patterns of the projected images. • Must solve two essential problems – correspondence and reconstruction explained later.

10. APPLICATION • Autonomous Robots, Vehicles • 3D object detection, location and depth perception calculated from the stereoscopic view of objects by the pair of cameras located on the robotic body can be used for the mobilization of the robot or robotic vehicle. • The calculated results from the stereoscopic views can be used to detect and avoid the obstacles and take a safe path of action • Industrial Inspection and Quality Control • Can be used to detect the cracks, damage, unusuality in goods, raw materials or products, during manufacturing process.

11. APPLICATION (cont..) • 3D Object Location and Detection • Reconstruction of the two stereoscopic views of an object can be done to obtain a 3D object which can be recognized or detected from a known database and depth calculation can determine its location in the 3D space. • Road Monitoring/Traffic Lights • Dynamic road monitoring and traffic light management rather than the existing static approach. • Dynamic determination of the frequency and duration of traffic light alteration can be made by the amount of load in the street using Stereo Vision.

12. APPLICATION (cont..) • Virtual Reality • 3D illusion can be created by matching & fusing two different stereoscopic views of objects, scenes, etc. • 3D Animations, game play station, interactive 3D virtual world are some of explorations in virtual reality. • MilitaryApplication • With and intension to target with precision heavy loss in enemy & few loss in offending side is key to military actions. • Use of stereoscopic views to precisely calculated depth, detected & located enemy targets, auto pilot or remote controlled vehicles, air-ships can have wide use of Stereo Vision

13. Usual Paradigm • Feature Extraction • Correspondence • Disparity Estimation

14. +-* %/{}[]() $#@ !!! What is: Focal Length! Vertical Position! Horizontal Position! of Camera? Presence of Guassian Noise in Lena’s Photo Solution (Algorithm 1) Problem Solution (Algorithm 2) Which is Match for Blue Point? Exact Match! Is this a Straight Line? Issues /Challenges • Complex mathematical formulae • Which algorithm to choose? • Correspondence problem • Occlusion of object • Camera calibration • Intrinsic parameters • Extrinsic parameters • Radial Distortion • Presence of Noise • More hints from Shadows, Symmetry and Texture but How?

15. Conclusion • Exact Distance of an object from camera can be estimated. • Dimension of any geometrical object is measurable.