Building Omnicam and Multisensor Cameras

1. Building Omnicam and Multisensor Cameras Presented by John M. Hart Research Engineer, Computer Vision and Robotics Laboratory, Beckman Institute

2. Omnicam Cameras • Omnicam – Omnifocus Non-Frontal Imaging Camera • Reorient the Sensor Plane to Cover the Entire Range of Focus • Pan the Camera While Take Overlapping Images with Varying Degrees of Focus • Search Each Image to Find the Best Focus Pixel for Each Scene Point in the Image • Build Final Image Pixel by Pixel • Concept Developed by Arun Krishnan • Patent Granted to Arun Krishnan and Professor Ahuja

3. Field of View • Capture a Large Field of View • Also Known as Panoramic Imaging • Known Distortion Problems Using Wide Angle Lenses • Use 1945 Circuit Camera Omnicam Image of Capital Building

4. Seamless Imaging • Combining Images to Construct a Panoramic • Leads to Problems with Alignment, Lighting and Focus Changes from One Image to Another • Precision Motion Control, Calibration, and Omnicam Focus Can Provide Seamless Images

5. Depth of Field • Capturing an Infinite Depth of Field • Normal Image Focused at 4 Feet Causes Objects at 2 and 30 Feet to be Out of Focus • Omnicam Image Captures all Objects in Focus Regardless of Their Distance From the Camera

6. The First Nicam • Original Prototype of Nicam • Motor to Pan Over Selected Area • Difficult to Calibrate • Not Easy To Conduct Experiments • Large Size >1.5 Cubic Feet • Slow Imaging • Camera Design by Arun Krishnan as Part of His Thesis

7. Portable Nicam • Experimental Platform • 6DOF Positioner Enclosed to Modify Sensor Orientation • CCD Sensor Removed from Commercial Camera • Considerable Size Reduction • Precision Motor for Pixel Level Accuracy • 30 Degrees per Second Rotation During Image Capture • Camera Design and Construction by John M. Hart

8. Self-Contained Unit • Requirements • Permanent Calibration • Self-Contained unit • Design • Commercial Camera w/Sensor Removed • Pentium PC-104 System Running Windows • Color LCD and Touch Pad for Simple User Interface • Zip Drive for Transferring Large Images Picture by Chris Brown • Camera Design by Manoj Aggarwal and John M. Hart

9. Remote Nicam • Requirements • Versatile Mounting Unit • Rotate Freely for 360s or Continuous Surveillance • Design • Smallest Motor with Accuracy Required • Slip Ring Contact System • Frame Grabber and Motion Controller Cards for Any PC • Camera Base Now 4”x4” • Camera Design by John M. Hart and Manoj Aggarwal

10. Results: Beckman Garden Panoramic • A 100 Degree Omnicam Panoramic Image of the Beckman Institute Garden • The Flowers are at 1.5 Feet, • The Tree is at 4 Feet, • The Bench is at 10 Feet, • And the Building is at 50 Feet

11. Results: Beckman Garden Panoramic • The Flowers are at 1.5 Feet, • The Tree is at 4 Feet, • The Bench is at 10 Feet, • And the Building is at 50 Feet • A 100 Degree Omnicam Image of the Beckman Institute Garden

12. Results:Omnifocus Panoramic • 360 Degree Omnifocussed Image of Computer Vision and Robotics Laboratory • Note Signs for 2 Feet and 30 Feet are in Focus

13. Results: Range Information • In Addition, We Know in which Image the Object Point was in Focus so there is a Depth Estimate for Each Pixel • Top Image is an Omnifocus Nicam Visual Image • Middle Image Contains Range Estimates for Each Pixel in the Visual Image and Represents the Structure of the Scene • Bottom Image is a Virtual Omnifocused 3D Structure

14. Multisensor Camera • Requirements • Allow Multiple Images to be Taken Simultaneously • Split Light from Lens to Multiple Sensors • Design • Custom Lens Design* • Novel Beam Splitter*+ • Symmetric Mounting of Sensors Picture by Don Hamerman • *Part of Manoj Aggarwal’s PhD Thesis • +Under Patent Application • Camera Design by Manoj Aggarwal and John M. Hart

15. A Look Inside the Multisensor Camera • Splits Scene into Three Identical Copies without Aberrations • CCD Sensor Settings can be Adjusted to Capture Same Scene with Different Setting • Design Allows for Additional Sensors in Future • How to Calibrate? Picture by Don Hamerman

16. Multisensor Camera Mechanical Parts • Over Sixty Machined Aluminum Parts each Requiring a Detailed Drawing • Including Screws - Over 150 Parts to Assemble • Average Camera Design, Construction, Calibration, and Testing Six Months to a Year Machining Completed by the ECE Machine Shop

17. Calibration:“Moving” Targets • They are Attached to Stages to Allow Aligning them with the Optical Axis of the Camera’s Lens • These Targets are Mounted to the Optical Table and are Used for Calibrating the 6DOF of the CCD Sensor

18. Calibration:Types of Targets • Dot Target • Used for Finding the Optical Axis of the Lens and Aligning the Center of the CCD Array • Texture Target • Used for “Flattening” the CCD Sensor and Measuring Relative Focus

19. Calibration:6DOF Positioner • Three Translational and Three Rotational Stages • Allows Orientation of the CCD Sensor with Respect to Lens • Order of Stages Important • Using 6DOF Positioner to Align 3 Sensors of Camera • Stand Allows for Taking Advantage of Camera Symmetry

20. Calibration:Sensor Calibration • Calibration Set-up on Optical Table • Keeps Camera, Calibration Unit and Targets Properly Aligned • Done in the Dark so Aperture can be Wide Open • Computer Automated Calibration Software

21. Camera Testing:Getting out of the Lab • Allows Images to be Taken Under Many Different Lighting Conditions • Outside for Example • Cart Used for Testing • Not Really Portable Need to Plug In • Laptop Stores Images Too Slowly

22. Results: Obtaining High Dynamic Range • Three CCD Sensors Have Settings Adjusted Differently • Captures Same Scene with Different Exposure Settings or Filters

23. Results: Intensity Space Mosaicing • Pixels are Selected from Images Containing Suitable Intensity Values to Compose A High Dynamic Range Image • Therefore, Detail Can be Seen in Direct Sunlight and in Shadows

24. Future • Firewire will Allow Us to Use a Laptop in Our Designs • Implement Algorithm for a Color Sensor • Video is Possible using Multisensor Camera • Combine Focus and Dynamic Range with Panoramic • Virtual Reality with 3D Scene of Actual Images from the Environment • Technology Transfer • Immersive Imaging Applications • If you need any assistance in creating your prototypes, please don’t hesitate to ask. A 180 Degree Omnicam Image of the Beckman Atrium

25. Special Thanks • Thanks to all the people who have contributed to our camera projects. • Arun Krishnan • PhD “Non-Frontal Imaging Camera” 1997 • Andres Castano • PhD “Range From A Non-Frontal Imaging Camera” 1998 • Manoj Aggarwal • PhD “High Performance Imaging” 2001 • Winner of the Ross Martin Graduate College Thesis Award • National Science Foundation • Office of Naval Research • DARPA • Prof. Ahuja and the Computer Vision and Robotics Laboratory • Photographs of Cameras by Don Hamerman • Mr. Henderson and the Guys at the ECE Machine Shop • Also the Beckman Machine Shop • The Visualization, Media and Imaging Laboratory Picture by Don Hamerman

26. Building Omnicam and Multisensor Cameras Picture by Don Hamerman Presented by John M. Hart Research Engineer, Computer Vision and Robotics Laboratory, Beckman Institute