General Imaging Model

1. General Imaging Model Michael Grossberg and Shree Nayar CAVE Lab, Columbia University ICCV Conference Vancouver, July 2001 Partially funded by NSF ITR Award, DARPA/ONR MURI

2. Imaging • What is a general imaging model ? • How do we Compute its Parameters ? Scene Imaging System Images

3. rays become image points rays selected Perspective Imaging Model Camera Obscura

4. Systems that are not perspective compound eyes catadioptric system multiple camera system fisheye lens

5. General Imaging Model • Essential components: • Photosensitive elements • optics i Pi • Maps incoming pixels to rays

6. Raxel symbol Index Geometry Radiometry Position Direction Point Spread Fall-off Response Raxel = Ray + Pixel • Small perspective camera • Simple lens • One pixel photo-detector • Most general model is a list of raxels

7. virtual detectors (raxels) • (qq, qf) (pX,pY,pZ) physical detectors (pixels) ray surface imaging optics Ray Surfaces Position: (pX,pY,pZ) Direction: (qq, qf)

8. caustic Rays in 2D perspective non-perspective • Singularity of rays called a caustic position-direction space q Y X position space

9. Solve for d Computing Caustics • Change coordinates • (x,y,d) (X,Y,Z)

10. Caustic Ray Surface • Caustic is a singularity or envelope of incoming rays • Caustic represents loci of view-points imaging optics raxels Caustic curve

11. Simple Examples perspective single viewpoint multi-viewpoint

12. h(x) • Linear fall-off of optical elements Normalized Fall-off Raxel index g(e) Normalized Response Normalized Exposure (e) Raxel Radiometry • Non-linear response of photosensitive element

13. y sb Image plane sa Point Spread • Elliptical gaussian model of point spread. • Major and minor deviation lengths, sa (d), sb (d) • Angle of axis y(when sa (d), sb (d) are different) Chief ray d, Scene depth Impulse at Scene point

14. Finding the Parameters • Known optical components: Compute • Unknown optical components: Calibration Environment

15. Calibration Apparatus • Structured light at two planes • Geometry from binary patterns • Radiometry from uniform patterns pf i pn qf z

16. Finding the parameters: Perspective System video camera with perspective lens laptop LCD sample image translating stage

17. Computed Raxel Model: Geometry 180 160 140 120 X in mm 100 80 60 180 160 Y in mm 140 360 120 340 320 100 300 80 280 Z in mm 260

18. Computed Raxel Model: Radiometry • Pointwise fall-offh(x,y) • Radiometric response g(e) 1 . 0 0 . 9 1 1 . . 0 0 0 . 8 0 0 . . 8 8 0 . 7 normalized response normalized fall-off 0 . 6 0 0 . . 6 6 0 . 5 0 0 . . 4 4 0 . 4 0 . 3 0 0 . . 2 2 0 . 2 0 0 . . 0 0 0 . 1 0 . 0 0 0 . . 0 0 0 0 . . 1 1 0 0 . . 2 2 0 0 . . 3 3 0 0 . . 4 4 0 0 . . 5 5 0 0 . . 6 6 0 0 . . 7 7 0 0 . . 8 8 0 0 . . 9 9 1 1 . . 0 0 0 50 100 150 200 250 300 normalized exposure radius in pixels

19. Finding the parameters: Non-single Viewpoint System video camera with perspective lens laptop LCD sample image parabolic Mirror translating stage

20. 10 5 0 -5 60 40 -10 20 0 -15 -20 -40 -20 -60 -25 -30 -35 -60 -40 -20 0 20 40 60 Computed Raxel Model: Geometry • Rotationally symmetric mm from caustic max mm from axis of symmetry mm from axis of symmetry

21. Computed Raxel Model: Radiometry • Fall-off toward edge as resolution increases: • less light collected normalized fall-off radius in pixels

22. Index Geometry Radiometry Position Direction Point Spread Fall-off Response x, y pX, pY, pZ qq, qf sa, sb, y h g(e) Summary • Most general model simply list of raxels • Caustics summarize geometry • Simple procedure for obtaining parameters from a black box system