Stereoscopic Light Stripe Scanning: Interference Rejection, Error Minimization and Calibration

1. Stereoscopic LightStripe Scanning:Interference Rejection,Error Minimizationand Calibration By: Geoffrey Taylor Lindsay Kleeman Presented by: Ali Agha April 13th, 2009

2. Motivation • Measuring arbitrary scenes in ambient indoor light (Purpose: Visual Servoing for a Humanoid Robot) • Addresses the problem of rejecting interference due to secondary specular reflections, cross-talk and other mechanisms in an active light stripe scanner.

3. Motivation

4. Basic Operation Color cameras capture stereo images of the stripe at 384 × 288 pixel Frame rate (25 Hz) on the 2.2 GHz dual Xeon host PC.

5. System Model Encoder measurement

6. Problem Statement Given the laser plane position and the measurements Lx, Rx, Rx́, one of the left/right candidate pairs, (Lx,Rx) or (Lx, Rx́), must be chosen as representing stereo measurements of the primary reflection. The measurements should then be combined to estimate the position of the ideal projection

7. Previous work • In Trucco, et al. (1994) and Nakano, et al. (1988), laser stripe measurements are validated by applying a fixed threshold to the difference between corresponding single-camera reconstructions Such a comparison requires a uniform reconstruction error over all depths, which this figure illustrates is clearly not the case.

8. General Solution Given is the optimal reconstruction Minimize S.t. Unconstrained Version The Plücker matrix L describing the back-projection line is The intersection X of the light plane and L is

9. General Solution Finally, the ideal projection corresponding to is obtained by projecting onto the left image plane: By some simplifications: And, the error function becomes

10. Special Case: Rectilinear Stereo and Pin-Hole Cameras • With and the image plane error E can be expressed as a function of a single unknown

11. Validation • determining which pair of measurements correspond to the primary reflection • 1) Light plane parameters α, β, and γ are calculated from e and the system parameters • 2) • 3) • 4) the optimal reconstruction is finally calculated

12. Laser Plane Error • The above solution assumes that the parameters of the laser plane are known exactly. • In practice, the encoder measurements are noisy • Let and , i = 1 . . . n, represent valid corresponding measurements of the laser stripe on the n scanlines in a frame. • Levenberg–Marquardt (LM) algorithm for minimization • The optimal correspondences and encoder count are calculated recursively.

13. Additional Constraints • 1) stripe candidates must be moving features • It has little effect on cross-talk or reflections. • 2) that valid measurements only occur within a subregion of the left and right image planes, depending on the angle of the light plane.

14. Active Calibration Unknown parameters in the model of the light stripe scanner The validation problem is approximated by recording only the brightest pair of features per scanline. Let and , represent the brightest corresponding features on njscanlines of t captured frames, and let ejrepresent the measured encoder value for each frame. where

15. Active Calibration where Initial estimate implemented numerically using LM minimization The system parameters and encoder values are then sequentially refined in an iterative process. The calibration technique presented here is practical, fast, and accurate. The method does not require accurate knowledge of camera parameters b and f.

16. Implementation • Output of the scanner is a 384 × 288 element range map • The shaft encoder and stereo images are recorded at regular 40 ms intervals (25 Hz PAL frame rate). • A complete scan requires approximately 384 processed frames (15 s).

17. Implementation: Light Stripe Measurement • Laser stripe extraction is performed using: • intensity data only (average of the color channels) • motion of the stripe (by subtracting the intensity values in consecutive frames) • predicted sub-region of the image. The intensity profile on each scanline is then examined to locate candidate stripe measurements.

18. Implementation: Range Data Post-Processing • 1) Despite robust scanning, the raw range map may still contain outliers • Thresholding: the minimum distance between each 3D point and its eight neighbors should be less than 10 mm • 2) Holes • fills these gaps with interpolated depth data. • The distance between the bracketing points must be less than 30 mm • 3) Finally, a color image is registered with the range map.

19. Implementation: Range Data Post-Processing

20. Experimental Results • A mirror behind the objects simulates the effect of cross-talk and reflections.

21. Experimental Results • output of the single-camera scanner • phantom surfaces appear (Erroneous associations between the phantom stripe and laser plane)

22. Experimental Results • output of the double-camera scanner • Based on Nakano et al. (1988) and Trucco et al. (1994) • The single-camera reconstructions XLand XRare calculated independently • Discarded when |XL−XR| exceeds a fixed distance • The final reconstruction is calculated as (1/2)(XL+ XR)

23. Experimental Results • robust scanner result

24. Discussion • Main limitation • unsuitable for dynamic scenes • Robot must remain stationary during a scan • The experimental prototype uses a red laser diode • 1) Only can sense surfaces which contain a high component of red • 2) laser diode could be replaced by a white light source • 3) Advantages of LDs: physical compactness, low power consumption and heat generation. • 4) The light plane could be generated using a triplet of red, green, and blue laser diodes. • 5) High cost of green and blue laser diodes

25. Discussion • surfaces with high specular and low Lambertian reflection may appear invisible

26. Summary and Conclusions • Measuring arbitrary scenes in ambient indoor light • Robustly identify the light stripe in the presence of secondary reflections, cross-talk and other sources of interference. • Optimization-based formulation • An image-based procedure for calibrating the light plane parameters

27. Future Research • Development of a multistripe scanner. • Multistripe scanners have the potential to solve a number of issues associated with single-stripe scanners: • Illuminating a target with two stripes could double the acquisition rate • Projecting the stripes from different positions reveals points that would otherwise be hidden in shadow. • single-camera multistripe systems mostly rely on color, sequences of illumination or epipolar constraints to disambiguate the stripes. However, the method proposed in this paper could allow the stripes to be uniquely identified using the same principles that provide validation for a single stripe.

28. Thank You Questions?