Faster and More Accurate Face Detection on Mobile Robots Using Geometric Constraints

1. Faster and More Accurate Face Detection on Mobile Robots Using Geometric Constraints Michael Dixon, Frederick Heckel, Robert Pless, William D. Smart Washington University in St. Louis

2. Motivation • Why is object detection important? • Examples • Autonomous vehicles • Human-robot interaction

3. Object detection in 2D • Common approach • Learn a classifier from labeled examples • Exhaustively classify all subwindows in the image Over 3 million subwindows in a 640×480 image

4. Object detection on a robot • Advantages of robot vision • Known camera position and calibration • Knowledge of environment • Additional sensors • Goal: use this additional information to reduce unnecessary tests by the detector • Less computation • Fewer opportunities for false positives

5. Images are projections • Image subwindows correspond to a range of possible 3D positions and sizes If certain 3D positions and sizes can be ruled out, you can avoid testing the corresponding subwindows

6. Geometric constraints • Establish bounds on an object’s physical position and size • Use the known camera to relate those physical bounds to the image • Only test a subwindow with the detector if it satisfies the geometric constraints

7. Face detection on robots • Perfect for geometric constraints • Robot has known camera • People’s heights and sizes are constrained Known calibration Known position Height constraint Size constraint

8. Applying geometric constraints • For each subwindow, compute the range of depths consistent with the constraints • If the range is empty, the subwindow can be safely ignored Satisfies height constraint Satisfies size constraint

9. Height constraints • Given: • height of camera, h • subwindow center, (u,v) • ground-plane normal, n • height constraints, hmin and hmax • Project (u,v) to ray in world space, r • Compute the range of depths consistent with the height constraints r hmax h hmin n

10. Size constraints • Given: • width of the subwindow, w • the camera’s focal length, fx • size constraints, smin and smax • Compute the range of depths consistent with the size constraints smax rs smin

11. Combining constraints • For each subwindow, (u,v,w): • Compute range of valid depths, • Can pre-compute quickly • Assuming no camera roll,u can be ignored • For all v, compute Dh • For all w, compute Ds • Store each D in v by wlook-up table • Only update if the camera tilts or zooms Ds(w) Dh(v) D

12. Incorporating depth measurements • For each subwindow, compute range of depths, M, consistent with external depth measurements • If D and M do not overlap, the subwindow can be safely ignored Stereo disparity

13. Incorporating laser range data • Project laser readings into image • Estimate depth at each pixel assuming a uniform footprint • Preprocess the range data to fill small gaps Laser rangefinder

14. Evaluation • B21r, SICK PLS Laser rangefinder (180 degrees), Bumblebee stereo camera • Control: Ran OpenCV detector at 25 scales from 20 to 200 pixels, scaling by a factor of 1.1, shifting window in increments of 0.5·w/w0 300 pairs of stereo images, 416 labeled faces

15. Reduction in computation • Geometric constraints alone cut computation in half • Incorporating depth from stereo or laser reduces computation by an average of 85%

16. Improved accuracy • Evaluated the detector over the full range of sensitivity thresholds • Three times fewer false positives

17. Example results Control • OpenCV face detector • Geometric constraints + laser measurement Laser