Optical Flow Math 680 Professor A. Hicks Given by: Bill Green & Rares Stanciu

1. Optical FlowMath 680Professor A. HicksGiven by: Bill Green & Rares Stanciu

2. What is Optical Flow? • Optical flow is the velocity field which warps one sequential image to another

3. First Condition • Camera is stationary • Only objects that are moving have optic flow vectors

4. Second Condition • Camera is moving • Relative to camera, all object appear to be moving

5. Characteristics • Objects in close proximity will appear to be moving faster than far away objects

6. Optical Flow Formula OF = w + (v/d) sin q

7. Other Applications • Conclude if objects are undergoing linear or rotational motion • Determine the distance between the camera and the object • Tracking moving objects

8. Challenges with Optical Flow • Correspondence Problem • Relating pixels in corresponding images via a rotation and translation • Aperture Problem • only able to measure the component of optical flow that is in the direction of the intensity gradient.

9. Autonomous Landing of UAVs • The UAVs altitude can be calculated from optical flow • “Obstacle” is the ground; q = 90 • UAV does not rotate during landing

10. Our Experiment • Determine distance to obstacle using 2 sequential images OF = v/d v d Top view

11. First Attempt • Calculate optic flow • Threshold and binarize the images • Find image centroids • OF value of 110 pixels/sec was calculated • distance, d, to the target was found • 1 inch (actual dist was 22 ft)

12. Find the Calibration Constant • Took 2 sequential images • Distance from object to camera was known • Camera’s velocity was also known • Compute optic flow • OF = 140; vcam = 12 in/s; d = 163 in • Find K => OF = vcam/dK • K = 5.26 E-4

13. Redid Calculations • Distance from object to camera was unknown • Camera’s velocity was known • Compute optic flow • OF = 108; vcam = 15 in/s • Find d => OF = vcam/dK • d = 22 feet • Actual distance was 22’ 6” Error = 2.22%

14. Load images Threshold Binarize Calculate distance between 2 centroids to obtain OF vector Compute centroid Matlab Code • Feature Tracker => centroid

15. Optic Flow Sensor

16. References • B.K.P. Horn and B.G. Schunck. “Determining optical flow”, AI Memo 572. Massachusetts Institute of Technology, 1980. • Barrows, G., Chahl, J.S., Srinivasan, M.V., "Biomimetic Visual Sensing and Flight Control", 2002 Bristol UAV Conference, Bristol, UK April 2002 • Barrows, G., Neely, C., "Mixed-Mode VLSI Optic Flow Sensors for In-Flight Control of a Micro Air Vehicle", SPIE, San Diego, CA, July 2000 • Barrows, G., "Future Visual Microsensors for Mini/Micro-UAV Applications“

17. Questions??

18. Mathematical Background • Assumption: The apparent brightness of moving objects remains constant between frames • I(x(t+Dt), y(t+Dt), t+Dt) = I(x(t), y(t), t) (1) • Taylor expansion of the left term • I(x(t+Dt), y(t+Dt), t+Dt) = I(x(t), y(t), t) + Ixu + Iyv + ItDt (2) where u & v are the optic flow vectors • From (1) and (2), optical flow equation is: • Ixu + Iyv + ItDt = 0