Lecture 11 Vision-based Landing of an Unmanned Air Vehicle

1. Lecture 11Vision-based Landing of an Unmanned Air Vehicle Invitation to 3D vision

2. Predator Global Hawk UCAV X-45 SR/71 Fire Scout Applications of Vision-based Control Invitation to 3D vision

3. Challenges Hostile environments Ground effect Pitching deck High winds, etc Why vision? Passive sensor Observes relative motion Goal: Autonomous landing on a ship deck Invitation to 3D vision

4. Simulation: Vision in the loop Invitation to 3D vision

5. Motion estimation algorithms Linear, nonlinear, multiple-view Error: 5cm translation, 4° rotation Real-time vision system Customized software Off-the-shelf hardware Vision in Control Loop Landing on stationary deck Tracking of pitching deck Vision-Based Landing of a UAV Invitation to 3D vision

6. Vision-based Motion Estimation Current pose Image plane Feature Points Pinhole Camera Landing target Invitation to 3D vision

7. Pinhole Camera: Epipolar Constraint: Planar constraint: More than 4 feature points Solve linearly for Project onto to recover Pose Estimation: Linear Optimization Invitation to 3D vision

8. Objective: minimize error Parameterize rotation by Euler angles Minimize by Newton-Raphson iteration Initialize with linear algorithm Pose Estimation: Nonlinear Refinement Invitation to 3D vision

9. Pinhole Camera Multiple View Matrix Multiple-View Motion Estimation Rank deficiency constraint Invitation to 3D vision

10. n points in m views Equivalent to finding s.t. Initialize with two-view linear solution Least squared solution: Use to linearly solve for Iterate until converge Multiple-View Motion Estimation Invitation to 3D vision

11. Ampro embedded Little Board PC Pentium 233MHz running LINUX 440 MB flashdisk HD robust to vibration Runs motion estimation algorithm Controls Pan/Tilt/Zoom camera Motion estimation algorithms Written and optimized in C++ using LAPACK Estimate relative position and orientation at 30 Hz UAV Pan/Tilt Camera Onboard Computer Real-time Vision System Invitation to 3D vision

12. On-board UAV Vision System Vision Computer Camera RS232 Vision Algorithm WaveLAN to Ground Frame Grabber RS232 Navigation System Navigation Computer INS/GPS RS232 RS232 WaveLAN to Ground Control & Navigation Hardware Configuration Invitation to 3D vision

13. Acquire Image Threshold Histogram Segmentation Target Detection Corner Detection Correspondence Feature Extraction Invitation to 3D vision

14. Pan/Tilt to keep features in image center Prevent features from leaving field of view Increased Field of View Increased range of motion of UAV Camera Control Invitation to 3D vision

15. Ground Station Comparing Vision with INS/GPS Invitation to 3D vision

16. Motion Estimation in Real Flight Tests Invitation to 3D vision

17. Landing on Stationary Target Invitation to 3D vision

18. Tracking Pitching Target Invitation to 3D vision

19. Contributions Vision-based motion estimation (5cm accuracy) Real-time vision system in control loop Demonstrated proof of concept prototype: first vision-based UAV landing Extensions Dynamic vision: Filtering motion estimates Symmetry-based motion estimation Fixed-wing UAVs: Vision-based landing on runways Modeling and prediction of ship deck motion Landing gear that grabs ship deck Unstructured environments: Recognizing good landing spots (grassy field, roof top etc) Conclusions Invitation to 3D vision