200 likes | 397 Views
Lecture 11 Vision-based Landing of an Unmanned Air Vehicle. Predator. Global Hawk. UCAV X-45. SR/71. Fire Scout. Applications of Vision-based Control. Challenges Hostile environments Ground effect Pitching deck High winds, etc Why vision? Passive sensor Observes relative motion.
E N D
Lecture 11Vision-based Landing of an Unmanned Air Vehicle Invitation to 3D vision
Predator Global Hawk UCAV X-45 SR/71 Fire Scout Applications of Vision-based Control Invitation to 3D vision
Challenges Hostile environments Ground effect Pitching deck High winds, etc Why vision? Passive sensor Observes relative motion Goal: Autonomous landing on a ship deck Invitation to 3D vision
Simulation: Vision in the loop Invitation to 3D vision
Motion estimation algorithms Linear, nonlinear, multiple-view Error: 5cm translation, 4° rotation Real-time vision system Customized software Off-the-shelf hardware Vision in Control Loop Landing on stationary deck Tracking of pitching deck Vision-Based Landing of a UAV Invitation to 3D vision
Vision-based Motion Estimation Current pose Image plane Feature Points Pinhole Camera Landing target Invitation to 3D vision
Pinhole Camera: Epipolar Constraint: Planar constraint: More than 4 feature points Solve linearly for Project onto to recover Pose Estimation: Linear Optimization Invitation to 3D vision
Objective: minimize error Parameterize rotation by Euler angles Minimize by Newton-Raphson iteration Initialize with linear algorithm Pose Estimation: Nonlinear Refinement Invitation to 3D vision
Pinhole Camera Multiple View Matrix Multiple-View Motion Estimation Rank deficiency constraint Invitation to 3D vision
n points in m views Equivalent to finding s.t. Initialize with two-view linear solution Least squared solution: Use to linearly solve for Iterate until converge Multiple-View Motion Estimation Invitation to 3D vision
Ampro embedded Little Board PC Pentium 233MHz running LINUX 440 MB flashdisk HD robust to vibration Runs motion estimation algorithm Controls Pan/Tilt/Zoom camera Motion estimation algorithms Written and optimized in C++ using LAPACK Estimate relative position and orientation at 30 Hz UAV Pan/Tilt Camera Onboard Computer Real-time Vision System Invitation to 3D vision
On-board UAV Vision System Vision Computer Camera RS232 Vision Algorithm WaveLAN to Ground Frame Grabber RS232 Navigation System Navigation Computer INS/GPS RS232 RS232 WaveLAN to Ground Control & Navigation Hardware Configuration Invitation to 3D vision
Acquire Image Threshold Histogram Segmentation Target Detection Corner Detection Correspondence Feature Extraction Invitation to 3D vision
Pan/Tilt to keep features in image center Prevent features from leaving field of view Increased Field of View Increased range of motion of UAV Camera Control Invitation to 3D vision
Ground Station Comparing Vision with INS/GPS Invitation to 3D vision
Motion Estimation in Real Flight Tests Invitation to 3D vision
Landing on Stationary Target Invitation to 3D vision
Tracking Pitching Target Invitation to 3D vision
Contributions Vision-based motion estimation (5cm accuracy) Real-time vision system in control loop Demonstrated proof of concept prototype: first vision-based UAV landing Extensions Dynamic vision: Filtering motion estimates Symmetry-based motion estimation Fixed-wing UAVs: Vision-based landing on runways Modeling and prediction of ship deck motion Landing gear that grabs ship deck Unstructured environments: Recognizing good landing spots (grassy field, roof top etc) Conclusions Invitation to 3D vision