Cooperative Air and Ground Surveillance

1. Cooperative Air and Ground Surveillance Wenzhe Li

2. Outline • Introduction • Experimental Testbed • Framework • Air-Ground coordination • Experiment Results • Conclusion

3. Introduction The use of robots in surveillance and exploration is gaining prominence. • Surveillance • Target detection • Tracking • Search and rescue operations

4. UAV and UGV • UAV(Unmanned aerial vehicle) Advantage: Move rapidly, Cover large area Disadvantage: Low accuracy for localization • UGV(Unmanned ground vehicle) Advantage: High accuracy for localization Disadvantage: Not move rapidly, can not see through obstacles. Main idea arise from answering question : How to make it both Move rapidly and Accurately locate target ?

5. Major topics covered in this paper In this paper, authors present the approach to cooperative search, identification, and localization of targets using a heterogeneous team of fixed-wing UAV and UGVs. Three major topics. • Synergy of UAVs and UGVs • Framework • Algorithms to search and localization

6. Contribution of paper • Framework is scalable to multiple vehicles. • Decentralized Algorithms for control of each vehicle • Easy Implemented, independent of number of vehicles, offer guarantee for search and localization

7. Before moving to next section… • How to integrate UAVs and UGVs ? • What UAVs and UGVs be responsible for? (to exhibit complementary capability) • Why such framework is scalable to large system? • What techniques to use to solve problem? • ……….

8. Outline • Introduction • Experimental Testbed • Framework • Air-Ground coordination • Experiment Results • Conclusion

9. UAV Airframe and Payload ◆ onboard embedded PC ◆ IMU 3DM-G from MicroStrain ◆ external global positioning system (GPS): Superstar GPS receiver from CMC electronics, 10 Hz data ◆ camera DragonFly IEEE-1394 1024 × 768 at 15 frames/s from Point Grey Research ◆ custom-designed camera-IMU Pod includes the IMU and the camera mounted on the same plate. The plate is soft mounted on four points inside the pod. Furthermore, the pan motion of the pod can be controlled through an external-user PWM port on the avionics.

11. Ground Station • Each UAV continuously communicate with Ground Station Communication : 1hz, up to 6mi • Performs GPS corrections and Flight Update • Concurrently monitor up to ten UAVs • Direct communication between UAVs via Ground Station and 802.11b • Ground station has an operator interface program

13. The UGV Platform

14. Outline • Introduction • Experimental Testbed • Framework • Air-Ground coordination • Experiment Results • Conclusion

15. Framework • Information-driven framework • ASN(Active sensor network) architecture • Key idea: sensing action -> reduction in uncertainty • Utility on robot and sensor state and actions • Target Detection • Target Localization

16. Screen clipping taken: 2010/3/29, 11:11 Target Detection • Certainty Grid : our representation certainty grid is a discretestate binary random field in which each element encodes the probability of the corresponding grid cell being in a particular state 1. Yd,i(k|k) = logP(x) = logP(s(Ci) = target). where subscript d denotes detection, stores the accumulated target detection certainty for cell i at time k 2. id,s(k) = logP(z(k)|x) Information associated with the likelihood of sensor measurements z 3. Updated by the log-likelihood form of Bayes rule: Identify cells that have an acceptably high probability of containing features or targets of interest.

17. Target Localization • Target Localization : Second part of task • Problem posed as a linearized Gaussian estimation problem • Kalman filter is used

18. Target Localization • Vector Yf : Coordinates of all the features detected by the target detection algorithm • Yf,i : denoting the (x, y) coordinates of the feature in a g • lobal coordinate system • Information filter maintains Yf,i(k | k) and matrix Yf,i(k | k) • Estimation mean and covariance by • Fusion of Ns sensor measurements

19. Uncertainty Reducing Control • Entropy-based measure • Mutual information measures • Control objective is to reduce estimate uncertainty • Uncertainty directly depends on the system state and action • Vehicle chooses an action that results in a maximum increase in utility or the best reduction in the uncertainty

20. Scalable Proactive Sensing Network • Can be deployed for searching for targets and for localization • Search and localization algorithms are driven by Information-based utility measures • Independent of the source of the information • Nodes automatically reconfigure themselves in this task • Scales to indefinitely large sensor platform teams

21. Outline • Introduction • Experimental Testbed • Framework • Air-Ground coordination • Experiment Results • Conclusion

22. Air-Ground Coordination • The search and localization task consists of two components: 1. First, detection of an unknown number of ground features in a specified search area ˆyd (k|k). 2. The refinement of the location estimates for each detected feature Yf,i(k|k).

23. Feature Observation Uncertainty

25. Optimal Reactive Controller for Localization • Controller is a gradient control law, which automatically generates sensing trajectories that actively reduce the uncertainty in feature estimates by solving: where U is the set of available actions, and If,i(ui(k)) is the mutual information gain for the feature location estimates given action ui(k). • For Gaussian error modeling of Nffeatures

26. Optimal Reactive Controller for Localization

27. Outline • Introduction • Experimental Testbed • Framework • Air-Ground coordination • Experiment Results • Conclusion

28. Aerial images of the test site captured during a typical UAV flyover at 65 m altitude. Three orange ground features highlighted by white boxes are visible during the pass.

31. 1. When only use UAVs : In excess of 50 passes (about 80 min of flight time) 2. When only use UGVs : In excess of half an hour for the ground vehicle 3. When they are collaborative: completes this task in under 10 min

32. Outline • Introduction • Experimental Testbed • Framework • Air-Ground coordination • Experiment Results • Conclusion

33. Conclusion Unique Features: 1. Methodology is transparent to the pecificity and the identity of the cooperating vehicles. 2. Computations for estimation and control are decentralized 3. Methodology presented here is scalable to large numbers of vehicles.