1 / 37

Geo-Spatial Aerial Processing for Scene Understanding and Object Tracking

Geo-Spatial Aerial Processing for Scene Understanding and Object Tracking. Jiangjian Xiao, Hui Cheng, Feng Han, Harpreet Sawhney. Problem. Given Aerial Video Understand the Scene Find buildings Trees Roads Cars Use understanding Object Detection Tracking Cool Idea

abe
Download Presentation

Geo-Spatial Aerial Processing for Scene Understanding and Object Tracking

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Geo-Spatial Aerial Processing for Scene Understanding and Object Tracking Jiangjian Xiao, Hui Cheng, Feng Han, Harpreet Sawhney

  2. Problem • Given Aerial Video • Understand the Scene • Find buildings • Trees • Roads • Cars • Use understanding • Object Detection • Tracking • Cool Idea • Trees and buildings are in 3D

  3. Related Work CVPR 2006 Hui Cheng, Darren Butler and Chumki Basu ViTex: Video To Tex and Its Applications in Aerial Video Survellance.

  4. Related Work CVPR2008 Jake Porway, Kristy Wang, Benjamin Yao, Song Chun Zhu A Hierarchical and Contextual Model for Aerial Image Understanding

  5. System Overview Stage 1 Initial camera location Geo-reference image Input Frames Pose estimation Depth estimation Geo-registration Non-ground object detection Planar + depth extension for structure detection Road Detection GIS Stage 2 Scene segmentation output

  6. Stage1 Stage 1 Initial camera location Geo-reference image Input Frames Pose estimation Depth estimation Geo-registration

  7. GeoRegistration GPS Aircraft Parameters Camera Parameters Geo-reference image Meta Data Input Frames Geo-registration

  8. GeoRegistration Frame To Frame transformations SIFT matching GPS Aircraft Parameters Camera Parameters Bundle Adjustment

  9. Stage1 Stage 1 Initial camera location Geo-reference image Input Frames Pose estimation Depth estimation Geo-registration

  10. Adjusting camera position • Metadata Gives camera position • Along with many other parameters • Metadata has error • In all parameters • Georegistration overcomes error • Returns a 3x3 homography matrix • Want to figure out the exact camera position

  11. Adjusting camera position Ground Point Image Point Project Ground point to image

  12. Adjusting camera position Alternatively the point can be projected using homography obtained from georegistration Get rid of translation parameters

  13. Adjusting camera position Extract rotation and calibration parameters using SVD smooth and Using Kalman filter Use refined and to estimate translation parameters

  14. Stage1 Stage 1 Initial camera location Geo-reference image Input Frames Pose estimation Depth estimation Geo-registration

  15. Depth Estimation • Use graphcuts to estimate depth • A difficult task due to poor image quality, and unconstrained motion • Solution • Fuse depthmaps • Project several depthmaps unto the DOQ • Take their average • Smooth out the average map • Depth is quantized along Z direction

  16. Depth Estimation

  17. Stage 2 Non-ground object detection Planar + depth extension for structure detection Road Detection GIS Stage 2 Scene segmentation output

  18. Detect Non-Ground Regions Threshold Depth Map

  19. Stage 2 Non-ground object detection Planar + depth extension for structure detection Road Detection GIS Stage 2 Scene segmentation output

  20. Detect Roofs Threshold Depth Map Fit Plane Remove Trees

  21. “Roof” Refinement Fit a plane to the detected “roofs”. We have a set of x,y,z points Want to fit

  22. “Roof” refinement v u z Z Depth Along u Must be invariant Z Y

  23. Building Detection Extend Roof To Ground Gives Building height

  24. Tree Detector Classify each pixel as tree non-tree 9D Gaussian Mixture Color, Depth, Texture Supervised offline training

  25. Stage 2 Non-ground object detection Planar + depth extension for structure detection Road Detection GIS Stage 2 Scene segmentation output

  26. GIS constrained Road Detection Want to determine Precise road center Road Width Road Information Provided by GIS

  27. Training • Sample Patches along roads • Align patches along road direction • Extract Features • Color • Gradient • Feature Vector = histogram of color and gradients • Model: Gaussian Mixture model • Offline Training

  28. Detection Align the Road Extract patches Feed patches into MOG model Response of the model Gradient Histogram Gives Road center Peaks Give Road bounds

  29. Road Detection

  30. Object Detection • Stabilization • Optical flow warping • Depth warping

  31. Tracking with/without depth without depth with depth

  32. Tracking with/without depth without depth with depth

  33. Quantitative Results Multiple object racking accuracy False acceptance count False rejection count False identity switches Ground truth object count

  34. Quantitative Results MOTA improvement: 0.740 to 0.851 (15% improvement) FAR improvement: 0.190 to 0.072 (62% improvement)

  35. More Results

  36. More Results

  37. More Results

More Related