1 / 29

Status of AiTR/ATR in Military Applications

Status of AiTR/ATR in Military Applications. James A. Ratches CERDEC NVESD January 2007. UMDAROATR. Outline. Definition Importance & Scenarios Performance Assessment Problem Statement Way Forward Summary & Conclusions. Military Definition.

yamal
Download Presentation

Status of AiTR/ATR in Military Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Status of AiTR/ATR in Military Applications James A. Ratches CERDEC NVESD January 2007 UMDAROATR

  2. Outline • Definition • Importance & Scenarios • Performance Assessment • Problem Statement • Way Forward • Summary & Conclusions

  3. Military Definition • Generic term to describe automated/semi-automated • functions carried out on imaging sensor data to perform • operations ranging from cuing a human observer to • complex fully autonomous object acquisition and • identification • Machine function: • - Detection - Classification - Recognition - Identification - Friend or Foe • Aided Target Recognition (AiTR) • - Machine makes some level of decision and annotates the image • - Human makes higher level decision. e.g. to identify and fire • ATR is fully autonomous • - No human in-the-loop after weapon firing, e.g. fire-and-forget seeker • ATR/AiTR may use information from other sensors to make • decision by fusing information AiTR (aided) ATR (autonomous)

  4. Rapid wide area search for close combat in high clutter, against difficult targets (occlusion, defilade, CC&D) and variable target signatures road debris road Urban terrain; 360 degree situational awareness, short ranges, human intent, transmission limitations New object Tree trunks UAV & UGV transmissions over limited bandwidth BDA Detection of Dismounts & intent, & bunkers 1030 1830 Scouts in Overwatch-Objects of interest and scene changes Scenarios Where AiTR Essential

  5. Missile Scenarios Where ATR Essential 1st Waypoint (Tower) Fire Units Obstacle 2nd Waypoint (Mountain) Engagement Area Field Of View Target of Opportunity 1 Km 4 Km Detect, Recognize & Identify Target (Engage Autotracker) Navigate To Emplacement Site Missile Auto-Navigate To Target Search Point (Enroute Recon) - Start Search (Wide FOV) - Locate Target (Narrow FOV) - Lock On - Aimpoint Update • Receive Target InformationThrough C2 Network • - Verify Target Selection • - Route & Salvo Selection • Launch Missile(s) - Power Up - Computer Initialization - Intelligence Preparation of Battlefield - Plan Missile Routes if necessary • Acquire GPS Satellites • Update GPS Position • Calibrate the Inertial System • Navigate to Target Area - In-Flight Intelligence - Target Marking - Target Reportingto C2 Network Warhead FunctionOn Impact

  6. AiTR Annotates Images – Not Maps Which pixels in image correspond to targets?

  7. Manual FLIR (300 FOR) Search Time > 60 Sec. Aided Target Search Time less than 4 Sec. Effects of Clutter Clutter levels: High Hunter Ligget Medium Yuma Low Grayling Effects of Occlusion + H M D H M D H M D N A V N A V N A V V I S M O D E W P U P D T W P A P L T S C R N B R T S L A V E W N 2 9 S E A X N B 2 3 2 0 4 4 5 6 F T N B 5 2 0 0 2 2 5 0 1 Y B C T R S Hunter Ligget Yuma Grayling + + T S D T S D T S D T S D T S D T S D H O M E S C L E T A C N A V C N T R W N D W WIDE AREA TARGET CUEING WITHIN 4 SECONDS Lab/Field Measured Performance

  8. COMMON SENSOR Long Range Scout Surveillance System 2nd Gen FLIR Modified f/ Gimbaled Scan Target Acquisition Sensor Suite (TASS) • SWIR CAMERA • Long Range Target Identification • Leverage ACT II LIVAR and CETS Program - EBCCD Technology • 4.5” Aperture • AiTD/R • Assess Maturity Ground Based AiTD/Rs in Varied Environmental Conditions • MTI Radar • Utilize AN/PPS-5D Laser Illumination/ Designation • Gimbaled Scan FLIR • Long Range Target Detection • 2nd Gen B-Kit (LWIR) • Assess maturity of • SOA AiTR in gimbal • scanning mode in • the field • SOA single color/ • shape LWIR based • algorithms from • multiple sources • Include urban bkgds • and man targets Evaluations yield ROC curves AiTR/ATR Continues to Be Tested in Realistic Environments

  9. Overall Assessment • DOD investment in AiTR has resulted in quantifiable • level of performance documented in ROC curves • 2. Performance measured under favorable conditions • Order of magnitude improvement in search time with • AiTR over human only • Discrimination levels above detection have not been • vigorously pursued* • 5. Detection performance can have degradation for sub- • optimal conditions* • - high clutter - low contrast - obscuration - extended ranges • Training target sets have been typically for < 10 targets • There are no human detection algorithms SOA AiTR Algorithms Have Known Limitations * Especially for ground-to-ground

  10. Robust AiTR/ATR Critical for Ground-to-Ground Close Fight Manned-shoot first Unmanned-autonomous operation Need for Robust AiTR/ATR For future combat scenario must be robust - High false alarm rate renders aid useless and operator will turn it off (AiTR) - Ground-to-ground presents high clutter - Target variability increases complexity - Low signature targets can be expected - Partial occlusion & defilade obscures the target - CC&D need to be mitigated - Detect human threats in urban terrain - Final ID can be man-in-the-loop (AiTR)

  11. The AiTR/ATR Problem 1 0.9 0.8 Aided 0.7 0.6 Probability of Detection 0.5 0.4 Manual 0.3 0.2 0.1 0 0 20 40 60 Search Time (sec) • ~$100M investment to realize SOA AiTR • Humans can still do better than SOA AiTR (Except for speed) • Robust AiTR required - Potential target set is largewith wide range of environmental and operational variations -AiTR for humans and urban terrain • New university concepts have not migrated to industry and military developers • AiTR/ATR Cannot Do As • Well As The Human • Alone-However, • It Can Do It Faster • Improvement that approaches • human performance will be • an enabling force multiplier ARL-SEDD DATA

  12. Perceived Impediments to ATR/AiTR • Required computational power • High cost, power and size • Proprietary issues • Tactical scene complexity • Required to be better than human • New CONOPS will be needed to • fully utilize benefits of ATR/AiTR Real Limitation Is The Lack of An Image Science- What Is Important in An Image?

  13. Possible Paths to Improvement • 3D LADAR – if can cover/search field of regard • - Otherwise, use for higher level discrimination • Multi/hyper-spectral/look/mode sensor and • Sensor Fusion • Untried University “New Ideas” • - Recognition by parts • - Advanced eye-brain understanding • - Gradient index flow and active contour analysis • - Frame-to-frame correlations • - Spatial contextual intelligence • - Hierarchical imaging • - Category theory • Off-board sensor features data via low bandwidth • tactical networks • Validatedsynthetic image generation to stress • algorithms during formulation • Investment in Image Science

  14. ARO/Duke University Workshop Computational Sensors for Target Acquisition and Tracking Beaufort, NC December 2-4, 2003 Representative Recommendations • Different approach to applying eye-brain understanding to • AiTR needed • - Does not necessarily mean that we need to mimic that process • Artists may give a unique insight into minimalist • representation • Poor performance of AiTRs relative to humans suggest there • are better features than have been found by AiTRs • The perspective of clutter rejection rather that object feature • extraction may present a different set of opportunities Eye-Brain Understanding Can Still Be A Fertile Ground of Investigation for AiTR Concepts

  15. Progression of Algorithms ALGORITHM FEATURE EXAMPLE CAPABILITY METRIC 1. Statistical a. no range ΔT, size, perimeter, etc. Comanche - target in open & in center of FOV - ~ 10 target set in low clutter - baseline performance b. w/range same + target window size - reduce search time (10X) - reduce FAR (10X) 2. Template Matching comparison of ROI to stored SAIP - expand target set, e.g. aspect, target templates articulation, dirurnal/seasonal, etc. 2. Model Based comparison of ROI to stored MSTAR - increase target set with stored data target model set reduction 4. Multi-spectral pixel value=f(λ,Δλ) MFS3 - penetrate camouflage - reduce FAR 5. Multi-look target indications at GPS Dynamic - reduce FAR (~ 10-100X) by coords from off-board Variable by correlating target detects sensors Threshold - detect obscured, defilade targets - missed target reduction (~2X) 6. Multi-mode non-imaging sensor indications ASM - mitigate CC&D (sound, vibration, magnetics) algorithms - reduce FAR * * * * * * * Classified data on false alarms and Pd exist for these algorithms

  16. Progression of Algorithms (con’d) ALGORITHM FEATURE EXAMPLE CAPABILITY METRIC * 7. Geographic Contour Maps terrain slope DTED - FAR reduction (potential ~ 75%) 8. Advanced Eye-Brain synapse maps NN, holographic NN, - intelligent search & detect Understanding & wavelets - FAR reduction Representation - reduce search timelines 9. Recognition-by-Parts target subelements - detect partially obscured targets detected - missed target reduction 10. Gradient Index Flow & 2D chips of humans - determine human intent Active Contour Analysis 11. Frame-to-Frame pixel change MTI - detect changes in scene Correlations correlations - reduce search times 12. Spatial Contextual target forbidden terrain - reduce search time Intelligence by reducing search area 13. Artists Insights hierarchical scene - reduce search time by characteristics focus on search area 14. Hierarchical Imaging activate/retard signals - bandwidth reduction by evaluating information before transmission 15. Category Theory sensor report & - geolocation accuracy locations improvement S2 Sw Sc Sf S1

  17. Theoretical Basis for Multi-Look • Different features have different ROC curves • Range dependent • Features from different sensors and platform can be passed over the network (low bandwidth information) • Performance gain proportional to ROC curves • Pick 2 features as example • local variation • wavelet ROC Curves for 2100m, 2500m, and Fused 1 0.9 10X FAR reduction 0.8 0.7 0.6 0.5 Pd 0.4 0.3 Fused Best: Pd Indep., Pfa Indep. Fused: Pd Indep., Pfa Fully Correl. 0.2 Fused Worst: Pd, Pfa Fully Correl. 2100m 0.1 2500m 0 0 Pfa Features from Off-Board Sensors Can Improve On-Board Sensor AiTR/ATR

  18. V2 V1 LWIR-2 LWIR-1 LWIR-1 R4 R2 LWIR-2 R3 R2 R1 V2 Most false alarms for LWIR-2 Most false alarms for LWIR-1 Most false alarms for LWIR-1 Most false alarms for LWIR-2 Side View Plan View False Alarms Uncorrelated between Sensors Ridge Sensor Valley Sensor

  19. Recognition is based on recognition of critical sub-components called geons Specifications S: S1. S2 data from sensors 1 & 2 Sc real world stimuli Sw ground truth Sf registration transformation between S1 & S2 Morphisms: arrows Functors: Relationships with other categories Sf T-72 tank S1 Sw S2 Sc Example of a category Category Theory Recognition-by-Parts • Category Theory is a mathematically • sound framework: • Designed for network applications • Describes information fusion systems • and processes in an elegant language • - Captures commonality and relationships between objects Hot spot Gun barrel geon2 Turret geon3 geon1 FLIR Image geon4 Tracks & wheels geon5 Engine exhaust O1 O2 O3 O4 c Kokar (Northeastern) a b Biderman (USC) cxaxc=(cxb)xa=cx(bxa) Composition operation that is associative Network supplies opportunity for sophisticated fusion techniques to be applied to AiTR Library of Geons for targets of interest forms the basis for recognition

  20. Context “Image Science” Based Algorithms SOA algorithms attempt to recognize static targets in single frames: Need to consider more image-based, e.g. parameters e.g., image temporal-spatial relationships. Sensor-Scene Dynamics Change detection & MTI Algorithms Must Extract More Contextual Information

  21. Gradient Vector Flow (GVF) Gradient of intensity (x, y) Eye-Brain Understanding Must Be Applied Faithfully (Active Contour Analysis) • Higher level process or user initializes any curve close to the • the object boundary (indication of a region of interest) • The parametric curves (snakes) then starts deformingand • moving towards the desired object boundary • In the end it completely “shrink-wraps” around the object Human intent GVF field is defined to be a vector field X [x(s), y(s)] for s in [0,1] Solve Euler equation αx''(s) - βx''''(s) - Eext = 0 to minimize energy functional E = ∫01 ½ (α│x'(s)│2 - β│x''(s)│2) + Eext (x(s))ds (α and β user defined constants) Zucker (Yale) & Xu & Prince (JHU)

  22. Artists Unique Insight This painting shows how Van Gogh was able to transmit detailed Information about a person (20-year old woman) to the viewer Using Only ~10 brush strokes for her face. From Falco (U of AZ)

  23. Hierarchical Imaging &Target Representation Elements of Network Make Localized Decisions Rather Than Simply Sending Raw Data to A Central Processor • Sensors Sample n Parameters • Network becomes large scale sensor • Hierarchical decisions • - Local decisions determine relevant information • - Global decisions develop global model • - Each node is a virtual point detector at the next • level • - Algorithms determine what is to be shared/when The Network Becomes The Sensor & AiTR

  24. Conclusions • SOA ATR/AiTR has attained a level of performance that • has some level of military value • - Targets in the open • - Low to medium clutter • - Target set ~ 10-15 • - No obscuration or camouflage • - No humans or human intent • - No high value targets, e.g. bunkers • Major new innovations are needed to get a leap • ahead in performance under operational environments • - New university concepts • - Network information

  25. Back Up Slides

  26. Aided TargetRecognition for Intelligent Search Neural Net Prescreener Feature Extraction Recognition Registration M35 @ 0°, scale = 1.0 outer ZSU @ 165°, scale = 0.85 Inner M60 @ 180°, scale = 0.85 M35 @ 270°, scale = 1.0 M35 @ 195°, scale = 1.0 Range x Sensor M-35 Representative Configuration of SOA AiTR

  27. Crater 3D from Optical Flow Above Surface Mound • Subtle motion provides substantial depth information • Memory/ processing advances permit harvesting of depth • information (Target/Sensor motion) • Algorithms have been developed that amplify motion vectors • and present them in a binocular display in real time to create • “hyperstereo” using advances in microlens technologies Processing Motion Information Can Provide Depth (Range) Courtesy of FOR 3D, Santa Rosa, CA

  28. Passive Ranging with DTED Data DTED Data Lines of constant range superimposed on FLIR image at Hunter- Liggett Lines of constant range superimposed on FLIR image where earth is flat at Yuma P.G. Flat Earth Approximations Cannot Be Used for All Scenarios Of Interest Work on passive ranging and imagery by Raytheon.

  29. Passive Ranging Optical flow GPS Lines of constant range DTED overlaid on imagery El & Az of gun known Near field objects Eye motion Far field objects “ Despite often heard claims to the contrary, without range data there is no way of knowing if the target is 100- times smaller than a pixel or 1000 times larger than the image as a whole. – Northrop-Grumman • Accurate range estimation can reduce false alarm rates in AiTR • Permits estimation of target size • Active ranging potentially reveals position • Most AiTR algorithms make a flat earth approximation • Passive ranging may provide more adequate accuracy DTED w/GPS

More Related