Sequential Adaptive Sensor Management – A. Hero

1. Sequential Adaptive Sensor Management – A. Hero • Sequential: only one sensor deployed at a time • Adaptive: next sensor selection based on present and past measurements • Multi-modality: sensor modes can be switched at each time • Detection/Classification/Tracking: task is to minimize decision error • Centralized decision making: sensor has access to entire set of previous measurements • Smart targets: may hide from active sensor Single-target state vector:

2. System Block Diagram Prediction Actions Sensor Scheduler a1 Confidence Perfmnce Monitor/ Predictor Feature Selector/ Extractor Pre- processor Feature Mapper Decisions Detector/ Classifier a3

3. Adaptive Sequential Acquisition • Sensor acquires data having density • Adaptive sensor scheduling • Sensor selection criteria: design to • Minimize predicted MSE, Pe, (Pm, Pf), time-to-detect, etc. • Maximize predicted information gain (Kreucher&etal:ISPN03): k=2 k=3 k=3 k=1

4. Progress (since June 04) Developed novel multitarget particle filter to represent the JMPD and propagate through time Developed method of adaptively factorizing the JMPD when applicable to allow for computationally tractable proposals Developed interacting multiple model formulation Studied the effect of mismatch in target motion models on filter performance Developed an importance density method for simultaneous detection and tracking that accounts for target arrival and removal Developed sensor models based on realistic GMTI, ATR, and SAR sensors Developed model for multimodality sensor that provides both kinematic and identification information and used for simultaneous detect, track, and ID of 10 real targets Multitarget Tracking via a Particle Filter Representation of the JMPD Time update : Evolve density according to Chapman- Kolmogorov Equation Propagate Particles Forward in Time Add/Remove Partitions to Particles to account for target birth/death Measurement Update density via Bayes’ Rule Update particle weights based on measurements z Resample

5. Progress (since June 04) Developed a method of information prediction based on computing the Expected Renyi Divergence between prior and posterior JMPD Implemented method using particle filter representation of the JMPD Studied the effect of mismatch in target motion models on filter performance Compared “task-driven” optimization to “information-driven” optimization Developed value-to-go approximation for tractable approximate non-myopic scheduling Developed reinforcement learning methods for non-myopic scheduling and applied to “smart” target problem using a multi-modality sensor Simulated sensor management for simultaneous detect/track and ID with multi modality sensor Information Based Sensor Resource Allocation Predict information gain for each possible sensing action Time update the JMPD Compute expected information gain between time updated JMPD and time/measurement updated JMPD Make best observation Measurement update the JMPD

6. Progress Highlighted Today • Particle Filtering for simultaneous detection, tracking, and identification (Kreucher&Etal Aerospace2005) • Investigation of sensitivity to model mismatch • Multi-modality non-myopic sensor management via Reinforcement Learning and Value-to-go Approximation (Kreucher&Hero:ICASSP2005) • Optimal multi-stage design of experiments for adaptive waveform design (Rangarajaran&etal:ICASSP2005)

7. Progress 1: PF for Simultaneous Detection, Tracking and Identification • JMPD formulation simultaneously addresses detection, tracking and identification • Until recently, our PF implementation has ignored the detection problem • Problem becomes significantly more complicated when target number is unknown and time varying • There is a non-zero probability for a new target arriving at each position within the surveillance area (leads to exponential explosion of possibilities) • Particle filter implementation must use an importance density that efficiently samples from distributions on target number and target state • Solution is a measurement-directed importance density that is biased towards proposing new targets in areas of high (accumulated) likelihood and is biased toward removing targets in areas of low likelihood • This extension allows us to solve the complete problem – target detection, tracking and identification via sensor management with no initial knowledge about the number and states of the targets.

8. Simultaneous Detection, Tracking and Identification • Simulation result • No tip-offs at startup • Unknown number of targets • Unknown position & velocities • Goal is to detect and track the ten real targets • Monte Carlo testing on the algorithm • Performance measured in two ways: • The number of targets correctly detected and tracked versus time (true number of targets is 10) • The filter estimate of target number versus time (true number of targets is 10)

9. Simultaneous Detection, Tracking and Identification • Simulation result • No tip-offs about anything at startup • Unknown number of targets • Unknown position, velocity, ID • Goal is to detect, track and identify the ten real targets • Performance measured in two ways: • The number of targets correctly detected and tracked versus time (truth is 10) • The filter estimate of target number versus time (truth is 10)

10. Progress 1: PF for Simultaneous Detection, Tracking and Identification • Simulation result • No tip-offs about anything at startup • Unknown number of targets • Unknown position, velocity, ID • Goal is to detect, track and identify the ten real targets • Performance measured in two ways: • The number of targets correctly detected and tracked versus time (truth is 10) • The filter estimate of target number versus time (truth is 10)

11. Progress 2: Effect of model mismatch Approach • We investigate the effect of mismatch between the filter estimate of SNR and the actual SNR • Experiment: 10 (real) targets with myopic SM. • CFAR detection w/ pf = .001, and pd = pf1/(1+SNR*M) • i.e. Rayleigh distributed energy returns from both background & signal. Threshold set for Pf = .001. • For a constant pf, SNR determines what pd is • Filter has an estimate of SNR (and hence pd) and uses this for SM and filtering. What is the effect on tracking of erroneous SNR info? • Bottom line: Filter appears quite robust to mismatch in SNR, pd, pf, target model.

12. We use a sensor model: p(y|S,a) For thresholded GMTI returns, this is characterized by Pd and Pf Simulation : 10 (real) targets tracking and (myopic) sensor management. How does misestimating Pd & Pf effect performance? Effect of Pd, Pf mismatch

13. Diffusive target model p(Sk,Tk|Sk-1,Tk-1) includes models of how individual targets move and how targets arrive and leave surveillance region We have been in a mismatch scenario all along since we use real targets This study quantifies how mismatch in motion model effects performance Effect of dynamic model mismatch Normalized tracking error (ratio of tracking error with mismatch to tracking error when matched) Mismatch of the filter (measured as amount of over estimation) True diffusivity of the targets

14. Progress 3: Non-Myopic Sensor Management • There are many situations where long-term planning provides benefit • Sensor platform motion creates time varying sensor/target visibility • Sensor/target line of sight may change resulting in targets becoming obscured • Delay measuring targets that will remain visible in order to interrogate targets that are predicted to become obscured • Convoy Movement may involve targets that overtake/pass one another • Targets may become closely spaced (and unresolvable to the sensor) • Plan ahead to measure targets before they become unresolvable to the sensor • Crossing Targets become unresolvable to the sensor • Sensor resolution may prohibit successful target identification if targets are too close together • Plan ahead to identify targets before they become too close • Planning ahead in these situations allows better prediction of reemergence point, target trajectory, target intention

15. Relevant Multi-target Tracking Scenario Sensor Position Shadowed Target Visible Target Region of Interest Extra dwells at time 1 help predict where target reemerges at time 6 Not made by myopic strategy Time 2 Time 3 Time 1 Non-myopic strategy scans regions that will become obscured while deferring regions that will remain visible in the future. Time 4 Time 5 Time 6

16. Value Function Approximation The Bellman equation describes the value of an action in terms of the immediate (myopic) benefit and the long-term (non-myopic) benefit. Bellman equation: Myopic part of V under action a Non-myopic correction under a Value of state • I. VTG approximation: • II. Linear Q-learning approximation: Calculate Update Qk to Qk+1 Generate s, a, s’, r s, a, s’, Qest

17. Example: Two Real Targets • Target Trajectories Taken From Real, Recorded Data • 2 moving ground targets • Need to estimate the position and velocity in x and y (4-d state vector for each target) • Time varying visibility taken from real elevation map & simulated platform trajectory • Sensor decides where to steer an agile antenna and illuminates a 100mx100m patch on the ground. Thresholded measurements indicate the presence or absence of a target (with pd and pfa) • At initialization the filter the target position is known to be in a 300m x 500m area on the ground (i.e. the prior for target position is uniform over this region)

18. Comparing the Management Strategies We Suspect that the training time for the RL algorithm could be reduced (perhaps by even an order of magnitude with a C-based implementation) • Non-myopic via RL timing • Generate Training Episodes : • (50 timesteps x 0.5s/second + 10s fixed cost per episode) * 2000 episodes = 1200 minutes • Batch training : • 36 possible actions (Q-functions to estimate) x 20 minutes per action = 720 minutes • Update value of Q function (i.e. 2nd pass) : 500 minutes • Batch train on second pass : 720 minutes

19. Example : Multiple Modality Sensor • A sensor has two waveforms • Waveform 1 (X-band) has good detection performance but is susceptible to line of sight visibility • Waveform 2 (HF) has poorer detection performance but is not susceptible to visibility • The platform is moving and so sensor to ground visibility changes with time • The filter is to detect and track a target in the surveillance area • No information about target location a priori • Q-learning used to learn the best non-myopic policy

20. Progress 4: Optimal Experimental Design • Upper left box - Beam scheduling, waveform selection, beam steering operator, and transmission into the medium, denoted by channel function • Right side box - Processes received signals and retransmits. • Lower left box - Processes output after reinsertion.

21. Motivation • Imaging a medium using an array of sensors. • Widely studied in mine detection, ultrasonic medical imaging, foliage penetrating radar, nondestructive testing, and active audio. • GOAL: Optimally design a sequence of measurements to image a medium of multiple scatterers using an array of transducers. • Four signal processing steps: • Transmission of time varying signals into the medium. • Recording of backscattered field from medium. • Transmission of the processed backscatter signals. • Measurement and spatial filtering of backscattered signals.

22. Mathematical Description • Channel between transmitted field and received backscattered field, • Four signal processing steps • where receiver noises are i.i.d • Design objective: minimize MSE under transmitted energy constraint

23. Analytical Results • Constraint: • Nearly optimal design: • MSE improvement factor:

24. Comments and Extensions • Results are robust to variation of estimator error residual esp at low SNR • Results apply to 2-stage min MSE design under average energy constraint when Greens function is known and non-random • Analytical results for multi-stage (>2) waveform design? • Random (Rayleigh/Rician) media? • Extension to non-quadratic objective functions? • Classification, detection, regularized image reconstruction?

25. Pubs Since June 2004 • Sequential adaptive sensor management • “Adaptive Multi-modality Sensor Scheduling for Detection and Tracking of Smart Targets”, C. Kreucher, D. Blatt, A. Hero, and K. Kastella, accepted for publication, Nov. 2004 • “Sensor Management Using An Active Sensing Approach ”, C. Kreucher, D. Blatt, A. Hero, and K. Kastella, accepted for publication, Oct 2004 • “Multitarget tracking using a particle filter representation of the joint multi-target probability density,” C. Kreucher, K. Kastella, and A. Hero, accepted for publication, Sept. 2004 • “Efficient methods of non-myopic sensor management for multitarget tracking,” C. Kreucher, A. Hero, K. Kastella, and D. Chang, 43rd IEEE Conference on Decision and Control, December 2004. • “Multiplatform Information based Sensor Management,” C. Kreucher, A. Hero, and K. Kastella, to appear at SPIE Defense and Security Symposium, March 2005 • “Non-myopic Approaches to Scheduling Agile Sensors for Multitarget Detection, Tracking, and Identification,” C. Kreucher, and A. Hero, to appear at IEEE ICASSP March 2005 • “Particle Filtering for Multitarget Detection and Tracking,” C. Kreucher, M. Morelande, A. Hero and K. Kastella, to appear at IEEE Aerospace Conference, March 2005

26. Pubs Since June 2004 (ctd) • Iterative function optimization • “A convergent incremental gradient algorithm with constant stepsize,” D. Blatt, A. Hero, H. Gauchman, SIAM Optimization, submitted Sept. 2004 • “Convergent incremental optimization transfer algorithms,” S. Ahn, J. Fessler, D. Blatt, A. Hero. IEEE Trans. on Medical Imaging, submitted Oct. 2004 • Predicting model mismatch • "Tests for global maximum of the likelihood function," D. Blatt and A. O. Hero, Proc. of ICASSP , Philadelphia, March, 2005. • "On tests for global maximum of the log-likelihood function," D. Blatt and A. O. Hero, , IEEE Trans. on Info Theory, submitted Jan. 2005. • Sequential waveform scheduling • "Optimal experimental design for an inverse scattering problem,“R. Rangangaran, R. Raich and A. O. Hero, to appear in Proc. of ICASSP, Philadelphia, March, 2005.

27. Synergistic Activities and Awards(2003-2004) • General Dynamics Medal Paper Award • C. Kreucher, K. Castella, and A. O. Hero, "Multitarget sensor management using alpha divergence measures,” Proc First IEEE Conference on Information Processing in Sensor Networks , Palo Alto, April 2003 • EMM-CVPR-03, ASP-03, EUSIPCO-04, ICASSP-05, SSP-05, A. Hero plenary speaker: • General Dynamics, Inc • K. Kastella: collaboration with A. Hero in sensor management, July 2002- • C. Kreucher: doctoral student of A. Hero, Sept. 2002-2004 • ARL • ARLTAB oversight: A Hero is member 2004- • ARL SEDD: A. Hero is member of yearly review panel, May 2002- • NAS-Robotics: A. Hero chaired cross-cutting review panel, May 2004. • B. Sadler: N. Patwari (doctoral student of A. Hero) internship in distributed sensor information processing, summer 2003 • ERIM Intl. • B. Thelen&N. Subotic: H. Neemuchwala (Hero’s PhD student) internship in applying entropic graphs to pattern classification, summer 2003 • Chalmers Univ., Sweden • M. Viberg: A. Hero was Opponent on multimodality landmine detection doctoral thesis, Aug 2003

28. Transitions • PF/SM to ISP Phase II (Schmidt at Raytheon) • MRF backscatter modeling to GD (Kastella/Onstott) • SM to NSF-ITR (UM, UW, BU) • SM approaches integrated into • Dynamic Machine Learning (Prof. Satinder Singh/Chris Kreucher) • Generalization error (Prof. Susan Murphy/Doron Blatt) • Collaboration with Prof. Hilllel Gauchman (UIUC Math) on distributed optimization • Collaboration with GD on Willow Run experiment for multi-modal tracking of dismounts and vehicles

29. Personnel on A. Hero’s sub-Project (2003-2004) • Chris Kreucher, 4th year grad student • UM-Dearborn • General Dynamics Sponsorship • Neal Patwari, 3rd year doctoral student • Virginia tech • NSF Graduate Fellowship/MURI GSRA • Doron Blatt, 3rd year doctoral student • Univ. Tel Aviv • Dept. Fellowship/MURI GSRA • Raghuram Rangarajan, 3rd year doctoral student • IIT Madras • Dept. Fellowship/MURI GSRA