Applied Mathematics in Defense Applications

1. Applied Mathematics in Defense Applications Andrea Bertozzi Department of Mathematics University of California, Los Angeles

2. Topics • Multisensor fusion • Image and human event fusion for statistical density estimation • Automated boundary tracking for autonomous robots and high dimensional imagery • Collaborative searching through swarming • Diffuse interface methods in imaging • Segmentation with corners • Imaging through turbulence • Direct sparse deblurring • Geographic profiling • Crime hotspots • Gang violence data • Predicting crime

3. Data Fusion – Multiple sensors • Pan Sharpening – panchromatic (greyscale) higher spatial resolution, multiband – lower spatial resolution – IKONOS and QUICKBIRD satellite • Hyperspectral sharpening – panchromatic obtained separately (may not be perfect match) – hyperspectral can have hundreds of bands – contain material information • Human Event data – events in space and time fused with geographical data (e.g. residential burgalaries) • Point sensor data – mobile sensor data

4. Data Fusion and Segmentation– Multiple sensors • Pan Sharpening – panchromatic (greyscale) higher spatial resolution, multiband – lower spatial resolution – IKONOS and QUICKBIRD satellite • Hyperspectral sharpening – panchromatic obtained separately (may not be perfect match) – hyperspectral can have hundreds of bands – contain material information • Human Event data – events in space and time fused with geographical data (e.g. residential burgalaries) • Point sensor data – mobile sensor data

5. Panchromatic signal is not a linear combination of isolated bands

6. Recent pansharpening techniques • IHS • Brovey • PCA • Wavelet Fusion • First variational approach: ’A Variational Model for P+XS Image Fusion’, Ballester, Casselles, Igual, Verdera, 2006

7. Intensity Hue Saturation Results Assumes panchromatic is a linear combination of spectral bands.

8. Variational Wavelet PansharpeningMichael Moeller, Todd Wittman, ALB, preprint

9. Wavelet matching – data must be registered to dyadic scaling (pansharpening)

10. Full VWP variational problem

11. Alternate VWP – avoids switching from wavelet to physical space

12. Numerical results

13. Hyperspectral data fusion Michael Moeller, Todd Wittman, and Andrea L. Bertozzi, A Variational Approach to Hyperspectral Image Fusion, Proc. SPIE Conference on Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XV. Orlando, Florida. April 2009.

14. Spatial detail inherited from master image – spectral detail from AVIRIS data

15. Spectral preservation - examples

16. Spectral angle is preserved

17. George Mohler, Andrea Bertozzi, Tom Goldstein, Stan Osher Fast TV regularization for 2D Maximum penalized likelihood estimation, preprint 2009 • Method for estimating non-smooth probability densities • Important for estimating threat level based on event data and other intel. • TV based regularization allows for best estimation of densities with spatial discontinuities. • Computationally challenging in multi-D • Challenge solved using Split Bregman L1 minimization technique. • Tested using V-fold Cross Validation with large 2D data sets. • Tested on data from LAPD for residential burglaries.

18. Maximum Penalized Likelihood Estimationbasic problem • Estimate probability density u(x) from point data x1, x2, x3, etc. • General approach for regularizer R(u). • For discontinuous densities, choose R = TV

19. Example from San Fernando ValleyData courtesy of LAPD • Point process data for residential burglaries • No residences in area in middle • Actual data TV method (new) kernel estimation (old) • TV method is much closer to real problem, does not bleed threat level into region where threat is not active. Can be fused with other types of data, such as spatial visual, infrared, LIDAR etc as long as one has a model to incorporate this into the problem.

20. Density Estimation for Sparse DataLaura Smith, Matthew Keegan, Todd Wittman, ALB UCLA • Point data of individual events that come from a background source • Examples – human event activity – burglaries – what is the probability of event as a function of space? • Data is sparse – want to fuse with other information e.g. overhead imagery Improving Density Estimation by Incorporating Spatial Information, preprint 2009.

21. Overhead imagery vs. human events

22. San Fernando Valley Burglaries

23. Orange County Coastline

24. Experimental Validation of Cooperative Environmental BoundaryTracking with On-board Sensors A. Joshi, T. Ashley, Y. Huang, and A. L. Bertozzi, Experimental validation of cooperative environmental boundary tracking with on-board sensors, American Control Conference, St. Louis, MO, June 2009, pp. 2630-2635.

25. Control Algorithms for Boundary Tracking • UUV-gas bang-bang type steering controller • Time-corrected algorithm • Robotic path planning – • Hsieh et al Amer. Contrl. Conf. 2005 • Jin and ALB, IEEE CDC 2008 • Joshi et al Amer. Contrl. Conf. 2009

26. Bang-bang type steering Control Law • UUV-gas algorithm:

27. Time-corrected Steering Control Law • Time-corrected algorithm: • Includes time difference between crossing points on boundary, • Reduces to the bang-bang type controller when,

28. Decision Algorithm CUSUM Filter • Upper : indicates teal tape • Lower : indicates black tape

29. Single Vehicle Implementation • A Kalman inspired pre-filter was used to weakly damp the noisy signal. • Essentially a simple proportional model with the empirical gain factor from Kalman filtering a previous data set. • The Gain value is in fact the convergent Kalman gain (this is expected in view of the fairly constant, though high, noise co-variance of the testbed)

30. Single Vehicle Implementation • The sharp features when on the black tape cause the decision algorithm to slip-up occasionally • Pre-filtering reduces these errors considerably

31. Single Vehicle Implementation • Vehicle speed 0.3m/s • Left raw data -> CUSUM • Right raw data -> prefilter -> CUSUM

32. Cooperative boundary tracking • The global control law held admirably • The time difference between data points is 24 s

33. A. Chen, T. Wittman, A. Tartakovsky, and A. L. Bertozzi Image segmentation through efficient boundary sampling, in SAMPTA '09, Marseille, May 18-22, 2009.

34. Sensor data processing Kalman filter to reduce noise CUSUM filter to check threshold Movement control of agents Models for search phase, target locating phase Target location estimated Performance Average time to locate a target Average error in estimate of location Number of false registers Scaling properties Estimate for swarm size, measured by diameter Optimal swarm diameter (analytical approximation, upper bound) W. Liu, M.B. Short, Y. E. Taima, and A. L. Bertozzi,Multiscale Collaborative Searching Through Swarming, preprint.

35. Overview of the algorithm Collaborative Searching through SwarmingW. Liu, M.B. Short, Y. E. Taima, and A. L. BertozziIntroduction 2

36. Collaborative Searching through SwarmingW. Liu, M.B. Short, Y. E. Taima, and A. L. BertozziMovement Control of Agents • Search phase (with Levy flight) • Target locating phase • Location estimation

37. Collaborative Searching through SwarmingW. Liu, M.B. Short, Y. E. Taima, and A. L. BertozziSensor data processing • Agent sensor reading • Kalman filter to reduce noise • CUSUM filter to detect threshold

38. Collaborative Searching through SwarmingW. Liu, M.B. Short, Y. E. Taima, and A. L. BertozziPerformance • Simulations of a 20 by 20 dimensionless board, 32 agents, target sensing radius 1.0, 200 trials • Divide-and-conquer and whole region search strategies • Average time to locate a target and average location estimate error measured • Larger swarms are more accurate, multiple smaller ones more efficient

39. Collaborative Searching through SwarmingW. Liu, M.B. Short, Y. E. Taima, and A. L. BertozziScaling Properties • Swarm diameter D scales with inter-agent distance • For 25% of agents to sense before deciding to locate, • Optimal D maximizes separation between the center of the swarm and the target location • The average time to locate a target is

40. Diffuse interface methods Total variation Ginzburg-Landau functional

41. Cahn-Hilliard Inpainting Bertozzi, Esedoglu, Gillette, IEEE Trans. Image Proc. 2007, SIAM MMS 2007 Patent pending. Transitioned to NGA for road inpainting. Transitioned to InQtel for document exploitation. Continue edges in the same direction – higher order method for local inpainting. Fast method using convexity splitting and FFT H-1 gradient flow for diffuse TV L2 fidelity with known data

42. Wavelet Allen-Cahn Image Processing • Dobrosotskaya, Bertozzi, IEEE Trans. Image Proc. 2008, Preprint subm. IFB. • Transitioned to NGA for road inpainting. • Transitioned to InQtel for document exploitation. • Nonlocal wavelet basis replaces Fourier basis in classical diffuse interface method. • Analysis theory in Besov spaces. • Gamma convergence to anisotropic TV. H-1 gradient flow for diffuse TV L2 fidelity with known data

43. Convex Splitting Schemes Schoenlieb and Bertozzi, submitted Basic idea: Art is to choose Ec to give an implicit problem that is easy to solve - e.g. Ec is H1 semi norm – can be solved using FFT - in wavelet case Ec is wavelet Laplace operator Contraints on Ec and Ee so that splitting is unconditionally stable Proof of convergence of splitting schemes for various higher order inpainting methods.

44. Marc Droske and Wenhua Gao Segmentation with Corners Idea – segmentation requires a regularization It is analogous to denoising. CV, Snakes reduce length of curve. Removes corners as well as noise. Instead regularize with the “curve” analogy of TV – nonlinear penalization of curvature-based functional. Low Curvature Image Simplifiers (Tumblin & Turk SIGGRAPH 2000, Bert. And Greer CPAM 2004) Extend to curve evolution using either Lagrangian framework or Level sets. Image Snakes (KWT ‘88) Chan-Vese 2001 Droske & Bertozzi – geometric corner snakes 2009 CV with corners 2009 Higher-order feature-preserving geometric regularization, SIAM J. Img. Sci. 2010.

45. Imaging through turbulence morning afternoon Images taken at China Lake – courtesy of Alan Vannevel and Gary Hewer When you image at a kilometer anisplanatic effects are relevant – we need better deblurring and deconvolution methods. Often imformation is known about the image – the difficulty is to extract features.

46. Lou, Bertozzi, Soatto, submitted 2009 Direct Sparse Deblurring Uses training data Dictionary based Inverse problem not solved Fit data to blurred dictionary then directly unblur Solves problem of amplifying noise with solution of inverse problem Blurry data ROF deblurring Our method

47. Geographic ProfilingGeorge Mohler and Martin Short

48. Estimation of probabilities

49. 2004 San Fernando Valley Data

50. 2007 Los Angeles Burglary Data