Detecting & Modeling Change in Time-Varying Imagery

1. Detecting & Modeling Change in Time-Varying Imagery Peggy Agouris Dept. of Spatial Information Engineering University of Maine

2. Overview • Problem(s) • Change Detection in Time-Varying Aerial Imagery • Tracking Positional Change and Modeling Spatiotemporal Behavior in Motion Imagery (incl. Video Sequences) • Examples

3. Problem Change detection: one component of successful conflation of geospatial information

4. Integrated Environment

5. Traditional Snakes • Semi-automatic tool for object extraction • Based on the optimization of a model of curve contrast and smoothness using content-derived forces and an elastodynamic model

6. Traditional Snakes Model • Total energy: • Continuity term: • Curvature term: • Edge term:

7. vi+1 vi-1 vi Optimization • Greedy algorithm : current point location is optimized, while previous and next points are fixed • Stop criteria : number of points moved, change of total energy

8. Quality Evaluation of Extracted Road Network • Need a posteriori evaluation of object extraction • Results are useful input for spatiotemporal change detection • Assumption: known energy function values • For sample points along an extracted object, values of uncertainty are generated using fuzzy rules

9. E High Medium Low DE Low High Low Low High High Medium High Quality Evaluation Rules • Points of interest are determined based on statistical properties of energy • Fuzzy rules of the form : • If Et is LOW and DEt is LOW then U is LOW

10. F out v i F in d v0 i Differential Snake Model • Additional energy term (uncertainty) • Action is similar to an elastic spring force

11. Eunc 1/Unc(v0i) Di d 0 Uncertainty Energy

12. Change Detection vs. Versioning • Change is detected if a road segment has moved beyond the stochastic range of older information • Versioning identifies road segments that can be delineated in the new image with better accuracy than their current database record

13. Example Prior (red) and current (blue) road shape information

14. Change Detection & Versioning Experiments Buffer zones of influence of prior information Result of change detection (blue line) Result of versioning (purple line)

15. Change Detection & Versioning Experiments (cont.) Prior and current road shape information Buffer zones of influence of prior information After change detection (blue) and versioning (purple)

16. Differential Snakes GUI (Change Detection & Versioning)

17. Performance and Accuracy Issues • Typical Performance Metrics • Average values for a road segment spanning a 512x512 image window: • Change Detection: 2.095 sec • Versioning: 0.561 sec

18. Change Detection for Closed Contour Objects • Important tool for dynamic scene analysis • Applications: surveillance, environmental, transportation, biomedical, etc. • Quick and efficient, requires proper initialization, assumes frequent monitoring (small movement of object between frames)

19. Differential Snakes Extracted Object Contour from Previous Frame New Frame Information New Object Contour Differential Snakes for Tracking Object Contours

20. Estimation of Translation and Rotation • Translation: difference of positions of two geometric centers • Rotation: difference of direction of principal axes 

21. Estimation of Uniform Expansion Ratio of areas = (ratio of perimeters)2

22. Estimation of Radial Deformation • Use of polygon clipping techniques: - calculate the intersections between two input polygons - label edges as inside, outside, or shared - find the minimal polygons which are created by intersection - classify all minimal polygons into the output sets AB, A/B, and A\B

23. Experiments with Moving Objects • Track changes in the shape of an object • Example: a liquid that deforms non-uniformly • We show five distinct frames and the detected change between them (frames n, n+1) • Area threshold to ignore small polygon changes • Integration of spatiotemporal tracking process in a GUI (in Matlab)

24. Experiment with a Moving Object

25. GUI for Spatiotemporal Change Detection

26. Remarks • Integration of object extraction and change detection • Introduction of uncertainty as external energy in a deformable model • Change detection using the uncertainty of the extraction • Framework for spatiotemporal tracking of object deformations • Estimate translation, rotation, radial deformations using geometric properties

27. Positional Change Detection & Modelingin Motion Imagery

28. Problem • Detecting change in position and shape/extent of objects or events across time and space • Modeling their spatiotemporal behavior

29. Rationale • Trends in imagery collection: from static to motion and from single to multiple sensors. • Tremendous amounts of data. • Bottleneck in the analyst workforce.

30. Needs Automated motion imagery analysis solutions Automation at various levels of the analysis process: • automated identification of trajectories in single video feeds (i.e. tracking positional change over time) • automated content analysis to identify interesting spatiotemporal activities and support queries

31. Essential Issues • Motion Trajectory Identification • Nodal Representation of Trajectories • Spatiotemporal Helix Modeling

34. Modeling Spatiotemporal Change Over time objects/events may change their: • location (movement) • outline (deformation) Need: • an integrated representation of movement and deformation

35. Moving Objects in the SpatioTemporal Domain Trajectories of moving objects: 3-d collections of points evolving through S-T space • Generalization • Summarization • Behavior Analysis

36. t t x x y y Summarization

37. The SpatioTemporal Helix • An integrated representation of movement and deformation, and • A signature of an object’s spatiotemporal behavior. • Comprises a spine and prongs • Spine models trajectory • Nodes: acceleration, deceleration, rotation • Prongs express deformation • Changes of a predefined magnitude • Recorded as time, percent change, azimuth

38. Helix Representation Spine:expresses spatio-temporal 3-D movement of the center of mass. Prongs:express expansion or collapse of the object’s outline

39. The Helix as a Spatiotemporal Database Index Helixobjidt1,t2={node1,…noden; prong1,..prongm} • Node:ni(x,y,t,q) • Prong: pj(t,r,a1,a2)

40. Collecting Spine & Prong Information • Two novel image analysis techniques: • SOM with geometric analysis (g-SOM) • Describes ST trajectory of center of mass • Differential snakes • Allows calculation of percent change in outline

42. Generalization

43. Hurricane Frances

44. Hurricane Charley

45. Hurricane Helixes

46. Hurricane Helixes: Charlie

47. Hurricane Helixes: Frances

48. Concluding Remarks • Incorporating uncertainty in change detection improves conflation and eliminates false positives • Detection of positional and shape change in motion imagery contributes to a better understanding of behavior of evolving events

49. For more information: Peggy Agouris Anthony Stefanidis {peggy, tony}@spatial.maine.edu http://dipa.spatial.maine.edu