1 / 38

A Forest of Sensors: Using adaptive tracking to classify and monitor activities

A Forest of Sensors: Using adaptive tracking to classify and monitor activities. MIT AI Lab Eric Grimson, Chris Stauffer , Lily Lee, Raquel Romano, Gideon Stein. A Forest of Sensors: Using adaptive tracking to classify and monitor activities. MIT AI Lab Eric Grimson, Chris Stauffer ,

glutz
Download Presentation

A Forest of Sensors: Using adaptive tracking to classify and monitor activities

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Forest of Sensors:Using adaptive tracking to classify and monitor activities MIT AI Lab Eric Grimson, Chris Stauffer , Lily Lee, Raquel Romano, Gideon Stein Forest of Sensors

  2. A Forest of Sensors:Using adaptive tracking to classify and monitor activities MIT AI Lab Eric Grimson, Chris Stauffer , Lily Lee, Raquel Romano, Gideon Stein Forest of Sensors

  3. A Forest of Sensors • Given autonomous vision modules (AVMs): low power, low cost, can compute wide range of visual routines • Create a forest of disposable AVMs: • attached to trees, buildings, vehicles • dropped by air • Question: Can the forest bootstrap itself to monitor sites for activities? Forest of Sensors

  4. Multi-camera version Forest of Sensors

  5. Video Frames Adaptive Background Model Pixel Consistent with Background Model? X,Y,Size,Dx,Dy X,Y,Size,Dx,Dy X,Y,Size,Dx,Dy Local Tracking Histories Forest of Sensors

  6. Capabilities and components • Self calibration • building rough site models • detecting visibility • primitive detection • moving object modeling • activity detection • activity calibration Forest of Sensors

  7. Working hypothesis • Can achieve all of these capabilities simply by accurately tracking moving objects in the scene. Forest of Sensors

  8. A robust, real-time tracker • Model each pixel as an independent process • Model previous n samples with weighted mixture of Gaussians, using exponentially decaying time window • Background defined as set of dominant models that account for T percent of data • Update weights and parameters • Pixel > 2 sigma from background is moving • Select significant blobs as objects Forest of Sensors

  9. Adaptive tracking Forest of Sensors

  10. Dynamic calibration • Track objects in multiple cameras • use correspondences to find a homography, assuming planar motion • modify homography by fitting image features • use non-planar features to solve for epipolar geometry, and refine Forest of Sensors

  11. Dynamic calibration Forest of Sensors

  12. Site modeling • N camera stereo from calibration • dynamic tracking updates • visibility detection and placement update • reconstruction from multiple moving views Forest of Sensors

  13. Site models by tracking Forest of Sensors

  14. Site models by tracking Forest of Sensors

  15. Site modeling from stereo • Multi-camera stereo reconstruction Forest of Sensors

  16. Stereo reconstruction Forest of Sensors

  17. Activity detection & calibration • Need to find common patterns in track data Forest of Sensors

  18. Pattern tracks Forest of Sensors

  19. Detect Regularities & Anomalies? Forest of Sensors

  20. Example track patterns • Running continuously for almost 1 year • during snow, wind, rain, ... • one can observe patterns over space and over time • need a system to detect automatically Forest of Sensors

  21. Classifying objects • Simple classifiers based on feature selection Example -- people vs. cars using aspect ratio and size Forest of Sensors

  22. A B match C QUERY Template DATABASE IMAGES IN THE QUERY-CLASS Flexible template querying Forest of Sensors

  23. The Starting Point • Sinha’s Ratio Template Forest of Sensors

  24. Using this template Forest of Sensors

  25. TEMPLATE UNITS A B R1 R2 B C TEMPLATE A(blue) > A(green) A(blue) > A(red) A R1: A(red) < B(red) A(green) < B(green) A(lum) < B(lum) R1 B R2: B(red) > C(red) B(green) > C (green) B(blue) > C(blue) B(lum) > C(lum) R2 C Build a Spatial Template Forest of Sensors

  26. Example • Recognition Forest of Sensors

  27. Example • Detection Forest of Sensors

  28. Example • Classification Forest of Sensors

  29. Example Results Forest of Sensors

  30. Classifying objects • Using flexible templates to detect vehicles Forest of Sensors

  31. Classifying activities • Quantize state space of continuous observations by fitting Gaussians • map sequence of observables to sequence of labels • compute co-occurrence of labels over sequences -- defines joint probability • cluster by E/M methods to find underlying probability distributions Forest of Sensors

  32. Automatic activity classification Forest of Sensors

  33. Multi-camera version Forest of Sensors

  34. Mapping patterns to maps Forest of Sensors

  35. Multi-camera coordination Forest of Sensors

  36. Finding unusual events Forest of Sensors

  37. Finding similar events Forest of Sensors

  38. Future plans • integration of pieces • site models from dynamic tracking • activity classification from multiple views • variations on classification methods • vocabulary of activities and interactions • interactions of activity tracks • fine scale actions Forest of Sensors

More Related