1 / 25

Towards a Learning Incident Detection System

Towards a Learning Incident Detection System. ICML 06 Workshop on Machine Learning for Surveillance and Event Detection June 29, 2006 Tomas Singliar Joint work with Dr. Milos Hauskrecht. Outline. Replace traffic engineers with ML algorithms for incident detection

zorion
Download Presentation

Towards a Learning Incident Detection System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Towards a Learning Incident Detection System ICML 06 Workshop on Machine Learning for Surveillance and Event Detection June 29, 2006 Tomas Singliar Joint work with Dr. Milos Hauskrecht

  2. Outline • Replace traffic engineers with ML algorithms for incident detection • Traffic data collection and quality • Why, who and for what purposes • Incident detection algorithms • Evaluation metrics • Individual feature performance • Sensor fusion with SVM • Noisy data problems • Attempts to model accident evolution with DBN • Conclusions and future work • Noisy data: Poor onset tagging and “bootstrap”

  3. Traffic data collection • Sensor network • Volumes • Speeds • Occupancy • Data aggregated over 5 minutes • Incidents • police • camera system

  4. Incident Annotation incident no incident incident

  5. Incident annotation • Incident labels not necessarily correct or timely • Do not correct timing (opportunity for more ML  )

  6. Incident detection algorithms, intuition • Incidents detected indirectly through caused congestion • Baseline: “California 2” algorithm: • If OCC(up) – OCC(down) > T1, next step • If [OCC(up) – OCC(down)]/ OCC(up) > T2, next step • If [OCC(up) – OCC(down)]/ OCC(down) > T3, possible accident • If previous condition persists for another time step, sound alarm • Hand-calibrated T1-T3 – very labor intensive • Why so few ML applications? • nontraditional data, anomaly detection – rare positives, common sense works well Occupancy spikes Occupancy falls

  7. Evaluation metrics • AMOC curve • Time to detection (TTD) vs False positive rate (FPR) • Don’t know when exactly incident happened • Maximal TTD (120min) • AU interesting region of C • Performance envelope • Detection rate (DR) vs FPR • Random gets over diagonal • Report ROC as a check • Sensitivity vs specificity • Low false positive region • 1 false alarm/day * 150 sensors

  8. Features • Sensor measurements • Temporal derivative • Spatial differences

  9. Features • Simple measurements: 3 per sensor, 6 total • Occupancy < threshold

  10. Temporal features • Capture abrupt changes • Occupancy spike – now minus previous time slice

  11. Spatial differences • “Discontinuities” in flow between sensor positions • Difference in speeds downstream - upstream

  12. Sensor fusion • Information in all simple detectors • How to combine their outputs? • Linear combination – SVM

  13. Baseline: California 2 • Hand-calibrated (+brute force) • Good low FAR performance, but poor detection rate

  14. SVM • Combines sensor measurements via a linear combination

  15. SVM • Spatial relations • Sensor measurements plus ratios and differences from the neighboring sensor

  16. SVM • Temporal derivatives • Sensor measurements plus differences and ratios to previous step

  17. Focus on low FAR • California better – persistency check

  18. A dynamic Naïve Bayes network • Problem: Incidents are recorded later than they occur • True state of highway is unobservable by sensors • Picture of incidents evolves in time • About 30 features: 3 readings up/down stream, differences, ratios to neighboring sensor, previous time point H H … H True hidden state On On On speed … … … … O1 O1 O1 Occupancy(t-5) I I I Incident observed

  19. H1 H2 H4 H3 A dynamic Naïve Bayes network • Evolution of an accident: • Normal traffic steady state • Accident happens, effects build up • Constricted steady state • Recovery • Model has 4 hidden states • Anchor hidden states to desired semantics: clamp p(I|H) • Raise alarm if p(H=acc_state|O) > threshold • Learned hidden state transition matrix: 0.9536 0.0332 0.0000 0.0133 0.0050 0.9577 0.0339 0.0034 0.0000 0.0882 0.9033 0.0084 0.0957 0.0000 0.0753 0.8290

  20. DNB Performance • Poor job at low FAR • Fairly insensitive to threshold

  21. Summary • Challenges to ML in traffic incident detection • Rare class – data sparsity, unequal misclassif cost • Incident annotations are noisy • Machine learning methods competitive though • SVM outperforms current practice • No manual tuning, readapts to data after changes • Lessons and surprises: • Richer feature sets do not help much • Neither does removing diurnal trends (?) • SVM has very stable performance • Dynamic Naïve Bayes weak

  22. Future work • Discriminate incident and benign congestion • Improve discriminative classification • SVM with nonlinearities (?) • Unequal misclassification cost models • Improve dynamical models • SVM handles time awkwardly – Dynamic Bayes Nets • Conditional random fields – discriminative + time • Improve Data • Bootstrap – use even a strawman to label incident start, learn from relabeled data (, iterate) • Supplemental materials available • http://www.cs.pitt.edu/~tomas/papers/icml06w/ • (AMOC curves that did not fit into the paper)

  23. Thank you • Questions? • Suggestions?

  24. SVM • California 2 measurements • Current and past occupancies

  25. DNB Performance

More Related