1 / 33

MULTI-OMNIDIRECTIONAL CAMERAS FOR AMBIENT INTELLIGENCE

MULTI-OMNIDIRECTIONAL CAMERAS FOR AMBIENT INTELLIGENCE. Bar ış Evrim Demiröz, Albert Ali Salah, Lale Akarun. 19.04.2012 Sinyal İşleme ve İletişim Uygulamaları Kurultayı. Omnidirectional Camera. Camera with large field of view ~180° >180° ~360 °. Omnidirectional Camera.

tamar
Download Presentation

MULTI-OMNIDIRECTIONAL CAMERAS FOR AMBIENT INTELLIGENCE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MULTI-OMNIDIRECTIONALCAMERAS FOR AMBIENTINTELLIGENCE Barış Evrim Demiröz, Albert Ali Salah, Lale Akarun 19.04.2012 Sinyal İşleme ve İletişim Uygulamaları Kurultayı

  2. Omnidirectional Camera • Camera with large field of view • ~180° • >180° • ~360 ° Multi-Omnidirectional Cameras for Ambient Intelligence

  3. Omnidirectional Camera • Camera with large field of view • ~180° • >180° • ~360 ° Multi-Omnidirectional Cameras for Ambient Intelligence

  4. Omnidirectional Cameras • Multiple images • PTZ, multiple cameras • Dioptric (lens) • Fisheye • Catadioptric (lens + mirror) Multi-Omnidirectional Cameras for Ambient Intelligence

  5. Omnidirectional Cameras • Throwable Panoramic Ball Camera Photo: Jonas Pfeil Multi-Omnidirectional Cameras for Ambient Intelligence

  6. Omnidirectional Cameras • Pan Tilt Zoom (PTZ) camera Photo: Wikimedia Commons Multi-Omnidirectional Cameras for Ambient Intelligence

  7. Omnidirectional Cameras • Fisheye lens Photo: Wikimedia Commons Multi-Omnidirectional Cameras for Ambient Intelligence

  8. Omnidirectional Cameras • Catadioptric (lens + mirror) B. Micusik and T. Pajdla, "Estimation of omnidirectional camera model from epipolar geometry" 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings., pp. I-485-I-490, 2003. Multi-Omnidirectional Cameras for Ambient Intelligence

  9. Catadioptric/Dioptric cameras • Pros/cons • 1 image  • Low resolution near boundaries  • Ideal for small spaces, surveillance Title of the presentation

  10. Dataset Multi-Omnidirectional Cameras for Ambient Intelligence

  11. Sample from dataset Multi-Omnidirectional Cameras for Ambient Intelligence

  12. The room Pencereler Masa 1 YanKamera Masa 2 Sandalye 1 Sandalye 2 Üst Kamera Lavabo Kapı Multi-Omnidirectional Cameras for Ambient Intelligence

  13. Cameras used • Oncam IPC, IPT • 5 Megapixel • No moving parts Multi-Omnidirectional Cameras for Ambient Intelligence

  14. Scenarios performed • Scenario #1 • Single person • 6 Actions • Sitting • Walking • Drinking • Washing hands • Fainting • Opening/Closing door • 5 performers (10 videos total) Multi-Omnidirectional Cameras for Ambient Intelligence

  15. Scenarios performed • Scenario #2 • 3 people • 5 Actions • Sitting • Walking • Standing • Handshaking • Interested in object • 5 performers (36 videos intotal by exchanging roles) Multi-Omnidirectional Cameras for Ambient Intelligence

  16. Video properties • 640x480 frame size • Bandwidth limitation • ~8 fps • MJPEG format Multi-Omnidirectional Cameras for Ambient Intelligence

  17. Annotations C • Annotation tool usedvatic* • Annotations • Bounding box of subject • State (action) of subject A B * C. Vondrick, D. Ramanan, and D. Patterson, “Efficiently Scaling Up Video Annotation with Crowdsourced Marketplaces,” ECCV 2010, pp. 610–623, 2010. Multi-Omnidirectional Cameras for Ambient Intelligence

  18. Illumination conditions • Intensity standard deviation Multi-Omnidirectional Cameras for Ambient Intelligence

  19. Occlusion • Pseudo occlusion: Multi-Omnidirectional Cameras for Ambient Intelligence

  20. Occlusion Multi-Omnidirectional Cameras for Ambient Intelligence

  21. How to evaluate tracker performance? • Multiple Object Tracking Precision (MOTP)* • Multiple Object Tracking Accuracy (MOTA)* * K. Bernardin and R. Stiefelhagen, “Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics,” EURASIP Journal on Image and Video Processing, vol. 2008, pp. 1–10, 2008. Multi-Omnidirectional Cameras for Ambient Intelligence

  22. How to evaluate tracker performance? o: object h: hypothesis d: distance btw o: and h c: match count m: misses fp: false positives mme: mismatches t: frame no k: video no i: object no Multi-Omnidirectional Cameras for Ambient Intelligence

  23. Released to public • Videos • Annotations • Performance Measurer (in Java) • Camera calibrations • Source codes Multi-Omnidirectional Cameras for Ambient Intelligence

  24. Base tracker Multi-Omnidirectional Cameras for Ambient Intelligence

  25. Outline System Components Multi-Omnidirectional Cameras for Ambient Intelligence

  26. Background/Foreground Segmentation • Mixture of Gaussians • Shadow removal Multi-Omnidirectional Cameras for Ambient Intelligence

  27. Threshold the areas filter small contours Blob formation • Morphological close • maintain connectivity × × Multi-Omnidirectional Cameras for Ambient Intelligence

  28. FAST*corner detection Feature Point Detection andDescriptor Extraction • BRIEF features • extended for opponent color space • *Features from Accelerated Segment Test Multi-Omnidirectional Cameras for Ambient Intelligence

  29. Blob Matching and Tracking Frame Blob Blob Blob … Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint … … … Keypoint Keypoint Keypoint Multi-Omnidirectional Cameras for Ambient Intelligence

  30. Blob Matching and Tracking Previous n frames Frame NEW FRAME Frame Frame … Blob Blob Blob Blob Blob Blob Blob Blob Blob Blob Blob Blob … … … … Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint … … … … … … … … … … … … Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Keypoint Multi-Omnidirectional Cameras for Ambient Intelligence

  31. Blob Matching and Tracking • Optimal assignment problem • Hungarian algorithm Multi-Omnidirectional Cameras for Ambient Intelligence

  32. Results Multi-Omnidirectional Cameras for Ambient Intelligence

  33. Questions?

More Related