Video Segmentation Based on Image Change Detection for Surveillance Systems

1. Video Segmentation Based on Image Change Detection for Surveillance Systems Tung-Chien Chen (djchen@soe.ucsc.edu) EE 264: Image Processing and Reconstruction

2. Outline • Background • Image Change Detection • Video Surveillance Systems • Implementation • Block diagram and algorithm description • Demo • Comment

3. Image Change Detection • Differencing • Significance and hypothesis tests • Predictive models • Shading Models • Background Models • Change mask consistency and • post processing • ….. • Video surveillance • Remote sensing • Medical diagnosis and • treatment, • Civil infrastructure, • Underwater sensing, • Driver assistance systems • ……

4. In My Project • Differencing • Significance and hypothesis tests • Predictive models • Shading Models • Background Models • Change mask consistency and • post processing • ….. • Video surveillance • Remote sensing • Medical diagnosis and • treatment, • Civil infrastructure, • Underwater sensing, • Driver assistance systems • ……

5. Video Surveillance Systems • A technological tool that assists humans by providing an extended perception and reasoning capability about situations of interest that occur in the monitored environments

6. Video Surveillance Systems • A technological tool that assists humans by providing an extended perception and reasoning capability about situations of interest that occur in the monitored environments

7. Reference Paper • Efficient moving object Segmentation Algorithm Using Background Registration Technique S-Y Chien, S-Y Ma, and L-G Chen, IEEE Fellow @ National Taiwan University IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2002

8. Block Diagram of the Framework

9. Step1 – Differencing (1/2) • Frame difference and thresholding • Difference between current frame and previous frame • FD: frame difference • FDM: frame difference mask

10. Step1 – Differencing (2/2) • Background differencing and thresholding • Difference between current frame and background • BD: background difference • BDM: background difference mask

11. Step2 – Background Registration • According to FDM, pixels not moving for a long time are considered as reliable background pixels • SI: Stationary index • BI: Background indicator • BG: Background information

12. Example of Background Registration (1/2)

13. Example of Background Registration (2/2) • Include the function of background updating

14. Step2- Object Detection and Initial Object Mask Generation • Object detection • Produce “Initial object mask” (IOM)

15. Object Detection • Look up table for object detection

16. Step4- Post-processing • Two main parts in post-processing: • Noise region elimination and boundary smoothing • Connected component algorithm to eliminate small regions • Morphological close–open operations are applied to smooth the object boundary

17. Example of Post Operation Initial Object Mask After Connect Component After Close-open Operation Final Object

18. Results and Demo

19. Results and Demo

20. Result Demo

21. Comments (1/2) • For change detection based segmentation algorithm for surveillance system • Speed is high, but not robust • Performance degrade with the uncovered background situation, still object situation, light changing, shadow, and noise • Post-process can promote, but lose efficiency • Should automatically decide the thresholds • Some limitations: • strong change in light source, difference luminance between background foreground, camera moving/zoom/rotation, foreground object should move

22. Reference [1] R. J. Radke, S. Andra, O. Al-Kofahi, and B. Roysam “Image Change Detection Algorithms: A Systematic Survey,” IEEE Trans. Image Processing, vol. 14, no. 3, pp. 294–303, March. 2005. [2] R. Collins, A. Lipton, and T. Kanade, “Introduction to the special section on video surveillance,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 745–746, Aug. 2000. [3] C. Stauffer and W. E. L. Grimson, “Learning patterns of activity using real-time tracking,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 747–757, Aug. 2000. [4] C. R. Wren, A. Azarbayejani, T. Darrell, and A. Pentland, “Pfinder: Real-time tracking of the human body,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 780–785, Jul. 1997. [5] R. Mech and M. Wollborn, “A noise robust method for 2D shape estimation of moving objects in video sequences considering a moving camera,” Signal Process., vol. 66, 1998. [6] S.-Y.Ma, S.-Y. Chien, and L.-G. Chen, “An efficient moving object segmentation algorithm for MPEG-4 encoding systems,” in Proc. Int. Symp. Intelligent Signal Processing and Communication Systems 2000, 2000. [7] S. Y. Chien, S. Y. Ma, and L. G. Chen “Efficient Moving Object Segmentation Algorithm Using Background Registration Technique,” IEEE Trans. on circuits and system for video technology, vol. 12, no. 7, pp. 577–586, JULY. 2002. [8] R. M. Haralick and L. G. Shapiro, Computer and Robot Vision. Reading, MA: Addison-Wesley, 1992. [9] J. Serra, Image Analysis and Mathematical Morphology. London, U.K.: Academic, 1982.

23. Thanks for listening !! Questions ?