Hierarchical Method for Foreground DetectionUsing Codebook Model

1. Hierarchical Method for Foreground DetectionUsing Codebook Model Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia, Min-Hsiung Shih, and Chih-Sheng Hsu IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 21, NO. 6, JUNE 2011

2. Outline • Background Model Construction • Block-Based Background Subtraction • Pixel-Based Background Subtraction • Hierarchical Foreground Detection • Background Models Updating with theShort-Term Information Models • Experimental Results

3. Background Model Construction • This method involves two types of codebooks(CBs), block-based and pixel-based CBs. • The modeling of two CBs is similar to the former CB[14] [14] K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis, “Real-time foreground-background segmentation using codebook model,” Real- Time Imaging, vol. 11, no. 3, pp. 172–185, Jun. 2005.

4. Background Model Construction

5. Background Model Construction • There are two different time intervals for training (xt). • (1 ≤ t ≤ T) and (t > T) for training the background models and foreground detection. • The updating algorithms are separated into two parts for different time zones.

6. The Features Used in Block-Based Background Subtraction • A frame xt of size P x Q is divied into multiple non-overlapped blocks of size M x N. • The former block truncation coding(BTC) reduce the frame into two means,high-mean and low-mean. • In this paper ,we have four means to represent a frame, high-top mean (μht ), high-bottom mean (μhb), low-top mean (μlt ), and low-bottom mean (μlb).

7. The Features Used in Block-Based Background Subtraction

8. The Features Used in Block-Based Background Subtraction • Each means have three colors(RGB),so each codebook have 12 dimensions.

9. Updating Block-Based Background Models (CBs) in the Training Phase • a specific block can be represented as a vector Vb = {vbt|1 ≤ t ≤ T }. • A CB for a block can be represented as C = {ci|1 ≤ i ≤ L}, consisting of L codewords • An additional weight wiis geared for indicating the importance of the ith codeword. • Codebook size is (P/M)x(Q/N)

10. Updating Block-Based Background Models (CBs) in the Training Phase

11. Updating Block-Based Background Models (CBs) in the Training Phase

12. Updating Pixel-Based Background Models (CBs) in theTraining Phase • The same as block-based method. • Codebook size is P x Q. • Each codebook is 3 dimensions (RGB)

13. Hierarchical Foreground Detection • After the background models training as indicated before the time point T, the two CBs are applied to the proposed hierarchical foreground detection. • The foreground is obtained by background subtraction.

14. Foreground Detection with the Block-Based CB • the input vector (vbt) extracted from a block is compared with the ith block-based codeword (ci) to determine whether a match is found • When a vbt is classified as background, the corresponding block is also used to update the pixel-based CB.

15. Foreground Detection with the Pixel-Based CB • This subsection introduces how to classify a pixel in a block to foreground or background. • The foregrounds are classified into one true foreground and two fake foregrounds (shadow and highlight).

16. Foreground Detection with the Pixel-Based CB

17. Foreground Detection with the Pixel-Based CB

18. Background Models Updating with theShort-Term Information Models • an additional variable timeics is involved to store the updated time for estimating whether the corresponding ith codeword (cis ) has been updated for a specific period or not. • If the duration is longer than a predefined parameter Dsdelete, the corresponding cis is simply a temporary foreground.

19. Background Models Updating with theShort-Term Information Models • When cis , is favor to strong stationary ( wics ≥ Dadd), the short-term information model can be considered as a part of the true background model. • This additional value is employed for filtering out ci which meets the states of eventually moving as foregrounds with the predefined parameter Ddelete.

20. Experimental Results • λ = 5 for block-based , λ = 6 for pixel-based, η = 0.7, θcolor = 3,β = 1.15 ,γ = 0.72 ,Dupdate = 3, and α = 0.05, Dadd = 100, Dsdelete = 200, and Ddelete = 200

21. Experimental Results • [9]MOG • [5]color model • [11][25] hierarchical MOG • [14]CB [9] C. Stauffer and W. E. L. Grimson, “Adaptive background mixture models for real-time tracking,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2. Jun. 1999, pp. 246–252. [5] R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detection moving objects, ghosts, and shadows in video streams,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 10, pp. 1337–1342, Oct. 2003. [11] Y.-T. Chen, C.-S. Chen, C.-R. Huang, and Y.-P. Hung, “Efficient hierarchical method for background subtraction,” Pattern Recognit., vol. 40, no. 10, pp. 2706–2715, Oct. 2007. [25] C.-C. Chiu, M.-Y. Ku, and L.-W. Liang, “A robust object segmentation system using a probability-based background extraction algorithm,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 4, pp. 518–528, Apr. 2010. [14] K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis, “Real-time foreground-background segmentation using codebook model,” Real- Time Imaging, vol. 11, no. 3, pp. 172–185, Jun. 2005.

22. C)MOG d)Color model e)CB f)g) hierarchical MOG

23. C)MOG d)Color model e)CB f)g) hierarchical MOG

24. C)MOG d)Color model e)CB f)g) hierarchical MOG

26. Experimental Results

27. Experimental Results

28. Experimental Results

29. Conclusion • The block-based stage can enjoy high speed processing speed and detect most of the foreground without reducing TP rate. • Pixel-based stage can further improve the precision of the detected foreground object with reducing FP rate. • Short-term information is employed to improve background updating