Codebook-based Background Subtraction (BGS) for Visual Surveillance

1. BG Modeling limages of raw and compressed input images Get an idea from the ‘t-test’ in statistics to obtain the difference between two means, here two colors in the transformed space <r,g,b> Codebook-based Background Subtraction (BGS) for Visual Surveillance Kyungnam Kim, Thanarat Horprasert, David Harwood, Larry Davis, Computer Vision Lab Key features of our BGS algorithm Color and Brightness Layered modeling and detection • resistance to artifactsof acquisition, digitization and compression. • capability of coping with local and global illumination changes. • adaptive and compressedbackground model that can capture structural background motion over a long period of time under limited memory. This allows us to encode moving backgrounds or multiple changing backgrounds. • unconstrained trainingthat allows moving foreground objects in the scene during the initial training period. • automatic parameter estimation • layered modeling and detectionallowing us to have multiple layers of background representing different depths • postprocessing, incorporating spatial shape information to obtain better silhouettes. Basic color distortion metric (having uncertainty in dark colors): The scene can change after initial training. These changes should update the background model. Additional model ‘cache’ - The values re-appearing for a certain amount of time enter the background model as non-permanent, short-term backgrounds. Input BG model Add brightness as a factor in computing color distortion: absorbed into BG Detection Result detected against both box and desk (a) The woman placed the box on the desk and then it has been absorbed into the background model as non-permanent. Then the purse is put in front of the box. It is detected against both the box and the desk. Background (BG) modeling Results on compressed image sequence and moving trees Input sequence BG Model (width) x (height) Codebooks (b) “time-indexed” detection with different color labeling: unloading two boxes from car • Each pixel _ 1 codebook (B) • Each B _M codewords (wm) • Each wm_ monochromatic images: 4-tuple <I, f, l,t> • _ color images: 8-tuple <r,g,I, Imin, Imax, f, l, t> (a) input image from MPEG sequence (b) zoomed image last access time frequency maximum negative run-length Temporal filtering: The true background, which includes both static pixels and moving background pixels, usually is quasi-periodic. (c) unattended suspicious objects Future work (c) single mode BGS method (d) our method • Background subtraction (BGS) • Clipping problem, Region-based approach, Temporal(motion) filtering, Parameter estimation for shadow & highlight, etc. • Region- and layer-based BGS • High-level analysis (for activity recognition) - Key frame segmentation • Rule-based analysis (expert system) • Decision and control by logic programming • Input image • including moving trees (c) our method with postprocessing (b) our method without postprocessing