Real-Time Detection of Multiple Moving Objects in Complex Image Sequences

Real-Time Detection of Multiple Moving Objects in Complex Image Sequences International journal of imaging systems and technology 10,305-317,1999 Speaker: M. Q. Jing

Outline • Introduction • Change Detection Algorithms • Simple difference method • Derivative model • Shading model • LIG model • Binary Statistical Morphology • Multiple object detection • Result & Conclusion

Introduction(1) • Change detection (CD) is an process for many machine vision systes • Track moving object • Analyze the traffic flow • Guide autonomous vehicles • Task: finding significant differences between images • Difference may be caused by • Motion of the camera • Entrance or exit of an object from the scene • Change in illuminations & Bad environmental conditions

Introduction(2) • CD is performed at pixel, edge or higher feature levels. • Feature levels require more computational effort • CD take two digital images as input. The output is a binary image

Change Detection Algorithms --Simple Difference(SD) method • Input: Two N x N digitized images It(x,y), It-1(x,y) • Output: Binary image Bt(x,y) • Dt(x,y)=| It(x,y), It-1(x,y) | Fig 2: a,b

Change Detection Algorithms --Derivative Model (DM) Fig 2: c,d

Change Detection Algorithms --Shading Model (SM) • The intensity Ip at point (x,y) Ip= Ii * Sp , where Ii = illumination Sp = shading coefficient • Calcuate the varance of the intensity Ip/Ip-1 0 => no changed • Fig2 (e,f)

Change Detection Algorithms -- Local intensity gradient(LIG) method • Pixels at the location having a high gray-level gradient form a part of an object • Nearly pixles having similar gray levels will be also part of the same object. • Large negative gradients at pixels belonging to object boundaries

Change Detection Algorithms -- Local intensity gradient(LIG) method • 1: I -> G(I) • 2: G(I) -> m x m blocks • Limit the effects of illumination changing • 3: for each block, calcuating the mean & variance • 4: regional means and variances are smoothed using the neighboring regions • 5. then interpolated to fill an m x m again • 6 compare with background and foregound Fig 2: I,l

Binary statistical morphology • Mathematical morphology(MM) describe images as sets and image-porcessing operators. • The main drawback of such operators is the high sensitivity to noise • Statistical morphology provides a noise-robust probabilistic generalization of MM.

Binary statistical morphology • SM interprets erosion and dilation as probabilistic lta and wta operator • Statistical erosion & statistical dilation = minimum variance estimators of the ouput distribution, P(H|I)

Binary statistical morphology • SD computed at an image location m as: • Binary Statistical Dilation(BSD) is obtained by threshold SD at value

In similar way, Binary Statistical Erosion(BSE)

Multiple object detection It(x,y) BCKt(x,y) Simple Difference Dt(x,y) First Level Ki bata Binary Statistical Erosion with TSE Fig 4: a,b Bt1(x,y)

Multiple object detection -- The first step • Temporal structuring element (TSE) = 3x3 mask • A low ki value should be selected to maximize the probability of detecting moving object points. • The first level is represent the pixel where are object or background.

Multiple object detection -- The Second step (noise reduction) Bt1(x,y) Binary Statistical Erosion with TSE1 Zt(x,y) Set filtering Yt(x,y) Binary Statistical dilation with TSE2 Bt2(x,y)

Multiple object detection -- The Second step (noise reduction) • (a) elimnate isolated points • (b) (c) taking into account the compactness constraint by favoring dense agglomerate of changed pixels. Fig 4. c,d

Multiple object detection -- The Third step (blob tracking) • Blob tracking is a difficult task • Many blobs may appear or vanish in successive frames • Blobs may be occluded by overlapping of other blobs • Problems are caused by the high dimensionality of the set of possible combinations • See Fig 4. e

Multiple object detection -- The Third step (blob tracking) • Step1: a blob is not expected to be too far from where it was in the previous frame. • Choose a mask,(2u*cosa+1 ) in x direction and 2u*sina+1 in y direction • The mask is centerred in I’s barycenter • The blobs in I+1 frame whose barycenter fall within this mask are selected.

Multiple object detection -- The Third step (blob tracking) • Step2: Let CBt+1(bi)= set of blobs on the frame t+1 which are candidate to match with the blob bi detected on the frame t => 給定一blob bi 選擇在下一個frame,落在 bi範圍內的blob 集合

The cost function * 當 blob 大小不變時,cost function值最小

Result-- The detection performance • Tests on Outdoor Scenes • Fig 5. • The detection performance • P=(correctly identified #) / (the method detected #) 即在此方法所有detected的blob中,有幾個是正確的. • R=(correct detected #)/ (original image #) • G={(correctly identified #) -(false detected #)}/ (original image #) Fig 7.

Result-- The Illumination conditions Fig 8;

Result-- Overflow error & gap error • r(i)=1, if i belong to the • detected blob • rgr(i) =1, if i belong to • the manually extracted • blobs Fig 9

Result-- Test on Noisy Images • A random noise with uniform distrubution was added to some frames of the image sequence. • Fig 10(a) • SD and SM method could not be applied to very noisy images. • Fig 10(b) is proposed method result. • Fig 11(a) is different noise level result

Result-- Time performance • The time performance of the different CD methods were compared. • SD and proposed method are 10 times fast than the SM and DM • 15 times faster than the LIG method

Conclusion • If the system works outdoors and in badly illuminated enviroments, => LIG , the proposed method • If real-time is required, => the proposed method

Fig1: Original

SD method DM method Fig 2

Fig 2: SM result

Fig 2: LIG Result

Fig4: the proposed result

Fig5: more complex

Fig 6: a The proposed

Fig6:b SD result

Fig 6: c SM result

Fig 6: d LIG result

Fig 7: a,b

Fig 7: c

Fig 8: illum result

Fig 9: Overflow err

Fig 10 Noise

Real-Time Detection of Multiple Moving Objects in Complex Image Sequences