Occlusion Tracking Using Logic Models

1. Occlusion Tracking Using Logic Models James H. von Brecht Sheshadri R. Thiruvenkadam Tony F. Chan This research supported by ONR grant N00014-06-1-0345, NSF grants DMS-0601395, DMS-0610079 and ARO MURI grant 50363-MA-MUR

2. Tracking Under Occlusions • Objective: Video tracking of objects under occlusions and in complex backgrounds. • Assumptions: • Availability of prior shape. • Affine motion. • Approach: • Use prior shape information to locate and segment object in current frame. • Incorporate shape prior intelligently using Logic Models SIP 2007

3. Tracking Under Occlusions When occlusions are present, image data alone can be unreliable, and is not sufficient to detect the object of interest. Must incorporate some prior knowledge about the object for successful detection when occluded. Prior Shape Motion Model Other energy minimization approaches incorporate prior shape as an additive constraint. Energy = IMAGE + SHAPE Additive constraints depend upon a notion of scale, and hence are sensitive to undesirable local minima issues. Using Logic Models, we couple prior shape and image data into one energy term Energy = IMAGE/SHAPE SIP 2007

4. Logic Models • Given two (registered) images, we can combine the objects in each image according to a pre-selected logical operation (AND/OR) into one segmentation for both images: SIP 2007

5. Segmentation Energy • Segmentation is achieved by minimizing a functional of the familiar form: • The image terms and vary depending upon the choice of Logic Model, and combine the images and in a particular way in order to achieve the desired segmentation. • is the Heaviside function, is the level-set function used for segmentation, and is some regularization term. For example, • Note that we must minimize a different energy if our choice of logic model were to change (must automate this choice for tracking). SIP 2007

6. The image terms extend the Chan-Vese segmentation energy to handle multiple images: Depending upon the desired Logic Model, we have: Segmentation Energy SIP 2007

7. Incorporating Prior Shape • By applying the correct Logic Model, we can utilize available prior shape information to locate and segment an occluded object. • Shape prior is represented as a binary image – i.e. • Correct logical combination depends upon the similarity of the object’s intensity and the occlusion’s intensity. • Similar => AND • Dissimilar => OR Original Image/ Shape Prior AND Model OR Model SIP 2007

8. Incorporating Prior Shape • Knowing the correct Logic Model to apply allows us to avoid segmentation local minima which can occur by simply introducing shape additively: • Additionally, using the opposite Logic Model (OR in the above example) by mistake results in an incorrect segmentation – must determine correct model automatically! AND Model Chan-Vese + Shape OR Model SIP 2007

9. Automatic Use of Logic Models • Given two segmentation results, from OR and from AND, the correct model will deviate least from the shape prior, and thus minimizes the quantity: • To enforce this in practice,we first compute and , check the above term for each, then choose the minimizer as the segmentation result. SIP 2007

10. Tracking With Logic Models • We assume the object in the current frame of a video sequence can be represented as an affine transformation of the shape prior modulo occlusion. • Thus, we wish to minimize our segmentation energy both with respect to the active contour and the transformation - SIP 2007

11. Tracking with Logic Models • General algorithm (sequential segmentation)- • Use the resultfrom the previous frame to generate an initial guess for the current frame . • Compute via gradient descent on the AND energy with as initial data. • Compute via gradient descent on the OR energy with as initial data. • Check for each and select as whichever gives a minimum. SIP 2007

12. Limitations and Future Work • Naïve approach to tracking: • No explicit motion model. • Can be sensitive to registration local minima when image quality is poor. • Sequential segmentation can fail if object motion is too great between frames. • Cannot handle total occlusions. • Computationally expensive. • Future work: • Incorporate logical framework into more sophisticated Bayesian-based tracking techniques, e.g. Particle Filtering, Kalman Filtering. • Goal: robust, real-time tracking. SIP 2007

13. Sample Results SIP 2007

14. Sample Results SIP 2007