What Are We Tracking : A Unified Approach of Tracking and Recognition

1. What Are We Tracking:A Unified Approach ofTracking and Recognition JialueFan, XiaohuiShen, Student Member, IEEE, and Ying Wu, Senior Member, IEEE

2. Outline • Introduction • Method a. overview description b. tracking procedure c. video-based object recognition • Experiment result • Conclusion

3. Introduction • One notable shortcoming of online models is that they are constructed and updated based on the previous appearance of the target without much semantic understanding.

4. Introduction • Once an object is discovered and tracked, the tracking results are continuously fed forward to the upper-level video-based recognition scheme. • Based on the feedback from the recognition results, different off-line models dedicated to specific categories are adaptively selected, and the location of the tracked object in the next frame is determined by integrated optimization of these selected detectors and the tracking evidence.

5. Method • Overview Description The target state : Xt. The target category : Ct. The input image at time t : It. The target measurement at time t : Zt = It(Xt). The online target model : . The object category : N different classes. The is the other class. Each object class is associated with a specific offline model

6. Method • They employ a two-step EM-like method here: at time t, we first estimate xt(i.e., “tracking”), and then estimate ctbased on the new tracking result zt= It (xt ) (i.e., “recognition”). • Tracking procedure Based on online target model ,the offine model selected by the previous recognition result ,and the current input image It.

7. Method • In a Bayesian perpective, they have • By defining the energy term E = - ln p,

8. Method • is the energy term related to tracking. • is the energy term related to detection. • Tracking term :

9. Method • Detection term :

10. Method • Video-based object recognition They instead find the optimal sequence {c1, . . . , ct} given the measurement z1, . . . , zt, which is indeed the video-based object recognition. • Denote by ct= {c1, . . . , ct}, zt= {z1, . . . , zt}, they have

11. Experiments • σsis the (time varying) confidence score corresponding to the energy term ws, which can be obtained by: • Linear SVM classifier : and is the SVM score.

12. Experiments

13. Experiments

14. Experiments

15. Experiments

16. Conclusion • The limitations : (1) The ambiguity of the tracking problem increases, as the number of object categories increases. (2) The wrong recognition result probably leads to error propagation. (3) The current design may not be appropriate for some tracking dataset, due to data type inconsistency.

17. Conclusion • As a mid-level task, visual tracking plays an important role for high-level semantic understanding or video analysis. Meanwhile the high-level understanding (e.g., object recognition) should feed back some guidance for low-level tracking. • Motivated by this, we propose a unified approach to object tracking and recognition.

18. The End