CVPR2013 Poster

1. Harry Potter’s Marauder’s Map: Localizing and Tracking MultiplePersons-of-Interest by Nonnegative Discretization CVPR2013 Poster

2. Outline • 1.Introduction • 2. Methodology 2.1 Notations 2.2 Manifold Learning in Appearance Space with Spatio Temporal Constraints 2.3 Trajectory Inference • 3. Experiments • 4. Conclusions

3. 1.Introduction • Harry Potter’s Marauder’s Map First work is used color person detection face recognition non-background detection ․Important cues of typical tracking and localization Because Didn’t deal with complex indoor scenes therefore Marauder’s Map algorithm ․tracking by detection paradigm[14][1] can be viewed as classification problem assign Class label Each person detection result ․use semi-supervised learning techniques

4. 1.Introduction [19] Y. Yang, F. Nie, D. Xu, J. Luo, Y. Zhuang, and Y. Pan. A multimedia retrieval framework based on semi-supervised ranking and relevance feedback. In IEEE TPAMI, 2012 ․manifold assumption manifold structure Inspired by [19], directly computing the affinity matrix statistical approach ․mutual exclusion constraint : one person detection result to be associated with only one person ․We perform nonnegative discretization: Partitionsthe detected data points into non-overlapping groups (mutual exclusion and the manifold assumption aresatisfied simultaneously)

5. 1.Introduction

6. 2. Methodology from different camera views ․Input set of person detection results at each time instant camera calibration Be mapped to a common 3D coordinate system by ground plane parameters be described by the color histogram ․Our algorithm’s main task : predict a label for each person detection result manifold learning in appearance space with spatio-temporal constraints trajectory inference by nonnegative discretization

7. 2.1 Notations c n : n data points generated by the person detector xi : color histogram for the i-th data point pi : 3D location of the i-th data point 0 0 1 0 0 ……0 1 0 0 0 0 ……0 0 0 1 0 0 ……0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0 1 0 0 0 ……0 f1 ti: video frame of the i-th data point f2 c: number of individuals to be tracked F :indicator matrix fn fi : label indicator vector for the i-th data point nj: number of data points in the j-th class

8. 2.1 Notations Tr(.) : trace operator |.|F : Frobenius norm of a matrix : is a column vector with all ones.

9. 2.2 Manifold Learning in Appearance Space with Spatio Temporal Constraints ․Nearest neighbor selection is a crucial step in learning manifold structure detail the method we used for nearest neighbor selection describe how this information is utilized in manifold learning. V : maximum possible velocity of a moving person

10. 2.2 Manifold Learning in Appearance Space with Spatio Temporal Constraints • Why this method of finding neighbors is robust ? Ans :Occlusions may cause the tracking target to be partially or completely occluded… ․T affects the tracker’s ability

11. 2.2 Manifold Learning in Appearance Space with Spatio Temporal Constraints ․we assume that the class labels of the i-th data point and its neighbors can be predicted by a local function gi(.) ․Following [19], we adopt the linear regression model as the local prediction function bias term local projection matrix ․ A local function only corresponds to a smallsegment of one trajectory. To exploit the structure of all thetrajectories in the entire video sequence, we minimize theprediction error of all the local models gi

12. 2.2 Manifold Learning in Appearance Space with Spatio Temporal Constraints :the regularization term on Wi: :color histograms of the points in :prediction scores of the points in

13. 2.2 Manifold Learning in Appearance Space with Spatio Temporal Constraints L : Laplacianmatrix which encodes all the neighborhood information. : Centering matrix

14. 2.3 Trajectory Inference NP-complete problem ․Equation 5 is a combinatorial problem ․Equation 2, F is orthogonal by definition all the elements is nonnegative by definition Orthogonal constraint ․According to [20] are satisfied for a matrix Nonnegative constraint In other words, we exploit the orthogonal and nonnegative constraints to perform discretization of F

15. 2.3 Trajectory Inference The update rule: ․The main problem of this optimization approach algorithm may converge to a severe local optima ※ So initialization is important therefore Resort to weak supervision to get a more stable initialization

16. 2.3 Trajectory Inference i-th data point is a ground truth positive for any class U : diagonal matrix 1 else ie. : label matrix ․global optimal solution for Equation 10 (as the initial value) ․final step + infer the trajectory for each individual Detections of each individual Viterbi search ․our formulation can naturally handle template updates

17. 3. Experiments • Data sets [3] two other trackers: 3D color particle filter (CPF)[14] [3]:Unsupervised method which only relies on background information CPF[14]:use particle filter Our method : use person detection result[6][8] and Probabilistic Occupancy Map [7] PETS 2009 Caremedia

18. 3. Experiments • Result

19. 3. Experiments Demo video

20. 3. Experiments • Discussion: Advantages and Limitations ․Advantages 1.Use the manifold assumption to deal with slight color differences of the tracking target at different times. 2.we utilize PittPattface recognition ․Limitations 1.our objective function does not have a spatial locality constraint on a trajectory (i.e., an individual cannot be at multiple places at the same time) 2.optimization may converge to a severe local optima

21. 4. Conclusions • semi-supervised learning framework • The nonnegative discretization groups the data points into non-overlapping groups such that mutual exclusion and manifold assumption are satisfied simultaneously.