20 12

1. 2012 Pedestrian Re-identification by Layered Pseudo-3D Pictorial Model Matching YuanluXu, SYSU, China merayxu@gmail.com • 2012.8.2

2. Episode 1 Difficulties, Empirical Studies, Intuitions, and Framework

3. Problem Matching

4. Difficulties Non-overlapping Camera Views Irrelevant negative samples, difficult to train classifiers

5. Difficulties View Changes

6. Difficulties Occlusions Carried objects block the appearance

7. Difficulties Illumination Changes Need illumination-invariant features or light-amending process

8. Difficulties Large Intra-class Variations & Limited Samples for Learning

9. Difficulties

10. Difficulties Poses in VIPeR

11. Difficulties Poses in VIPeR

12. Difficulties Occlusions in VIPeR

13. Difficulties Poses in VIPeR

14. Difficulties Occlusions in VIPeR

15. Difficulties Occlusions in VIPeR

16. Difficulties Occlusions in VIPeR

17. Difficulties View Difference in VIPeR

18. Framework Pedestrian Modeling Matching Inference Part Detection Computing Part Signature Annotated Parts Prior 3D-Pose Templates Pose, View Estimation Ranking by Coarse Model Comparison Learned Part Classifiers Matching Results Occluded Parts Recovery Pedestrian Image Re-ranking by Layered Graph Matching Pseudo-3D Pictorial Model

19. Episode 2 PedestrianModeling

20. Framework Pedestrian Modeling Learned Part Classifiers Part Detection Annotated Parts Prior 3D-Pose Templates Pedestrian Image Pose, View Estimation Learned Part Classifiers Prior 3D-Pose Templates Pseudo-3D Pictorial Model Occluded Parts Recovery Model Learning Model Inference

21. Pictorial model A part-based appearance model to represent non-rigid objects. D.S. Cheng, M. Cristani, et al., "Custom pictorial structures for reidentification,“ BMVC 2011

22. Pictorial model Characteristics: The body is decomposed into a set of parts, their configuration . Each part , position, orientation and scale, respectively. M. Andriluka et al, “Pictorial Structures Revisited: People Detection and Articulated Pose Estimation”, CVPR 09

23. Pictorial model Given an image of a pedestrian , the posterior of is modeled as : pictorial model prior, formed as a directed tree. : the likelihood of the image given a pictorial model, by discriminative appearance model

24. Pictorial model The body model is decomposed into N = 6 part: chest, head, thighs and legs

25. Pictorial model The kinematic dependencies between parts is represented by a directed tree: denotes the set of all directed edges in the kinematic tree and assign to be the root node (torso).

26. Pictorial model The prior for the root part configuration is simply assumed to be uniform. The part relations are modeled using Gaussian distributions. Although the part relations are intuitively not Gaussian, we can transform it to a different space.

27. Pictorial model To model , we transform the part configuration into the coordinate system of the joint between the two parts using the transformation:

28. Pictorial model the part relation is modeled as a Gaussian in the transformed space:

29. Pictorial model • Estimate the likelihood : • Different part evidence maps are conditionally independent given the configuration • The part map for part only depends on its own configuration

30. Pictorial model Estimate the likelihood : The likelihood simplifies as Justifiable as long as parts do not occlude each other significantly. Constraints!

31. Pictorial model Train an AdaBoostclassifier with simple decision stumps:

32. Pictorial model To integrate the discriminative classifiers into the generative probabilistic framework described above The posterior over the configuration of parts factorizes as:

33. Episode 3 Matching Inference

34. Framework Matching Inference Computing Part Signature Ranking by Coarse Model Comparison Pseudo-3D Pictorial Model Matching Results Re-ranking by Layered Graph Matching

35. Part Signature Color Histograms: HSV characterization, where hue and saturation are jointly taken by a 2D histogram to retain much of the chromatic specificity. Maximally Stable Color Region (MSCR): detects a set of blob regions by looking at successive steps of an agglomerative clustering of image pixels. M. Farenzena, L. Bazzani, A. Perina, V. Murino, and M. Cristani. Person Re-Identification by Symmetry-Driven Accumulation of Local Features. CVPR, 2010. Source Image MSCR

36. Part Signature Distance Measures Given two part signatures , the distance between and is defined as where is the Bhattacharyya distance,

37. Coarse matching Distance Measures is defined as measures the Euclidean distance between MSCR centroids, measures the Euclidean distance between their mean color.

38. Coarse ranking For each pseudo-3D pictorial model, concatenating each part and normalizing them into a single feature vector. To represent parts with different size and depth, multiply the part signatures with a set of weights (large, front parts having large weights and vice versa), we get a coarse model signature . By calculating the distance model signatures, we get a coarse ranking.

39. Fine re-ranking by layered graph matching To further improve the matching results, a composite parts clustering approach is employed. Given a query pedestrian , to find the best match , define a candidacy graph . By calculating the distance model signatures, we get a coarse ranking. Liang Lin, Xiaobai Liu, and Song-Chun Zhu, "Layered Graph Matching with Composite Cluster Sampling", TPAMI, 2010

40. Layered graph matching Input: two graphs Output: layered matching configuration

41. Layered graph matching Input: source graph and target graph Output: layered matching configuration 1. Construct candidate graph. 2. Sample composite clusters. a. Generate a composite cluster. b. Re-assign color to the composite cluster. c. Convert to a new state.

42. Layered graph matching Construct candidate graph - vertices 1. Start with a linelet, find the set of matching candidates. 2. Grow , reduce the matching candidates. 3. Repeat 1 and 2 until only less than k matching candidates.

43. Layered graph matching Construct candidate graph - vertices Let a matching pair be a vertices in the candidate graph.

44. Layered graph matching Construct candidate graph - edges Establish the negative and positive edges and calculate their edge probabilities between vertices.

45. Layered graph matching Construct candidate graph - edges as a negative edge in two cases: 1. two candidates are mutually exclusive: . 2. the two candidates overlap: .

46. Layered graph matching Construct candidate graph - edges as a positive edge: the similarity transformation to align and .

47. Layered graph matching

48. Layered graph matching Generate a composite cluster CCP: Candidates connected by the positive “on” edges form a CCP. (blue lines) Composite Cluster: A few CCPs connected by negative “on” edges form a composite cluster.(red lines)

49. Layered graph matching Generate a composite cluster

50. Layered graph matching Re-assign color • Primitives connected by positive edges receive the same color. The ones connected by negative edges receive different color.