20 12 - PowerPoint PPT Presentation

slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
20 12 PowerPoint Presentation
play fullscreen
1 / 91
20 12
Download Presentation
Download Presentation

20 12

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. 2012 Pedestrian Re-identification by Layered Pseudo-3D Pictorial Model Matching YuanluXu, SYSU, China merayxu@gmail.com • 2012.8.2

  2. Episode 1 Difficulties, Empirical Studies, Intuitions, and Framework

  3. Problem Matching

  4. Difficulties Non-overlapping Camera Views Irrelevant negative samples, difficult to train classifiers

  5. Difficulties View Changes

  6. Difficulties Occlusions Carried objects block the appearance

  7. Difficulties Illumination Changes Need illumination-invariant features or light-amending process

  8. Difficulties Large Intra-class Variations & Limited Samples for Learning

  9. Difficulties

  10. Difficulties Poses in VIPeR

  11. Difficulties Poses in VIPeR

  12. Difficulties Occlusions in VIPeR

  13. Difficulties Poses in VIPeR

  14. Difficulties Occlusions in VIPeR

  15. Difficulties Occlusions in VIPeR

  16. Difficulties Occlusions in VIPeR

  17. Difficulties View Difference in VIPeR

  18. Framework Pedestrian Modeling Matching Inference Part Detection Computing Part Signature Annotated Parts Prior 3D-Pose Templates Pose, View Estimation Ranking by Coarse Model Comparison Learned Part Classifiers Matching Results Occluded Parts Recovery Pedestrian Image Re-ranking by Layered Graph Matching Pseudo-3D Pictorial Model

  19. Episode 2 PedestrianModeling

  20. Framework Pedestrian Modeling Learned Part Classifiers Part Detection Annotated Parts Prior 3D-Pose Templates Pedestrian Image Pose, View Estimation Learned Part Classifiers Prior 3D-Pose Templates Pseudo-3D Pictorial Model Occluded Parts Recovery Model Learning Model Inference

  21. Pictorial model A part-based appearance model to represent non-rigid objects. D.S. Cheng, M. Cristani, et al., "Custom pictorial structures for reidentification,“ BMVC 2011

  22. Pictorial model Characteristics: The body is decomposed into a set of parts, their configuration . Each part , position, orientation and scale, respectively. M. Andriluka et al, “Pictorial Structures Revisited: People Detection and Articulated Pose Estimation”, CVPR 09

  23. Pictorial model Given an image of a pedestrian , the posterior of is modeled as : pictorial model prior, formed as a directed tree. : the likelihood of the image given a pictorial model, by discriminative appearance model

  24. Pictorial model The body model is decomposed into N = 6 part: chest, head, thighs and legs

  25. Pictorial model The kinematic dependencies between parts is represented by a directed tree: denotes the set of all directed edges in the kinematic tree and assign to be the root node (torso).

  26. Pictorial model The prior for the root part configuration is simply assumed to be uniform. The part relations are modeled using Gaussian distributions. Although the part relations are intuitively not Gaussian, we can transform it to a different space.

  27. Pictorial model To model , we transform the part configuration into the coordinate system of the joint between the two parts using the transformation:

  28. Pictorial model the part relation is modeled as a Gaussian in the transformed space:

  29. Pictorial model • Estimate the likelihood : • Different part evidence maps are conditionally independent given the configuration • The part map for part only depends on its own configuration

  30. Pictorial model Estimate the likelihood : The likelihood simplifies as Justifiable as long as parts do not occlude each other significantly. Constraints!

  31. Pictorial model Train an AdaBoostclassifier with simple decision stumps:

  32. Pictorial model To integrate the discriminative classifiers into the generative probabilistic framework described above The posterior over the configuration of parts factorizes as:

  33. Episode 3 Matching Inference

  34. Framework Matching Inference Computing Part Signature Ranking by Coarse Model Comparison Pseudo-3D Pictorial Model Matching Results Re-ranking by Layered Graph Matching

  35. Part Signature Color Histograms: HSV characterization, where hue and saturation are jointly taken by a 2D histogram to retain much of the chromatic specificity. Maximally Stable Color Region (MSCR): detects a set of blob regions by looking at successive steps of an agglomerative clustering of image pixels. M. Farenzena, L. Bazzani, A. Perina, V. Murino, and M. Cristani. Person Re-Identification by Symmetry-Driven Accumulation of Local Features. CVPR, 2010. Source Image MSCR

  36. Part Signature Distance Measures Given two part signatures , the distance between and is defined as where is the Bhattacharyya distance,

  37. Coarse matching Distance Measures is defined as measures the Euclidean distance between MSCR centroids, measures the Euclidean distance between their mean color.

  38. Coarse ranking For each pseudo-3D pictorial model, concatenating each part and normalizing them into a single feature vector. To represent parts with different size and depth, multiply the part signatures with a set of weights (large, front parts having large weights and vice versa), we get a coarse model signature . By calculating the distance model signatures, we get a coarse ranking.

  39. Fine re-ranking by layered graph matching To further improve the matching results, a composite parts clustering approach is employed. Given a query pedestrian , to find the best match , define a candidacy graph . By calculating the distance model signatures, we get a coarse ranking. Liang Lin, Xiaobai Liu, and Song-Chun Zhu, "Layered Graph Matching with Composite Cluster Sampling", TPAMI, 2010

  40. Layered graph matching Input: two graphs Output: layered matching configuration

  41. Layered graph matching Input: source graph and target graph Output: layered matching configuration 1. Construct candidate graph. 2. Sample composite clusters. a. Generate a composite cluster. b. Re-assign color to the composite cluster. c. Convert to a new state.

  42. Layered graph matching Construct candidate graph - vertices 1. Start with a linelet, find the set of matching candidates. 2. Grow , reduce the matching candidates. 3. Repeat 1 and 2 until only less than k matching candidates.

  43. Layered graph matching Construct candidate graph - vertices Let a matching pair be a vertices in the candidate graph.

  44. Layered graph matching Construct candidate graph - edges Establish the negative and positive edges and calculate their edge probabilities between vertices.

  45. Layered graph matching Construct candidate graph - edges as a negative edge in two cases: 1. two candidates are mutually exclusive: . 2. the two candidates overlap: .

  46. Layered graph matching Construct candidate graph - edges as a positive edge: the similarity transformation to align and .

  47. Layered graph matching

  48. Layered graph matching Generate a composite cluster CCP: Candidates connected by the positive “on” edges form a CCP. (blue lines) Composite Cluster: A few CCPs connected by negative “on” edges form a composite cluster.(red lines)

  49. Layered graph matching Generate a composite cluster

  50. Layered graph matching Re-assign color • Primitives connected by positive edges receive the same color. The ones connected by negative edges receive different color.