1 / 16

Recovering Human Body Configurations with Pairwise Constraints in Parts

This paper explores the challenges in finding people such as pose, clothing, and lighting and presents a novel approach using pairwise constraints between body parts. It discusses the importance of beyond-tree cues and outlines a methodology involving preprocessing, detecting candidate parts, learning constraints, and assembling parts using Integer Quadratic Programming. The technique involves Constrained Delaunay Triangulation, detecting parts using parallelism, and learning pairwise constraints from labeled images. The assembly process is detailed, showcasing how IQP efficiently solves the configuration problem. The study concludes that going beyond traditional tree models and emphasizing pairwise constraints are crucial in accurately recovering human body configurations under various conditions.

calliope
Download Presentation

Recovering Human Body Configurations with Pairwise Constraints in Parts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Recovering Human Body Configurations using Pairwise Constraints between Parts Xiaofeng Ren, Alex Berg, Jitendra Malik 1

  2. Finding People Challenges: Pose, Clothing, Lighting, Clutter, … 2

  3. Previous Work • Related Domains • Tracking People • Detecting Pedestrians • ... … • Localizing Human Figures • Exemplar-based: [Toyama & Blake 01], [Mori & Malik 02], [Sullivan & Carlsson 02], [Shakhnarovich, Viola & Darrell 03], … • Part-based: [Felzenswalb & Huttenlocher 00], [Ioffe & Forsyth 01], [Song, Goncalves & Perona 03], [Mori, Ren, Efros & Malik 04], … • … … 3

  4. Beyond “Trees” ? • A hard problem! More information is needed. • Important cues that are NOT in the tree model: • Symmetry of clothing/color • “V-shape” formed by the upper legs • Distance/smooth connection between arms and legs • …… 4

  5. Our Approach • Preprocessing with Constrained Delaunay Triangulation • Detecting Candidate Parts from Bottom-up • Learning Pairwise Constraints between Parts • Assembling Parts by Integer Quadratic Programming (IQP) 5

  6. Constrained Delaunay Triangulation • Detect edges with Pb (Probability of Boundary) • Trace contours with Canny’s hysteresis • Recursively split contours into piecewise straight lines • Complete the partial graph with Constrained Delaunay Triangulation 6

  7. Detecting Parts using Parallelism N C1 (L1,1) C2 (L2,2) T • Candidate parts as parallel line segments (Ebenbreite) • (Scale-invariant) Features for parallelism: |Pb1+Pb2|/2, |1-2|, |L1-L2|/|L1+L2|, |(C1-C2)T|/|L1+L2|, |(C1-C2)N|/|L1+L2| • Logistic Classifier 7

  8. C1 C2 Pairwise Constraints between Parts • Scale (width) consistency • Use anthropometric data as groundtruth • Symmetry of appearance (color) • Orientation consistency • Connectivity • Short distance between adjacent parts • “Smooth” connection between non-adjacent parts • short “gaps” on shortest path (on CDT graph) • small maximum angle on the shortest path • few T-junctions/turns on the shortest path 8

  9. Learning Pairwise Constraints 15 hand-labeled images from a skating sequence Empirical distributions of some pairwise features For simplicity, assume all features are Gaussian (future work here as they are clearly non-Gaussian) 9

  10. Assembling Parts as Assignment (Lj1,Ci1=(Lj1)) (Lj2,Ci2=(Lj2)) Candidates {Ci} Parts {Lj} assignment  Cost for a partial assignment {(Lj1,Ci1), (Lj2,Ci2)}: 10

  11. Assignment by IQP Q(x)=xTHx • Suppose there are m parts and n candidates, the optimal assignment  minimizes a quadratic function where x is a mn1 indicator vector and H is of size mnmn. • This is a well-formulated Integer Quadratic Programming (IQP) problem and has efficient approximate solutions. • We choose an approximation scheme which solves mn linear programs followed by gradient descent. • The approximate scheme produces a ranked list of torso candidates. We consider the top 5 torso candidates and solve the corresponding 5 IQP problems. • We have m=9 and n~150; the total time is less than a minute. 11

  12. 12

  13. 13

  14. 14

  15. Conclusion • To find people under general conditions, we need to go beyond the traditional tree-based model; • Most important constraints for the human body are between pairs of body parts; • Pairwise constraints may be learned from a small set of training examples; • Integer Quadratic Programming (IQP) efficiently finds optimal configurations under pairwise constraints. 15

  16. 16

More Related