1 / 37

POSE–CUT Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts

POSE–CUT Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts. Mathieu Bray Pushmeet Kohli Philip H.S. Torr Department of Computing Oxford Brookes University. Objective. Image. Segmentation. Pose Estimate. [Images courtesy: M. Black, L. Sigal].

farica
Download Presentation

POSE–CUT Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. POSE–CUTSimultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts Mathieu Bray Pushmeet Kohli Philip H.S. Torr Department of Computing Oxford Brookes University

  2. Objective Image Segmentation Pose Estimate [Images courtesy: M. Black, L. Sigal]

  3. Outline • Image Segmentation Problem • Pose-Specific Segmentation • The Pose Inference Problem • Optimization • Results • Conclusion and Future Work

  4. Outline • Image Segmentation Problem • Pose-Specific Segmentation • The Pose Inference Problem • Optimization • Results • Conclusion and Future Work

  5. The Image Segmentation Problem Segments Image

  6. Problem – MRF Formulation • Notation • Labelling x over the set of pixels • The observed pixel intensity values y (constitute data D) • Energy E (x) = - log Pr(x|D) + constant • Unary term • Likelihood based on colour • Pairwise terms • Prior • Contrast term • Find best labelling x* = arg min E(x)

  7. MRF for Image Segmentation xi = {segment1, …, segmentk} for instance {obj, bkg} Pairwise Potential ij(xi, xj) xi x(labels) xj Unary Potential i(D|xi) i j D(pixels) Image Plane

  8. Can be solved using graph cuts Unary likelihood Contrast Term Ising Model Maximum a-posteriori (MAP) solution x*= Data (D) Unary likelihood Pair-wise Terms MAP Solution MRF for Image Segmentation

  9. MRF for Image Segmentation Need for a human like segmentation Unary likelihood Contrast Term Uniform Prior Maximum-a-posteriori (MAP) solution x*= Data (D) Unary likelihood Pair-wise Terms MAP Solution

  10. Outline • Image Segmentation Problem • Pose-Specific Segmentation • The Pose Inference Problem • Optimization • Results • Conclusion and Future Work

  11. Shape-Priors and Segmentation OBJ-CUT [Kumar et al., CVPR ’05] • Shape-Prior: Layered Pictorial Structure (LPS) • Learned exemplars for parts of the LPS model • Obtained impressive results = + Spatial Layout (Pairwise Configuration) Layer 1 Layer 2

  12. Shape-Priors and Segmentation OBJ-CUT [Kumar et al., CVPR ’05] • Shape-Prior: Layered Pictorial Structure (LPS) • Learned exemplars for parts of the LPS model • Obtained impressive results Unary likelihood colour Shape-Prior Colour + Shape Image

  13. Problems in using shape priors • Intra-class variability • Need to learn an enormous exemplar set • Infeasible for complex subjects (Humans) • Multiple Aspects? • Inference of pose parameters

  14. Do we really need accurate models? • Interactive Image Segmentation [Boykov & Jolly, ICCV’01] • Rough region cues sufficient • Segmentation boundary can be extracted from edges additional segmentation cues user segmentation cues

  15. Do we really need accurate models? • Interactive Image Segmentation • Rough region cues sufficient • Segmentation boundary can be extracted from edges

  16. Rough Shape Prior - The Stickman Model • 26 degrees of freedom • Can be rendered extremely efficiently • Over-comes problems of learning a huge exemplar set • Gives accurate segmentation results

  17. Pose-specific MRF Formulation  (pose parameters) Unary Potential i(xi|) Pairwise Potential ij(xi, xj) xi x(labels) xj Unary Potential i(D|xi) i j D(pixels) Image Plane

  18. Pose-specific MRF Energy to be minimized Pairwise potential Unary term Potts model Shape prior distance transform

  19. Pose-specific MRF Energy to be minimized Pairwise potential Unary term Potts model Shape prior + = Colour likelihood colour+ shape MAP Solution Shape Prior Data (D)

  20. What is the shape prior? Energy to be minimized Pairwise potential Unary term Potts model Shape prior How to find the value of ө?

  21. Outline • Image Segmentation Problem • Pose-Specific Segmentation • The Pose Inference Problem • Optimization • Results • Conclusion and Future Work

  22. Formulating the Pose Inference Problem

  23. Formulating the Pose Inference Problem

  24. Resolving ambiguity using multiple views Pose specific Segmentation Energy

  25. Outline • Image Segmentation Problem • Pose-Specific Segmentation • The Pose Inference Problem • Optimization • Results • Conclusion and Future Work

  26. Solving the Minimization Problem To solve: Let F(ө) = Minimize F(ө) using Powell Minimization Computational Problem: Each evaluation of F(ө) requires a graph cut to be computed. (computationally expensive!!) BUT.. Solution: Usethe dynamic graph cut algorithm [Kohli&Torr, ICCV 2005]

  27. solve SA differences between A and B PB* Simpler problem A and B similar SB Dynamic Graph Cuts PA cheaper operation PB computationally expensive operation

  28. solve xa differences between A and B PB* Simpler problem A and B similar Dynamic Graph Cuts 20 msec xb 400 msec

  29. Outline • Image Segmentation Problem • Pose-Specific Segmentation • The Pose Inference Problem • Optimization • Results • Conclusion and Future Work

  30. Segmentation Results Only Colour Colour + Smoothness Colour + Smoothness + Shape Prior Image [Images courtesy: M. Black, L. Sigal]

  31. Segmentation Results - Accuracy

  32. Segmentation + Pose inference [Images courtesy: M. Black, L. Sigal]

  33. Segmentation + Pose inference [Images courtesy: Vicon]

  34. Outline • Image Segmentation Problem • Pose-Specific Segmentation • The Pose Inference Problem • Optimization • Results • Conclusion and Future Work

  35. Conclusions • Efficient method for using shape priors for object-specific segmentation • Efficient Inference of pose parameters using dynamic graph cuts • Good segmentation results • Pose inference • Needs further evaluation • Segmentation results could be used for silhouette intersection

  36. Future Work • Use dimensionality reduction to reduce the number of pose parameters. • results in less number of pose parameteres to optimize • would speed up inference • Use of features based on texture • Appearance models for individual part of the articulated model (instead of using a single appearance model).

  37. Thank You

More Related