Using strong shape priors for multiview reconstruction
Download
1 / 37

Using Strong Shape Priors for Multiview Reconstruction - PowerPoint PPT Presentation


  • 219 Views
  • Uploaded on

Using Strong Shape Priors for Multiview Reconstruction. Yunda Sun Pushmeet Kohli Mathieu Bray Philip HS Torr. Department of Computing Oxford Brookes University. Objective. Images Silhouettes. Parametric Model. +. Pose Estimate Reconstruction. [Images Courtesy: M. Black, L. Sigal].

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Using Strong Shape Priors for Multiview Reconstruction' - issac


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Using strong shape priors for multiview reconstruction l.jpg

Using Strong Shape Priors for Multiview Reconstruction

Yunda Sun Pushmeet Kohli

Mathieu Bray Philip HS Torr

Department of Computing

Oxford Brookes University


Objective l.jpg
Objective

Images

Silhouettes

Parametric Model

+

Pose

Estimate

Reconstruction

[Images Courtesy: M. Black, L. Sigal]


Outline l.jpg
Outline

  • Multi-view Reconstruction

  • Shape Models as Strong Priors

  • Object Specific MRF

  • Pose Estimation

  • Results


Outline4 l.jpg
Outline

  • Multi-view Reconstruction

  • Shape Models as Strong Priors

  • Object Specific MRF

  • Pose Estimation

  • Results


Multiview reconstruction l.jpg
Multiview Reconstruction

Need for Shape Priors


Multiview reconstruction6 l.jpg
Multiview Reconstruction

  • No Priors

    • Silhouette Intersection

    • Space Carving

  • Weak Priors

    • Surface smoothness

      • Snow et al. CVPR ’00

    • Photo consistency and smoothness

      • Kolmogorov and Zabih [ECCV ’02]

      • Vogiatzis et al. [CVPR ’05]

[Image Courtesy: Vogiatzis et al.]


Outline7 l.jpg
Outline

  • Multi-view Reconstruction

  • Shape Models as Strong Priors

  • Object Specific MRF

  • Pose Estimation

  • Results


Shape priors for segmentation l.jpg
Shape-Priors for Segmentation

  • OBJ-CUT [Kumar et al., CVPR ’05]

    • Integrate Shape Priors in a MRF

  • POSE-CUT [Bray et al., ECCV ’06]

    • Efficient Inference of Model Parameters


Parametric object models as strong priors l.jpg
Parametric Object Models as Strong Priors

  • Layered Pictorial Structures

  • Articulated Models

  • Deformable Models


Outline10 l.jpg
Outline

  • Multi-view Reconstruction

  • Shape Models as Strong Priors

  • Object Specific MRF

  • Pose Estimation and Reconstruction

  • Results



Object specific mrf12 l.jpg
Object-Specific MRF

Energy Function

Shape Prior

Unary Likelihood

Smoothness Prior

x:Voxel label

θ: Model Shape


Object specific mrf13 l.jpg
Object-Specific MRF

Shape Prior

: shortest distance of voxel i from the rendered model

x:Voxel label

θ: Model Shape


Object specific mrf14 l.jpg
Object-Specific MRF

Smoothness Prior

Potts Model

x:Voxel label

θ: Model Shape


Object specific mrf15 l.jpg
Object-Specific MRF

Unary Likelihood

For a soft constraint we use a large constant K instead of infinity

x:Voxel label

θ: Model Shape

: Visual Hull


Object specific mrf16 l.jpg
Object-Specific MRF

Energy Function

Shape Prior

Unary Likelihood

Smoothness Prior

Can be solved using Graph cuts

[Kolmogorov and Zabih, ECCV02 ]


Object specific mrf17 l.jpg
Object-Specific MRF

Energy Function

Shape Prior

Unary Likelihood

Smoothness Prior

How to find the optimal Pose?


Outline18 l.jpg
Outline

  • Multi-view Reconstruction

  • Shape Models as Strong Priors

  • Object Specific MRF

  • Pose Estimation

  • Results


Inference of pose parameters l.jpg
Inference of Pose Parameters

Rotation and Translation of Torso in X axes

Rotation of left shoulder in X and Z axes


Inference of pose parameters20 l.jpg
Inference of Pose Parameters

Let F(ө) =

Minimize F(ө) using Powell Minimization

Computational Problem:

Each evaluation of F(ө) requires a graph cut to be computed. (computationally expensive!!) BUT..

Solution: Usethe dynamic graph cut algorithm

[Kohli&Torr, ICCV 2005]


Outline21 l.jpg
Outline

  • Multi-view Reconstruction

  • Shape Models as Strong Priors

  • Object Specific MRF

  • Pose Estimation

  • Results


Experiments l.jpg
Experiments

  • Deformable Models

  • Articulated Models

    • Reconstruction Results

    • Human Pose Estimation


Deformable models l.jpg
Deformable Models

Visual Hull

  • Four Cameras

  • 1.5 x 105 voxels

  • DOF of Model: 5

Our Reconstruction

Shape Model



Articulated models25 l.jpg
Articulated Models

  • Four Cameras

  • 106 voxels

  • DOF of Model: 26

Camera Setup

Shape Model


Articulated models26 l.jpg
Articulated Models

  • 500 function evaluations of F(θ) required

  • Time per evaluation: 0.15 sec

  • Total time: 75 sec

Let F(ө) =


Articulated models27 l.jpg
Articulated Models

Visual Hull

Our Reconstruction


Pose estimation results l.jpg
Pose Estimation Results

Visual Hull

Reconstruction

Pose Estimate


Pose estimation results29 l.jpg
Pose Estimation Results

  • Quantitative Results

    • 6 uniformly distributed cameras

    • 12 degree (RMS) error over 21 joint angles


Pose estimation results30 l.jpg
Pose Estimation Results

  • Qualitative Results


Pose estimation results31 l.jpg
Pose Estimation Results

Video 1, Camera 1


Pose estimation results32 l.jpg
Pose Estimation Results

Video 1, Camera 2


Pose estimation results33 l.jpg
Pose Estimation Results

Video 2, Camera 1


Pose estimation results34 l.jpg
Pose Estimation Results

Video 2, Camera 2


Future work l.jpg
Future Work

  • Use dimensionality reduction to reduce the number of pose parameters.

    • results in less number of pose parameters to optimize

    • would speed up inference

  • High resolution reconstruction by a coarse to fine strategy

  • Parameter Learning in Object Specific MRF



Object specific mrf37 l.jpg
Object-Specific MRF

Energy Function

Shape Prior

Unary Likelihood

Smoothness Prior

+


ad