using strong shape priors for multiview reconstruction
Download
Skip this Video
Download Presentation
Using Strong Shape Priors for Multiview Reconstruction

Loading in 2 Seconds...

play fullscreen
1 / 37

Using Strong Shape Priors for Multiview Reconstruction - PowerPoint PPT Presentation


  • 221 Views
  • Uploaded on

Using Strong Shape Priors for Multiview Reconstruction. Yunda Sun Pushmeet Kohli Mathieu Bray Philip HS Torr. Department of Computing Oxford Brookes University. Objective. Images Silhouettes. Parametric Model. +. Pose Estimate Reconstruction. [Images Courtesy: M. Black, L. Sigal].

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Using Strong Shape Priors for Multiview Reconstruction' - issac


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
using strong shape priors for multiview reconstruction

Using Strong Shape Priors for Multiview Reconstruction

Yunda Sun Pushmeet Kohli

Mathieu Bray Philip HS Torr

Department of Computing

Oxford Brookes University

objective
Objective

Images

Silhouettes

Parametric Model

+

Pose

Estimate

Reconstruction

[Images Courtesy: M. Black, L. Sigal]

outline
Outline
  • Multi-view Reconstruction
  • Shape Models as Strong Priors
  • Object Specific MRF
  • Pose Estimation
  • Results
outline4
Outline
  • Multi-view Reconstruction
  • Shape Models as Strong Priors
  • Object Specific MRF
  • Pose Estimation
  • Results
multiview reconstruction
Multiview Reconstruction

Need for Shape Priors

multiview reconstruction6
Multiview Reconstruction
  • No Priors
    • Silhouette Intersection
    • Space Carving
  • Weak Priors
    • Surface smoothness
      • Snow et al. CVPR ’00
    • Photo consistency and smoothness
      • Kolmogorov and Zabih [ECCV ’02]
      • Vogiatzis et al. [CVPR ’05]

[Image Courtesy: Vogiatzis et al.]

outline7
Outline
  • Multi-view Reconstruction
  • Shape Models as Strong Priors
  • Object Specific MRF
  • Pose Estimation
  • Results
shape priors for segmentation
Shape-Priors for Segmentation
  • OBJ-CUT [Kumar et al., CVPR ’05]
    • Integrate Shape Priors in a MRF
  • POSE-CUT [Bray et al., ECCV ’06]
    • Efficient Inference of Model Parameters
parametric object models as strong priors
Parametric Object Models as Strong Priors
  • Layered Pictorial Structures
  • Articulated Models
  • Deformable Models
outline10
Outline
  • Multi-view Reconstruction
  • Shape Models as Strong Priors
  • Object Specific MRF
  • Pose Estimation and Reconstruction
  • Results
object specific mrf12
Object-Specific MRF

Energy Function

Shape Prior

Unary Likelihood

Smoothness Prior

x:Voxel label

θ: Model Shape

object specific mrf13
Object-Specific MRF

Shape Prior

: shortest distance of voxel i from the rendered model

x:Voxel label

θ: Model Shape

object specific mrf14
Object-Specific MRF

Smoothness Prior

Potts Model

x:Voxel label

θ: Model Shape

object specific mrf15
Object-Specific MRF

Unary Likelihood

For a soft constraint we use a large constant K instead of infinity

x:Voxel label

θ: Model Shape

: Visual Hull

object specific mrf16
Object-Specific MRF

Energy Function

Shape Prior

Unary Likelihood

Smoothness Prior

Can be solved using Graph cuts

[Kolmogorov and Zabih, ECCV02 ]

object specific mrf17
Object-Specific MRF

Energy Function

Shape Prior

Unary Likelihood

Smoothness Prior

How to find the optimal Pose?

outline18
Outline
  • Multi-view Reconstruction
  • Shape Models as Strong Priors
  • Object Specific MRF
  • Pose Estimation
  • Results
inference of pose parameters
Inference of Pose Parameters

Rotation and Translation of Torso in X axes

Rotation of left shoulder in X and Z axes

inference of pose parameters20
Inference of Pose Parameters

Let F(ө) =

Minimize F(ө) using Powell Minimization

Computational Problem:

Each evaluation of F(ө) requires a graph cut to be computed. (computationally expensive!!) BUT..

Solution: Usethe dynamic graph cut algorithm

[Kohli&Torr, ICCV 2005]

outline21
Outline
  • Multi-view Reconstruction
  • Shape Models as Strong Priors
  • Object Specific MRF
  • Pose Estimation
  • Results
experiments
Experiments
  • Deformable Models
  • Articulated Models
    • Reconstruction Results
    • Human Pose Estimation
deformable models
Deformable Models

Visual Hull

  • Four Cameras
  • 1.5 x 105 voxels
  • DOF of Model: 5

Our Reconstruction

Shape Model

articulated models25
Articulated Models
  • Four Cameras
  • 106 voxels
  • DOF of Model: 26

Camera Setup

Shape Model

articulated models26
Articulated Models
  • 500 function evaluations of F(θ) required
  • Time per evaluation: 0.15 sec
  • Total time: 75 sec

Let F(ө) =

articulated models27
Articulated Models

Visual Hull

Our Reconstruction

pose estimation results
Pose Estimation Results

Visual Hull

Reconstruction

Pose Estimate

pose estimation results29
Pose Estimation Results
  • Quantitative Results
    • 6 uniformly distributed cameras
    • 12 degree (RMS) error over 21 joint angles
pose estimation results30
Pose Estimation Results
  • Qualitative Results
pose estimation results31
Pose Estimation Results

Video 1, Camera 1

pose estimation results32
Pose Estimation Results

Video 1, Camera 2

pose estimation results33
Pose Estimation Results

Video 2, Camera 1

pose estimation results34
Pose Estimation Results

Video 2, Camera 2

future work
Future Work
  • Use dimensionality reduction to reduce the number of pose parameters.
    • results in less number of pose parameters to optimize
    • would speed up inference
  • High resolution reconstruction by a coarse to fine strategy
  • Parameter Learning in Object Specific MRF
object specific mrf37
Object-Specific MRF

Energy Function

Shape Prior

Unary Likelihood

Smoothness Prior

+

ad