Reconstructing relief surfaces
Download
1 / 29

Reconstructing Relief Surfaces - PowerPoint PPT Presentation


  • 105 Views
  • Uploaded on

Reconstructing Relief Surfaces. George Vogiatzis, Philip Torr, Steven Seitz and Roberto Cipolla BMVC 2004. Stereo reconstruction problem:. Input Set of images of a scene I={I 1 ,…,I K } Camera matrices P 1 ,…,P K Output Surface model. Shape parametrisation. Disparity-map parametrisation

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Reconstructing Relief Surfaces' - belva


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Reconstructing relief surfaces

Reconstructing Relief Surfaces

George Vogiatzis, Philip Torr, Steven Seitz and Roberto Cipolla

BMVC 2004


Stereo reconstruction problem
Stereo reconstruction problem:

  • Input

    • Set of images of a scene I={I1,…,IK}

    • Camera matrices P1,…,PK

  • Output

    • Surface model


Shape parametrisation
Shape parametrisation

  • Disparity-map parametrisation

    • MRF formulation – good optimisation techniques exist (Graph-cuts, Loopy BP)

    • MRF smoothness is viewpoint dependent

    • Disparity is unique per pixel – only functions represented


Shape parametrisation1
Shape parametrisation

  • Volumetric parametrisation – e.g. Level-sets, Space carving etc.

    • Able to cope with non-functions

    • Convergence properties not well understood, Local minima

    • Memory intensive

    • For Space carving, no simple way to impose surface smoothness


Solution
Solution ?

  • Cast volumetric methods in MRF framework

  • Key assumption: Approximate scene geometry given

  • Benefits:

    • General surfaces can be represented

    • Optimisation is tractable (MRF solvers)

    • Occlusions are approximately modelled

    • Smoothness is viewpoint independent


Reconstructing relief surfaces
MRFs

  • The labelling problem:


Reconstructing relief surfaces
MRFs

  • A set of random variables h1,…,hM

  • A binary neighbourhood relation N defined on the variables

  • Each can take a label out of a set H1,…,HL

  • Ci(hi) (Labelling cost)

  • Ci,j(hi,hj) for (i,j)N (Compatibility cost)

    -log P(h1,…,hM) =  Ci(hi) +  Ci,j(hi,hj)


Mrf inference
MRF inference

  • Minimise  Ci(hi) +  Ci,j(hi,hj)

  • Not in polynomial time in general case

  • Special cases (e.g. no loops or 2 label MRF) solved exactly

  • General cases solved approximately via Graph-cuts or Loopy Belief Propagation. Approx. 10-15mins for MRF with 250,000 nodes.


Relief surfaces
Relief Surfaces

  • Approximate base surface

    • Triangulated feature matches

    • Visual hull from silhouettes

    • Initialised by hand


Relief surfaces1

labels :

Relief Surfaces


Relief surfaces2

labelling cost :

Low cost

High cost

Relief Surfaces

Xi+hini

ni

Xi

Ci(hi)=photoconsistency(Xi+hini)


Relief surfaces3
Relief Surfaces

Compatibility cost :

Xj+hjnj

Low cost

Xi+hini

nj

Xj

ni

Xi


Relief surfaces4
Relief Surfaces

Neighbour cost :

Xi+hini

High cost

Xj+hjnj

ni

Xi

Ci,j(hi, hj)= ||(Xi+hini)-(Xj+hjnj)||


Relief surfaces5
Relief Surfaces

  • Base surface is the occluding volume

  • If base surface ‘contains’ true surface (e.g. visual hull) then

    • Points on the base surface Xi are not visible by cameras they shouldn’t be [Kutulakos, Seitz 2000]

  • Approximation:

    • Visibility is propagated from Xi to Xi+hini


Loopy belief propagation

mi,j

i

j

Loopy Belief Propagation

min  Ci(hi) +  Ci,j(hi,hj)

  • Iterative message passing algorithm

  • m(t)i,j (hj) is the message passed from i to j at time step t

  • It is a L-dimensional vector

  • Represents what node i ‘believes’ about the true state of node j.


Loopy belief propagation1

m(t+1)i,j (hj)= min{ Cij(hi,hj) +Ci(hi) +m(t)k,i (hi)}

hi

kN(i)

hi*= min{Ci(hi) +m()k,i (hi)}

mi,j

kN(i)

hi

i

j

Loopy Belief Propagation

  • Message passing rule:

  • After convergence, optimal state is given by


Loopy belief propagation2
Loopy Belief Propagation

  • O(L2) to compute a message (L is number of allowable heights)

  • Message passing schedule can be asynchronous which can accelerate convergence [Tappen & Freeman ICCV 03]


Iterative scheme
Iterative Scheme

  • BP is memory intensive.

  • Can consider few possible labels at a time

  • After convergence we ‘zoom in’ to heights close to the optimal


Evaluation

True surface

Texture-mapped

Reconstruction

Evaluation

  • Artificial deformed sphere

  • Textured with random patern

  • 20 images

  • 40,000 sample points on sphere base surface


Evaluation1
Evaluation

  • Benchmark: 2-view, disparity based Loopy Belief Propagation [Sun et al ECCV02]

  • BP run on 10 pairs of nearby views

  • Compare Disparity Maps given by

    • 2-view BP

    • Relief surfaces

    • Ground truth


Evaluation2

Relief surface

Ground truth

2-view BP

Evaluation


Results
Results

  • Sarcophagus


Results1
Results

  • Sarcophagus


Results2
Results

  • Sarcophagus


Results3
Results

  • Building facade


Results4
Results

  • Building facade


Results5
Results

  • Stone carving

Relief surface

with texture

Base surface

Relief surface


Summary
Summary

  • MRF methods can be extended in the volumetric domain

  • Advantages

    • General surfaces can be represented

    • Optimisation is tractable (MRF solvers)

    • Smoothness is viewpoint independent


Future work
Future work

  • Photoconsistency beyond Lambertian surface models. (Optimise both height and surface normal fields)

  • Change in topology

  • In cases where Cmn(hm,hn)=|| hm-hn||or || hm-hn||2 we can compute messages in O(L) time instead of O(L2) (Felzenszwalb & Huttenlocher CVPR 04).