reconstructing relief surfaces
Download
Skip this Video
Download Presentation
Reconstructing Relief Surfaces

Loading in 2 Seconds...

play fullscreen
1 / 29

Reconstructing Relief Surfaces - PowerPoint PPT Presentation


  • 105 Views
  • Uploaded on

Reconstructing Relief Surfaces. George Vogiatzis, Philip Torr, Steven Seitz and Roberto Cipolla BMVC 2004. Stereo reconstruction problem:. Input Set of images of a scene I={I 1 ,…,I K } Camera matrices P 1 ,…,P K Output Surface model. Shape parametrisation. Disparity-map parametrisation

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Reconstructing Relief Surfaces' - belva


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
reconstructing relief surfaces

Reconstructing Relief Surfaces

George Vogiatzis, Philip Torr, Steven Seitz and Roberto Cipolla

BMVC 2004

stereo reconstruction problem
Stereo reconstruction problem:
  • Input
    • Set of images of a scene I={I1,…,IK}
    • Camera matrices P1,…,PK
  • Output
    • Surface model
shape parametrisation
Shape parametrisation
  • Disparity-map parametrisation
    • MRF formulation – good optimisation techniques exist (Graph-cuts, Loopy BP)
    • MRF smoothness is viewpoint dependent
    • Disparity is unique per pixel – only functions represented
shape parametrisation1
Shape parametrisation
  • Volumetric parametrisation – e.g. Level-sets, Space carving etc.
    • Able to cope with non-functions
    • Convergence properties not well understood, Local minima
    • Memory intensive
    • For Space carving, no simple way to impose surface smoothness
solution
Solution ?
  • Cast volumetric methods in MRF framework
  • Key assumption: Approximate scene geometry given
  • Benefits:
    • General surfaces can be represented
    • Optimisation is tractable (MRF solvers)
    • Occlusions are approximately modelled
    • Smoothness is viewpoint independent
slide6
MRFs
  • The labelling problem:
slide7
MRFs
  • A set of random variables h1,…,hM
  • A binary neighbourhood relation N defined on the variables
  • Each can take a label out of a set H1,…,HL
  • Ci(hi) (Labelling cost)
  • Ci,j(hi,hj) for (i,j)N (Compatibility cost)

-log P(h1,…,hM) =  Ci(hi) +  Ci,j(hi,hj)

mrf inference
MRF inference
  • Minimise  Ci(hi) +  Ci,j(hi,hj)
  • Not in polynomial time in general case
  • Special cases (e.g. no loops or 2 label MRF) solved exactly
  • General cases solved approximately via Graph-cuts or Loopy Belief Propagation. Approx. 10-15mins for MRF with 250,000 nodes.
relief surfaces
Relief Surfaces
  • Approximate base surface
    • Triangulated feature matches
    • Visual hull from silhouettes
    • Initialised by hand
relief surfaces2

labelling cost :

Low cost

High cost

Relief Surfaces

Xi+hini

ni

Xi

Ci(hi)=photoconsistency(Xi+hini)

relief surfaces3
Relief Surfaces

Compatibility cost :

Xj+hjnj

Low cost

Xi+hini

nj

Xj

ni

Xi

relief surfaces4
Relief Surfaces

Neighbour cost :

Xi+hini

High cost

Xj+hjnj

ni

Xi

Ci,j(hi, hj)= ||(Xi+hini)-(Xj+hjnj)||

relief surfaces5
Relief Surfaces
  • Base surface is the occluding volume
  • If base surface ‘contains’ true surface (e.g. visual hull) then
    • Points on the base surface Xi are not visible by cameras they shouldn’t be [Kutulakos, Seitz 2000]
  • Approximation:
    • Visibility is propagated from Xi to Xi+hini
loopy belief propagation

mi,j

i

j

Loopy Belief Propagation

min  Ci(hi) +  Ci,j(hi,hj)

  • Iterative message passing algorithm
  • m(t)i,j (hj) is the message passed from i to j at time step t
  • It is a L-dimensional vector
  • Represents what node i ‘believes’ about the true state of node j.
loopy belief propagation1

m(t+1)i,j (hj)= min{ Cij(hi,hj) +Ci(hi) +m(t)k,i (hi)}

hi

kN(i)

hi*= min{Ci(hi) +m()k,i (hi)}

mi,j

kN(i)

hi

i

j

Loopy Belief Propagation
  • Message passing rule:
  • After convergence, optimal state is given by
loopy belief propagation2
Loopy Belief Propagation
  • O(L2) to compute a message (L is number of allowable heights)
  • Message passing schedule can be asynchronous which can accelerate convergence [Tappen & Freeman ICCV 03]
iterative scheme
Iterative Scheme
  • BP is memory intensive.
  • Can consider few possible labels at a time
  • After convergence we ‘zoom in’ to heights close to the optimal
evaluation

True surface

Texture-mapped

Reconstruction

Evaluation
  • Artificial deformed sphere
  • Textured with random patern
  • 20 images
  • 40,000 sample points on sphere base surface
evaluation1
Evaluation
  • Benchmark: 2-view, disparity based Loopy Belief Propagation [Sun et al ECCV02]
  • BP run on 10 pairs of nearby views
  • Compare Disparity Maps given by
    • 2-view BP
    • Relief surfaces
    • Ground truth
evaluation2

Relief surface

Ground truth

2-view BP

Evaluation
results
Results
  • Sarcophagus
results1
Results
  • Sarcophagus
results2
Results
  • Sarcophagus
results3
Results
  • Building facade
results4
Results
  • Building facade
results5
Results
  • Stone carving

Relief surface

with texture

Base surface

Relief surface

summary
Summary
  • MRF methods can be extended in the volumetric domain
  • Advantages
    • General surfaces can be represented
    • Optimisation is tractable (MRF solvers)
    • Smoothness is viewpoint independent
future work
Future work
  • Photoconsistency beyond Lambertian surface models. (Optimise both height and surface normal fields)
  • Change in topology
  • In cases where Cmn(hm,hn)=|| hm-hn||or || hm-hn||2 we can compute messages in O(L) time instead of O(L2) (Felzenszwalb & Huttenlocher CVPR 04).
ad