1 / 29

Reconstructing Relief Surfaces

Reconstructing Relief Surfaces. George Vogiatzis, Philip Torr, Steven Seitz and Roberto Cipolla BMVC 2004. Stereo reconstruction problem:. Input Set of images of a scene I={I 1 ,…,I K } Camera matrices P 1 ,…,P K Output Surface model. Shape parametrisation. Disparity-map parametrisation

belva
Download Presentation

Reconstructing Relief Surfaces

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Reconstructing Relief Surfaces George Vogiatzis, Philip Torr, Steven Seitz and Roberto Cipolla BMVC 2004

  2. Stereo reconstruction problem: • Input • Set of images of a scene I={I1,…,IK} • Camera matrices P1,…,PK • Output • Surface model

  3. Shape parametrisation • Disparity-map parametrisation • MRF formulation – good optimisation techniques exist (Graph-cuts, Loopy BP) • MRF smoothness is viewpoint dependent • Disparity is unique per pixel – only functions represented

  4. Shape parametrisation • Volumetric parametrisation – e.g. Level-sets, Space carving etc. • Able to cope with non-functions • Convergence properties not well understood, Local minima • Memory intensive • For Space carving, no simple way to impose surface smoothness

  5. Solution ? • Cast volumetric methods in MRF framework • Key assumption: Approximate scene geometry given • Benefits: • General surfaces can be represented • Optimisation is tractable (MRF solvers) • Occlusions are approximately modelled • Smoothness is viewpoint independent

  6. MRFs • The labelling problem:

  7. MRFs • A set of random variables h1,…,hM • A binary neighbourhood relation N defined on the variables • Each can take a label out of a set H1,…,HL • Ci(hi) (Labelling cost) • Ci,j(hi,hj) for (i,j)N (Compatibility cost) -log P(h1,…,hM) =  Ci(hi) +  Ci,j(hi,hj)

  8. MRF inference • Minimise  Ci(hi) +  Ci,j(hi,hj) • Not in polynomial time in general case • Special cases (e.g. no loops or 2 label MRF) solved exactly • General cases solved approximately via Graph-cuts or Loopy Belief Propagation. Approx. 10-15mins for MRF with 250,000 nodes.

  9. Relief Surfaces • Approximate base surface • Triangulated feature matches • Visual hull from silhouettes • Initialised by hand

  10. labels : Relief Surfaces

  11. labelling cost : Low cost High cost Relief Surfaces Xi+hini ni Xi Ci(hi)=photoconsistency(Xi+hini)

  12. Relief Surfaces Compatibility cost : Xj+hjnj Low cost Xi+hini nj Xj ni Xi

  13. Relief Surfaces Neighbour cost : Xi+hini High cost Xj+hjnj ni Xi Ci,j(hi, hj)= ||(Xi+hini)-(Xj+hjnj)||

  14. Relief Surfaces • Base surface is the occluding volume • If base surface ‘contains’ true surface (e.g. visual hull) then • Points on the base surface Xi are not visible by cameras they shouldn’t be [Kutulakos, Seitz 2000] • Approximation: • Visibility is propagated from Xi to Xi+hini

  15. mi,j i j Loopy Belief Propagation min  Ci(hi) +  Ci,j(hi,hj) • Iterative message passing algorithm • m(t)i,j (hj) is the message passed from i to j at time step t • It is a L-dimensional vector • Represents what node i ‘believes’ about the true state of node j.

  16. m(t+1)i,j (hj)= min{ Cij(hi,hj) +Ci(hi) +m(t)k,i (hi)} hi kN(i) hi*= min{Ci(hi) +m()k,i (hi)} mi,j kN(i) hi i j Loopy Belief Propagation • Message passing rule: • After convergence, optimal state is given by

  17. Loopy Belief Propagation • O(L2) to compute a message (L is number of allowable heights) • Message passing schedule can be asynchronous which can accelerate convergence [Tappen & Freeman ICCV 03]

  18. Iterative Scheme • BP is memory intensive. • Can consider few possible labels at a time • After convergence we ‘zoom in’ to heights close to the optimal

  19. True surface Texture-mapped Reconstruction Evaluation • Artificial deformed sphere • Textured with random patern • 20 images • 40,000 sample points on sphere base surface

  20. Evaluation • Benchmark: 2-view, disparity based Loopy Belief Propagation [Sun et al ECCV02] • BP run on 10 pairs of nearby views • Compare Disparity Maps given by • 2-view BP • Relief surfaces • Ground truth

  21. Relief surface Ground truth 2-view BP Evaluation

  22. Results • Sarcophagus

  23. Results • Sarcophagus

  24. Results • Sarcophagus

  25. Results • Building facade

  26. Results • Building facade

  27. Results • Stone carving Relief surface with texture Base surface Relief surface

  28. Summary • MRF methods can be extended in the volumetric domain • Advantages • General surfaces can be represented • Optimisation is tractable (MRF solvers) • Smoothness is viewpoint independent

  29. Future work • Photoconsistency beyond Lambertian surface models. (Optimise both height and surface normal fields) • Change in topology • In cases where Cmn(hm,hn)=|| hm-hn||or || hm-hn||2 we can compute messages in O(L) time instead of O(L2) (Felzenszwalb & Huttenlocher CVPR 04).

More Related