Multi-view Stereo via Volumetric Graph-cuts - PowerPoint PPT Presentation

multi view stereo via volumetric graph cuts n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Multi-view Stereo via Volumetric Graph-cuts PowerPoint Presentation
Download Presentation
Multi-view Stereo via Volumetric Graph-cuts

play fullscreen
1 / 29
Multi-view Stereo via Volumetric Graph-cuts
115 Views
Download Presentation
hedley-ellison
Download Presentation

Multi-view Stereo via Volumetric Graph-cuts

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Multi-view Stereo via Volumetric Graph-cuts George Vogiatzis, Philip H. S. Torr Roberto Cipolla

  2. Shape From Images

  3. Dense Stereo reconstruction problem: • Input • Set of images of a scene I={I1,…,IK} • Camera matrices P1,…,PK • Output • Surface model

  4. Shape parametrisation • Disparity-map parameterisation • MRF formulation – good optimisation techniques exist (Graph-cuts, Loopy BP) • MRF smoothness is viewpoint dependent • Disparity is unique per pixel – only functions represented

  5. Shape parametrisation • Volumetric parameterisation – e.g. Level-sets, Space carving etc. • Able to cope with non-functions • Convergence properties not well understood • Memory intensive • For Space carving, no simple way to impose surface smoothness

  6. Solution ? • Cast volumetric methods in MRF framework • Benefits: • General surfaces can be represented • Optimisation is tractable (MRF solvers) • Occlusions can be approximately modelled • Smoothness is viewpoint independent

  7. Graph cuts 24 5 40 21 23 20 5 12 1 13 50 40 4 13 3 30

  8. Graph cuts 24 5 40 21 23 20 5 12 1 13 50 40 4 13 3 30

  9. Graph cuts 24 5 40 21 23 20 5 12 1 13 50 40 4 13 3 30 5+5+1+4+3=18

  10. Volumetric Graph cuts for segmentation • Volume is discretized • A binary MRF is defined on the voxels • Regular grid (6 or 26 neighbourhood) • Voxels are labelled as OBJECT and BACKGROUND • Labelling cost set by OBJECT / BACKGROUND intensity statistics • Compatibility cost set by edge intensities

  11. Volumetric Graph cuts for stereo • How to define ‘Inside’ and ‘Outside’ labels • How to deal with occlusion

  12. Sink Source Min cut Volumetric Graph cuts

  13. Face

  14. Face - Visual Hull

  15. Face - Slice

  16. Face - Slice with graphcut

  17. Face - Reconstruction

  18. Protrusion problem • ‘Balooning’ force • favouring bigger volumes L.D. Cohen and I. Cohen. Finite-element methods for active contour models and balloons for 2-d and 3-d images. PAMI, 15(11):1131–1147, November 1993.

  19. Protrusion problem • ‘Balooning’ force • favouring bigger volumes L.D. Cohen and I. Cohen. Finite-element methods for active contour models and balloons for 2-d and 3-d images. PAMI, 15(11):1131–1147, November 1993.

  20. Protrusion problem

  21. Protrusion problem

  22. Graph

  23. Results • Model House

  24. Results • Model House – Visual Hull

  25. Results • Model House

  26. Results • Stone carving

  27. Results • Haniwa

  28. Summary • Novel formulation for multiview stereo • Volumetric scene representation • Computationally tractable global optimisation using Graph-cuts. • Visual hull for occlusions and geometric constraint

  29. Benefits • General surfaces and objects can be fully represented and computed as a single surface. • The representation and smoothness constraint is image and viewpoint independent. • Multiple views of the scene can be used with occlusions approximately modelled. • Optimisation is computationally tractable, using existing max-flow algorithms.