# Multi-view Stereo via Volumetric Graph-cuts - PowerPoint PPT Presentation

Multi-view Stereo via Volumetric Graph-cuts

1 / 29
Multi-view Stereo via Volumetric Graph-cuts

## Multi-view Stereo via Volumetric Graph-cuts

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
##### Presentation Transcript

1. Multi-view Stereo via Volumetric Graph-cuts George Vogiatzis, Philip H. S. Torr Roberto Cipolla

2. Shape From Images

3. Dense Stereo reconstruction problem: • Input • Set of images of a scene I={I1,…,IK} • Camera matrices P1,…,PK • Output • Surface model

4. Shape parametrisation • Disparity-map parameterisation • MRF formulation – good optimisation techniques exist (Graph-cuts, Loopy BP) • MRF smoothness is viewpoint dependent • Disparity is unique per pixel – only functions represented

5. Shape parametrisation • Volumetric parameterisation – e.g. Level-sets, Space carving etc. • Able to cope with non-functions • Convergence properties not well understood • Memory intensive • For Space carving, no simple way to impose surface smoothness

6. Solution ? • Cast volumetric methods in MRF framework • Benefits: • General surfaces can be represented • Optimisation is tractable (MRF solvers) • Occlusions can be approximately modelled • Smoothness is viewpoint independent

7. Graph cuts 24 5 40 21 23 20 5 12 1 13 50 40 4 13 3 30

8. Graph cuts 24 5 40 21 23 20 5 12 1 13 50 40 4 13 3 30

9. Graph cuts 24 5 40 21 23 20 5 12 1 13 50 40 4 13 3 30 5+5+1+4+3=18

10. Volumetric Graph cuts for segmentation • Volume is discretized • A binary MRF is defined on the voxels • Regular grid (6 or 26 neighbourhood) • Voxels are labelled as OBJECT and BACKGROUND • Labelling cost set by OBJECT / BACKGROUND intensity statistics • Compatibility cost set by edge intensities

11. Volumetric Graph cuts for stereo • How to define ‘Inside’ and ‘Outside’ labels • How to deal with occlusion

12. Sink Source Min cut Volumetric Graph cuts

13. Face

14. Face - Visual Hull

15. Face - Slice

16. Face - Slice with graphcut

17. Face - Reconstruction

18. Protrusion problem • ‘Balooning’ force • favouring bigger volumes L.D. Cohen and I. Cohen. Finite-element methods for active contour models and balloons for 2-d and 3-d images. PAMI, 15(11):1131–1147, November 1993.

19. Protrusion problem • ‘Balooning’ force • favouring bigger volumes L.D. Cohen and I. Cohen. Finite-element methods for active contour models and balloons for 2-d and 3-d images. PAMI, 15(11):1131–1147, November 1993.

20. Protrusion problem

21. Protrusion problem

22. Graph

23. Results • Model House

24. Results • Model House – Visual Hull

25. Results • Model House

26. Results • Stone carving

27. Results • Haniwa

28. Summary • Novel formulation for multiview stereo • Volumetric scene representation • Computationally tractable global optimisation using Graph-cuts. • Visual hull for occlusions and geometric constraint

29. Benefits • General surfaces and objects can be fully represented and computed as a single surface. • The representation and smoothness constraint is image and viewpoint independent. • Multiple views of the scene can be used with occlusions approximately modelled. • Optimisation is computationally tractable, using existing max-flow algorithms.