Simultaneous super-resolution and 3D video using graph-cuts - PowerPoint PPT Presentation

simultaneous super resolution and 3d video using graph cuts n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Simultaneous super-resolution and 3D video using graph-cuts PowerPoint Presentation
Download Presentation
Simultaneous super-resolution and 3D video using graph-cuts

play fullscreen
1 / 12
Simultaneous super-resolution and 3D video using graph-cuts
149 Views
Download Presentation
arav
Download Presentation

Simultaneous super-resolution and 3D video using graph-cuts

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Simultaneous super-resolution and 3D video using graph-cuts Tony Tung ShoheiNobuhara Takashi Matsuyama Graduate School of Informatics, Kyoto University, Japan CVPR 2008

  2. Outline • Introduction • Super-resolution on multi-view images • Super-resolution on single-view video frames • Simultaneous super-resolution and 3D video • Results • Conclusion

  3. Introduction • In 3D video • models are in motion • limited number of cameras • details cannot always be recovered • mapped textures may lack of precision • This paper • Super-resolution(SR) for video frames • Markov Random Field(MRF) energy formulation • Graph-cut • Coarse-to-fine strategy

  4. Super-resolution on multi-view images • Super-resolution (SR) • to recover detailed information from degraded data • one can reconstruct high resolution (HR) images from several low resolution (LR) images • SR image reconstruction method • 1) magnifying every image from LR to HR • 2) accurate alignments of LR images onto HR grids to gain subpixel information in each HR image • 3) HR image regularization using MRF energy formulation and minimization with graph-cuts

  5. MRF energy formulation • enable discontinuity preservation and image regularization using graph-cuts • one energy is formulated for each R, G, B channel

  6. Super-resolution on single-view video frames • Feature matching • tracking regions of interest(ROI) in consecutive frames • the Scale Invariant Feature Transforms (SIFT) detector • Texture warping and mapping • create a mesh based on the detected features • ROI is then the mesh texture

  7. Simultaneous super-resolution and 3D video • Coarse-to-finestrategy

  8. Simultaneous energy minimization • the global compound energy • simultaneous SR • 3D shape reconstruction using graph-cuts

  9. Results • The studio • diameter is 6 m, height is 2.5 m • an object can be reconstructed without defect is approximately 3m*3m*2m • PC cluster system composed of 15 PCs and one master PC • multi-viewpoint images at 25 fps • Japanese dance performed by maikos • wears a kimono

  10. 3D model reconstruction is challenging • cloths are large • non-rigid and contain lots of details to recover • The whole pipeline is fully automatized • The most expensive step is the SR energy minimization

  11. Conclusion and further work • 3D video is a new media developed these last years to represent 3D objects in motion • Super-resolution methods dedicated to 3D video • May be combined to a 3D video compression technique • Could tune the coarse-to-finestrategy to update iteratively the texture map