2006 11 3 fri young ki baik computer vision lab n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
3D Surface Reconstruction from 2D Images (Survey) PowerPoint Presentation
Download Presentation
3D Surface Reconstruction from 2D Images (Survey)

Loading in 2 Seconds...

play fullscreen
1 / 33

3D Surface Reconstruction from 2D Images (Survey) - PowerPoint PPT Presentation


  • 291 Views
  • Uploaded on

2006. 11. 3 (Fri) Young Ki Baik, Computer Vision Lab. . 3D Surface Reconstruction from 2D Images (Survey). 3D Surface Reconstruction from 2D Images. References A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms Steven M. Seitz, Richard Szeliski et. al. (CVPR 2006)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '3D Surface Reconstruction from 2D Images (Survey)' - carolina


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
3d surface reconstruction from 2d images
3D Surface Reconstruction from 2D Images
  • References
    • A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
      • Steven M. Seitz, Richard Szeliski et. al. (CVPR 2006)
    • A Survey of Methods for Volumetric Scene Reconstruction
      • Greg Slabaugh et. al. (VG 2001) : Volume Graphics
3d surface reconstruction from 2d images1
3D Surface Reconstruction from 2D Images
  • Contents
    • Introduction
    • Camera calibration
    • Shape from 2D images, techniques
    • Conclusion
3d surface reconstruction from 2d images2
3D Surface Reconstruction from 2D Images
  • Introduction
    • Volumetric data representations
      • Gaining importance since their introduction in the early 70’s

→3D medical imaging [Greenleaf 70]

3d surface reconstruction from 2d images3
3D Surface Reconstruction from 2D Images
  • Introduction
    • Volumetric data representation
      • The exponential growth of computational storage and processing

→practical alternatives to surface based geometrical representation for many applications in computer graphics and scientific visualization

3d surface reconstruction from 2d images4
3D Surface Reconstruction from 2D Images
  • Application
    • Reverse engineering
    • Augmented reality
    • Human computer interaction
    • Animation, Game
    • Etc.
3d surface reconstruction from 2d images5
3D Surface Reconstruction from 2D Images
  • Methods for volumetric reconstruction
    • By hand
      • 3D tool (3DMAX, MAYA, …)
      • Limitation
        • Too much time, tedious
3d surface reconstruction from 2d images6
3D Surface Reconstruction from 2D Images
  • Methods for volumetric reconstruction
    • Laser scanner (Range data)
      • Advantage
        • High quality and accuracy
      • Limitation
        • Too much money
        • Specific configuration
3d surface reconstruction from 2d images7
3D Surface Reconstruction from 2D Images
  • Methods for volumetric reconstruction
    • CCD camera (Images)
      • Advantage
        • Cheap price
        • Usefulness
3d surface reconstruction from 2d images8
3D Surface Reconstruction from 2D Images
  • Condition of 3D reconstruction

3D point

3D object

mapping

Image plane

Camera

Camera

Camera system for obtaining images

3d surface reconstruction from 2d images9
3D Surface Reconstruction from 2D Images
  • Condition of 3D reconstruction
  • Point correspondence
  • Camera parameter and motion

3D point

3D object

Camera

Camera

3D reconstruction system to make 3D object

3d surface reconstruction from 2d images10
3D Surface Reconstruction from 2D Images
  • Camera Calibration
    • Camera parameters:
      • Extrinsic: Translation T, Rotation R.
      • Intrinsic: Focal Length f, image center (ox ,oy), effective pixel size (sx ,sy), radial distortion k.
    • Recover parameters from 3D points and their projections.

Object

View9

View1

View6

Camera Motion

3d surface reconstruction from 2d images11
3D Surface Reconstruction from 2D Images
  • Camera Calibration
    • Simple overall flow
  • Camera is set fixed location.
  • Obtaining camera parameters using projected 2D image and world 3D data with Known plane or 3D rig.
  • We can acquire 3D volumetric representation by applying various reconstruction algorithm.
3d surface reconstruction from 2d images12
3D Surface Reconstruction from 2D Images
  • Camera Calibration
    • Calibration with pattern:
      • Tsai’s method [Tsai87]
      • Zhang’s method [Zhang00]
    • Self-calibration [Maybank92]
    • Bundle adjustment [Triggs], evenly distribute errors.
3d surface reconstruction from 2d images13
3D Surface Reconstruction from 2D Images
  • Shape from 2D Images
    • Shape from silhouette
    • Shape from structured light
    • Shape from Illumination
      • Shape from shading
      • Photometric stereo
    • Shape from Color(or intensity)
      • Voxel coloring
      • Stereo vision
    • Fusion method
3d surface reconstruction from 2d images14
3D Surface Reconstruction from 2D Images
  • Shape from Silhouette(SFS)
    • Early works of vision
    • Effective method
    • Sculpturing a statue
3d surface reconstruction from 2d images15
3D Surface Reconstruction from 2D Images
  • Shape from Silhouette(SFS)

- Back-project each silhouette along the ray

- Obtain 3D volumetric data from intersecting back-projected volume

- Calibrated cameras and object

- Set initial 3D volumetric region including object

O1

O2

O3

3d surface reconstruction from 2d images16
3D Surface Reconstruction from 2D Images
  • Shape from Silhouette(SFS)
    • Advantages:
      • Simple to implement and fairly robust
      • Fast execution
      • complete closed surface

→ commonly used as the effective initial boundary

    • Limitations:
      • only produced line hull
      • can’t detect non-convex region
      • sensitive to segmentation result

→ specific color is used as the background

3d surface reconstruction from 2d images17
3D Surface Reconstruction from 2D Images
  • Shape from Structured Light
    • Shape from Structured Light
      • Rays coming out of light source hit the object surface and captured by image sensor (usually a camera) in a different angle. [Levoy00, Allen03]
  • Problem
  • - Optically uncooperative materials
  • - Scanning in the presence of occlusion
  • - Filling holes in dense polygon models
3d surface reconstruction from 2d images18
3D Surface Reconstruction from 2D Images
  • Shape from Illumination
    • Shape from Shading
      • Assume distance point light source, orthographic projection, local shading and Lambertian surface
      • Given image intensity, determine depth by solving reflectance map in the fields of Radiometry.
  • Lambertian
  • A surface point is equally bright from all directions.
  • Limitation
  • - Do not provide qualified results
3d surface reconstruction from 2d images19
3D Surface Reconstruction from 2D Images
  • Shape from Illumination
    • Photometric stereo
      • Advanced version of shape from shading
      • Method to determine surface shape using multiple images taken by varying illumination direction, while fixed camera position
  • Advantage
  • - Provide good results relative to shape from shading
  • Limitation
  • - Have to know the location of light sources
3d surface reconstruction from 2d images20

Sees blue

Sees blue

3D Surface Reconstruction from 2D Images
  • Shape from Color
    • Voxel Coloring
      • Images can be constraints on 3D scene:
        • a valid 3D scene model projected must produce synthetic images same as the corresponding real input images.
      • SFS+color consistency
      • Opaque or not

Sees red

Sees green

3d surface reconstruction from 2d images21
3D Surface Reconstruction from 2D Images
  • Shape from Color
    • Voxel Coloring (overall flow)

- Place the object to the fixed location.

- Set the camera on the fixed location.

- Set up voxel region covering object.

3d surface reconstruction from 2d images22
3D Surface Reconstruction from 2D Images
  • Shape from Color
    • Voxel Coloring (overall flow)

- Iterate this algorithm about all voxel in the region.

- Select a voxel and project onto the each image.

- Judge opaqueness by thresholding variance of colors.

3d surface reconstruction from 2d images23
3D Surface Reconstruction from 2D Images
  • Voxel Coloring
    • Advantages:
      • simple to implement and fairly robust
    • Limitations:
      • performance depends on voxel and image resolution.

→reconstruct object in small area

→high computational cost

      • occlusion and illumination problem
3d surface reconstruction from 2d images24
3D Surface Reconstruction from 2D Images
  • Voxel coloring
    • More advanced algorithm
      • Space carving
      • Generalized voxel coloring
      • Multi-hypothesis voxel coloring
3d surface reconstruction from 2d images25

Scene object point

Epipolar plane

Optical axes

Epipolar lines

Image plane

Left Camera

Left Camera

3D Surface Reconstruction from 2D Images
  • Shape from color
    • Stereo vision
      • Features
        • popular method
        • pixel based method
        • mimics the behavior of human vision
        • apply feature matching criterion at all pixels simultaneously
        • search only over epipolar lines

(fewer candidate positions)

3d surface reconstruction from 2d images26
3D Surface Reconstruction from 2D Images
  • Shape from color
    • Stereo vision
      • Matching cost
        • Squared Intensity Differences (SD,SSD).
        • Absolutely Intensity Differences (AD,MSE).
        • Normalized Cross-correlation, normalized SSD.
3d surface reconstruction from 2d images27
3D Surface Reconstruction from 2D Images
  • Stereo vision
    • Advantages
      • gives detailed surface estimates
      • covering wide area object
        • Building, topography
      • Fast execution
      • multi-view aggregation improves accuracy
3d surface reconstruction from 2d images28
3D Surface Reconstruction from 2D Images
  • Stereo vision
    • Limitation
      • narrow baseline give rise to noisy estimates
      • fails in ~
        • textureless and occlusion areas
        • sparse, in complete surface
      • sensitive to non-Lambertian effects.
    • Other effective methods
      • http://cat.middlebury.edu/stereo/
3d surface reconstruction from 2d images29
3D Surface Reconstruction from 2D Images
  • Fusion methods
    • Silhouette and Stereo Fusion for 3D Object Modeling

- Carlos Hernandez Esteban and Francis Schmitt (CVIU 2004)

# SFS +multi +stereo correlation voting +Gradient vector flow +Snake

    • High-Fidelity Image-Based Modeling

- Yasutaka Furukawa, Jean Ponce (CVR-TR-2006)

# SFS + wide baseline matching + propagation + Energy minimization

    • Multi-View Stereo Revisited

- Michael Goesele et. al. (CVPR2006)

# SFS +Stereo matching + volumetric method (range data + Level Set)

    • MultiView Geometry for Texture mapping 2D Images Onto 3D Range Data

- Lingyoun Liu et. Al. (CVPR2006)

# SFS +Stereo matching + volumetric method (range data)

3d surface reconstruction from 2d images30
3D Surface Reconstruction from 2D Images
  • Trend of 3D reconstruction method
    • ~1995 Simple algorithm
    • ~1999 VC and variants (treating occlusion),

LevelSet, optimization

    • ~2001 Probabilistic formulation
    • ~2003 Non-Lambatian surface, specular surface,

textureless regions

    • ~2006 PGM, Fusion method

(SFS + Stereo + Level set + …)

3d surface reconstruction from 2d images31
3D Surface Reconstruction from 2D Images
  • Conclusion
    • Survey of methods for volumetric scene reconstruction from photographs.
    • States of the arts shows very good reconstruction results.
    • All algorithm do not solve problems yet.
      • occlusion ,illumination changes
      • non-Lambatian surface
      • Real data (no silhouette)
    • There is room for improvement