2006 11 3 fri young ki baik computer vision lab n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
3D Surface Reconstruction from 2D Images (Survey) PowerPoint Presentation
Download Presentation
3D Surface Reconstruction from 2D Images (Survey)

Loading in 2 Seconds...

play fullscreen
1 / 33

3D Surface Reconstruction from 2D Images (Survey) - PowerPoint PPT Presentation


  • 309 Views
  • Uploaded on

2006. 11. 3 (Fri) Young Ki Baik, Computer Vision Lab. . 3D Surface Reconstruction from 2D Images (Survey). 3D Surface Reconstruction from 2D Images. References A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms Steven M. Seitz, Richard Szeliski et. al. (CVPR 2006)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

3D Surface Reconstruction from 2D Images (Survey)


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
3d surface reconstruction from 2d images
3D Surface Reconstruction from 2D Images
  • References
    • A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
      • Steven M. Seitz, Richard Szeliski et. al. (CVPR 2006)
    • A Survey of Methods for Volumetric Scene Reconstruction
      • Greg Slabaugh et. al. (VG 2001) : Volume Graphics
3d surface reconstruction from 2d images1
3D Surface Reconstruction from 2D Images
  • Contents
    • Introduction
    • Camera calibration
    • Shape from 2D images, techniques
    • Conclusion
3d surface reconstruction from 2d images2
3D Surface Reconstruction from 2D Images
  • Introduction
    • Volumetric data representations
      • Gaining importance since their introduction in the early 70’s

→3D medical imaging [Greenleaf 70]

3d surface reconstruction from 2d images3
3D Surface Reconstruction from 2D Images
  • Introduction
    • Volumetric data representation
      • The exponential growth of computational storage and processing

→practical alternatives to surface based geometrical representation for many applications in computer graphics and scientific visualization

3d surface reconstruction from 2d images4
3D Surface Reconstruction from 2D Images
  • Application
    • Reverse engineering
    • Augmented reality
    • Human computer interaction
    • Animation, Game
    • Etc.
3d surface reconstruction from 2d images5
3D Surface Reconstruction from 2D Images
  • Methods for volumetric reconstruction
    • By hand
      • 3D tool (3DMAX, MAYA, …)
      • Limitation
        • Too much time, tedious
3d surface reconstruction from 2d images6
3D Surface Reconstruction from 2D Images
  • Methods for volumetric reconstruction
    • Laser scanner (Range data)
      • Advantage
        • High quality and accuracy
      • Limitation
        • Too much money
        • Specific configuration
3d surface reconstruction from 2d images7
3D Surface Reconstruction from 2D Images
  • Methods for volumetric reconstruction
    • CCD camera (Images)
      • Advantage
        • Cheap price
        • Usefulness
3d surface reconstruction from 2d images8
3D Surface Reconstruction from 2D Images
  • Condition of 3D reconstruction

3D point

3D object

mapping

Image plane

Camera

Camera

Camera system for obtaining images

3d surface reconstruction from 2d images9
3D Surface Reconstruction from 2D Images
  • Condition of 3D reconstruction
  • Point correspondence
  • Camera parameter and motion

3D point

3D object

Camera

Camera

3D reconstruction system to make 3D object

3d surface reconstruction from 2d images10
3D Surface Reconstruction from 2D Images
  • Camera Calibration
    • Camera parameters:
      • Extrinsic: Translation T, Rotation R.
      • Intrinsic: Focal Length f, image center (ox ,oy), effective pixel size (sx ,sy), radial distortion k.
    • Recover parameters from 3D points and their projections.

Object

View9

View1

View6

Camera Motion

3d surface reconstruction from 2d images11
3D Surface Reconstruction from 2D Images
  • Camera Calibration
    • Simple overall flow
  • Camera is set fixed location.
  • Obtaining camera parameters using projected 2D image and world 3D data with Known plane or 3D rig.
  • We can acquire 3D volumetric representation by applying various reconstruction algorithm.
3d surface reconstruction from 2d images12
3D Surface Reconstruction from 2D Images
  • Camera Calibration
    • Calibration with pattern:
      • Tsai’s method [Tsai87]
      • Zhang’s method [Zhang00]
    • Self-calibration [Maybank92]
    • Bundle adjustment [Triggs], evenly distribute errors.
3d surface reconstruction from 2d images13
3D Surface Reconstruction from 2D Images
  • Shape from 2D Images
    • Shape from silhouette
    • Shape from structured light
    • Shape from Illumination
      • Shape from shading
      • Photometric stereo
    • Shape from Color(or intensity)
      • Voxel coloring
      • Stereo vision
    • Fusion method
3d surface reconstruction from 2d images14
3D Surface Reconstruction from 2D Images
  • Shape from Silhouette(SFS)
    • Early works of vision
    • Effective method
    • Sculpturing a statue
3d surface reconstruction from 2d images15
3D Surface Reconstruction from 2D Images
  • Shape from Silhouette(SFS)

- Back-project each silhouette along the ray

- Obtain 3D volumetric data from intersecting back-projected volume

- Calibrated cameras and object

- Set initial 3D volumetric region including object

O1

O2

O3

3d surface reconstruction from 2d images16
3D Surface Reconstruction from 2D Images
  • Shape from Silhouette(SFS)
    • Advantages:
      • Simple to implement and fairly robust
      • Fast execution
      • complete closed surface

→ commonly used as the effective initial boundary

    • Limitations:
      • only produced line hull
      • can’t detect non-convex region
      • sensitive to segmentation result

→ specific color is used as the background

3d surface reconstruction from 2d images17
3D Surface Reconstruction from 2D Images
  • Shape from Structured Light
    • Shape from Structured Light
      • Rays coming out of light source hit the object surface and captured by image sensor (usually a camera) in a different angle. [Levoy00, Allen03]
  • Problem
  • - Optically uncooperative materials
  • - Scanning in the presence of occlusion
  • - Filling holes in dense polygon models
3d surface reconstruction from 2d images18
3D Surface Reconstruction from 2D Images
  • Shape from Illumination
    • Shape from Shading
      • Assume distance point light source, orthographic projection, local shading and Lambertian surface
      • Given image intensity, determine depth by solving reflectance map in the fields of Radiometry.
  • Lambertian
  • A surface point is equally bright from all directions.
  • Limitation
  • - Do not provide qualified results
3d surface reconstruction from 2d images19
3D Surface Reconstruction from 2D Images
  • Shape from Illumination
    • Photometric stereo
      • Advanced version of shape from shading
      • Method to determine surface shape using multiple images taken by varying illumination direction, while fixed camera position
  • Advantage
  • - Provide good results relative to shape from shading
  • Limitation
  • - Have to know the location of light sources
3d surface reconstruction from 2d images20

Sees blue

Sees blue

3D Surface Reconstruction from 2D Images
  • Shape from Color
    • Voxel Coloring
      • Images can be constraints on 3D scene:
        • a valid 3D scene model projected must produce synthetic images same as the corresponding real input images.
      • SFS+color consistency
      • Opaque or not

Sees red

Sees green

3d surface reconstruction from 2d images21
3D Surface Reconstruction from 2D Images
  • Shape from Color
    • Voxel Coloring (overall flow)

- Place the object to the fixed location.

- Set the camera on the fixed location.

- Set up voxel region covering object.

3d surface reconstruction from 2d images22
3D Surface Reconstruction from 2D Images
  • Shape from Color
    • Voxel Coloring (overall flow)

- Iterate this algorithm about all voxel in the region.

- Select a voxel and project onto the each image.

- Judge opaqueness by thresholding variance of colors.

3d surface reconstruction from 2d images23
3D Surface Reconstruction from 2D Images
  • Voxel Coloring
    • Advantages:
      • simple to implement and fairly robust
    • Limitations:
      • performance depends on voxel and image resolution.

→reconstruct object in small area

→high computational cost

      • occlusion and illumination problem
3d surface reconstruction from 2d images24
3D Surface Reconstruction from 2D Images
  • Voxel coloring
    • More advanced algorithm
      • Space carving
      • Generalized voxel coloring
      • Multi-hypothesis voxel coloring
3d surface reconstruction from 2d images25

Scene object point

Epipolar plane

Optical axes

Epipolar lines

Image plane

Left Camera

Left Camera

3D Surface Reconstruction from 2D Images
  • Shape from color
    • Stereo vision
      • Features
        • popular method
        • pixel based method
        • mimics the behavior of human vision
        • apply feature matching criterion at all pixels simultaneously
        • search only over epipolar lines

(fewer candidate positions)

3d surface reconstruction from 2d images26
3D Surface Reconstruction from 2D Images
  • Shape from color
    • Stereo vision
      • Matching cost
        • Squared Intensity Differences (SD,SSD).
        • Absolutely Intensity Differences (AD,MSE).
        • Normalized Cross-correlation, normalized SSD.
3d surface reconstruction from 2d images27
3D Surface Reconstruction from 2D Images
  • Stereo vision
    • Advantages
      • gives detailed surface estimates
      • covering wide area object
        • Building, topography
      • Fast execution
      • multi-view aggregation improves accuracy
3d surface reconstruction from 2d images28
3D Surface Reconstruction from 2D Images
  • Stereo vision
    • Limitation
      • narrow baseline give rise to noisy estimates
      • fails in ~
        • textureless and occlusion areas
        • sparse, in complete surface
      • sensitive to non-Lambertian effects.
    • Other effective methods
      • http://cat.middlebury.edu/stereo/
3d surface reconstruction from 2d images29
3D Surface Reconstruction from 2D Images
  • Fusion methods
    • Silhouette and Stereo Fusion for 3D Object Modeling

- Carlos Hernandez Esteban and Francis Schmitt (CVIU 2004)

# SFS +multi +stereo correlation voting +Gradient vector flow +Snake

    • High-Fidelity Image-Based Modeling

- Yasutaka Furukawa, Jean Ponce (CVR-TR-2006)

# SFS + wide baseline matching + propagation + Energy minimization

    • Multi-View Stereo Revisited

- Michael Goesele et. al. (CVPR2006)

# SFS +Stereo matching + volumetric method (range data + Level Set)

    • MultiView Geometry for Texture mapping 2D Images Onto 3D Range Data

- Lingyoun Liu et. Al. (CVPR2006)

# SFS +Stereo matching + volumetric method (range data)

3d surface reconstruction from 2d images30
3D Surface Reconstruction from 2D Images
  • Trend of 3D reconstruction method
    • ~1995 Simple algorithm
    • ~1999 VC and variants (treating occlusion),

LevelSet, optimization

    • ~2001 Probabilistic formulation
    • ~2003 Non-Lambatian surface, specular surface,

textureless regions

    • ~2006 PGM, Fusion method

(SFS + Stereo + Level set + …)

3d surface reconstruction from 2d images31
3D Surface Reconstruction from 2D Images
  • Conclusion
    • Survey of methods for volumetric scene reconstruction from photographs.
    • States of the arts shows very good reconstruction results.
    • All algorithm do not solve problems yet.
      • occlusion ,illumination changes
      • non-Lambatian surface
      • Real data (no silhouette)
    • There is room for improvement