- 135 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' CSCE 641 Computer Graphics: Image-based Modeling' - laasya

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### CSCE 641 Computer Graphics: Image-based Modeling

Jinxiang Chai

Image-based modeling

- Estimating 3D structure
- Estimating motion, e.g., camera motion
- Estimating lighting
- Estimating surface model

Traditional modeling and rendering

Geometry Reflectance Light source Camera model

rendering

modeling

User inputTexture map survey data

Images

For photorealism:

- Modeling is hard

- Rendering is slow

What do we want to do for this model?

Image based modeling and rendering

Image-based modeling

Image-based rendering

Imagesuser input range scans

Model

Images

Spectrum of IBMR

Model

Panoroma

Image-based rendering

Image based modeling

Images + Depth

Geometry+ Images

Camera + geometry

Imagesuser input range scans

Images

Geometry+ Materials

Light field

Kinematics

Dynamics

Etc.

Spectrum of IBMR

Model

Panoroma

Image-based rendering

Image based modeling

Images + Depth

Geometry+ Images

Camera + geometry

Imagesuser input range scans

Images

Geometry+ Materials

Light field

Kinematics

Dynamics

Etc.

Spectrum of IBMR

Model

Panoroma

Image-based rendering

Image based modeling

Images + Depth

Geometry+ Images

Camera + geometry

Imagesuser input range scans

Images

Geometry+ Materials

Light field

Kinematics

Dynamics

Etc.

Stereo reconstruction

- Given two or more images of the same scene or object, compute a representation of its shape
- What are some possible applications?

known

camera

viewpoints

3D modeling

- From one stereo pair to a 3D head model
- [Frederic Deverney, INRIA]

3D modeling

The Digital Michelangelo Project, Levoy et al.

Optical mocap

Vicon mocap system

Z-keying: mix live and synthetic

- Takeo Kanade, CMU (Stereo Machine)

Virtualized RealityTM

- [Takeo Kanade et al., CMU]
- collect video from 50+ stream
- reconstruct 3D model sequences
- steerable version used forSuperBowl XXV “eye vision”
- http://www.cs.cmu.edu/afs/cs/project/VirtualizedR/www/VirtualizedR.html

View interpolation

- input depth image novel view
- [Szeliski & Kang ‘95]

View morphing

- Morph between pair of images using epipolar geometry [Seitz & Dyer, SIGGRAPH’96]

Performance Interface

- Microsoft Natal project

Additional applications?

- Real-time people tracking (systems from Pt. Gray Research and SRI)
- “Gaze” correction for video conferencing [Ott,Lewis,Cox InterChi’93]
- Other ideas?

Stereo matching

- Given two or more images of the same scene or object, compute a representation of its shape
- What are some possible representations for shapes?
- depth maps
- volumetric models
- 3D surface models
- planar (or offset) layers

Outline

- Stereo matching
- - Traditional stereo
- - Multi-baseline stereo
- - Active stereo
- Volumetric stereo
- - Visual hull
- - Voxel coloring
- - Space carving

Papers

- Stereo matching
- Masatoshi Okutomi and Takeo Kanade. A multiple-baseline stereo. IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI), 15(4), 1993, pp. 353--363.
- D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms.International Journal of Computer Vision, 47(1/2/3):7-42, April-June 2002.
- Visual-hull reconstruction
- Szeliski, “Rapid Octree Construction from Image Sequences”, Computer Vision, Graphics, and Image Processing: Image Understanding, 58(1), 1993, pp. 23-32.
- Matusik, Buehler, Raskar, McMillan, and Gortler , “Image-Based Visual Hulls”, Proc. SIGGRAPH 2000, pp. 369-374.
- Photo-hull reconstruction
- Seitz & Dyer, “Photorealistic Scene Reconstruction by Voxel Coloring”, Intl. Journal of Computer Vision (IJCV), 1999, 35(2), pp. 151-173.
- Kutulakos & Seitz, “A Theory of Shape by Space Carving”, International Journal of Computer Vision, 2000, 38(3), pp. 199-218.

Stereo

- Basic Principle: Triangulation
- Gives reconstruction as intersection of two rays

- Requires
- calibration
- point correspondence

Camera calibration

- From world coordinate to image coordinate

Perspective projection

View transformation

Viewport projection

u

sx

a

u0

v

0

-sy

v0

1

0

0

1

2D projections

3D points

Camera parameters

epipolar line

Stereo correspondence- Determine Pixel Correspondence
- Pairs of points that correspond to same scene point

epipolar line

- Epipolar Constraint
- Reduces correspondence problem to 1D search along conjugateepipolar lines
- Java demo: http://www.ai.sri.com/~luong/research/Meta3DViewer/EpipolarGeo.html

Stereo image rectification

- reproject image planes onto a common
- plane parallel to the line between optical centers
- pixel motion is horizontal after this transformation
- two homographies (3x3 transform), one for each input image reprojection
- C. Loop and Z. Zhang. Computing Rectifying Homographies for Stereo Vision. IEEE Conf. Computer Vision and Pattern Recognition, 1999.

Stereo matching algorithms

- Match Pixels in Conjugate Epipolar Lines
- Assume brightness constancy
- This is a tough problem
- Numerous approaches
- A good survey and evaluation: http://www.middlebury.edu/stereo/

For each pixel in the left image

- Improvement: match windows
- This should look familiar..
- Can use Lukas-Kanade or discrete search (latter more common)

- compare with every pixel on same epipolar line in right image

- pick pixel with minimum matching cost

Stereo results

- Data from University of Tsukuba
- Similar results on other images without ground truth

Scene

Ground truth

Better methods exist...

- State of the art method
- Boykov et al., Fast Approximate Energy Minimization via Graph Cuts,
- International Conference on Computer Vision, September 1999.

Ground truth

Stereo reconstruction pipeline

- Steps
- Calibrate cameras
- Rectify images
- Compute disparity
- Estimate depth

Stereo reconstruction pipeline

- Steps
- Calibrate cameras
- Rectify images
- Compute disparity
- Estimate depth

- Camera calibration errors
- Poor image resolution
- Occlusions
- Violations of brightness constancy (specular reflections)
- Large motions
- Low-contrast image regions

- What will cause errors?

Outline

- Stereo matching
- - Traditional stereo
- - Multi-baseline stereo
- - Active stereo
- Volumetric stereo
- - Visual hull
- - Voxel coloring
- - Space carving

Choosing the stereo baseline

- What’s the optimal baseline?
- Too small: large depth error
- Too large: difficult search problem

all of these

points project

to the same

pair of pixels

width of

a pixel

Large Baseline

Small Baseline

Multi-baseline stereo

- Basic Approach
- Choose a reference view
- Use your favorite stereo algorithm BUT
- replace two-view SSD with SSD over all baselines
- Limitations
- Must choose a reference view (bad)
- Visibility!
- CMU’s 3D Room Video

Outline

- Stereo matching
- - Traditional stereo
- - Multi-baseline stereo
- - Active stereo
- Volumetric stereo
- - Visual hull
- - Voxel coloring
- - Space carving

camera 1

projector

projector

camera 2

Active stereo with structured light- Project “structured” light patterns onto the object
- simplifies the correspondence problem

Li Zhang’s one-shot stereo

Laser scanning

- Optical triangulation
- Project a single stripe of laser light
- Scan it across the surface of the object
- This is a very precise version of structured light scanning

- Digital Michelangelo Project
- http://graphics.stanford.edu/projects/mich/

Laser scanned models

The Digital Michelangelo Project, Levoy et al.

Laser scanned models

The Digital Michelangelo Project, Levoy et al.

Desktop scanner

- Convenient to use
- Good quality
- Relatively low-cost
- - next engine (about 2k)

Download Presentation

Connecting to Server..