3d photography image based model acquisition
Download
1 / 37

3D Photography - PowerPoint PPT Presentation


  • 432 Views
  • Updated On :

3D Photography (Image-based Model Acquisition). Funky Image Goes Here. “Analog” 3D photography !. “3D stereoscopic imaging” been around as long as cameras have Use camera with 2 or more lenses (or stereo attachment) Use stereo viewer to create impression of 3D. Motivation .

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '3D Photography' - mike_john


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
3d photography image based model acquisition l.jpg

3D Photography(Image-based Model Acquisition)

Funky Image Goes Here

Deepak Bandyopadhyay / UNC Chapel Hill


Analog 3d photography l.jpg
“Analog” 3D photography !

  • “3D stereoscopic imaging”

    • been around as long as cameras have

    • Use camera with 2 or more lenses (or stereo attachment)

    • Use stereo viewer to create impression of 3D

Deepak Bandyopadhyay / 258 / 3D Photography


Motivation l.jpg
Motivation

  • Digitizing real world objects

  • Getting realistic models

places

objects

humans

Deepak Bandyopadhyay / 258 / 3D Photography


3d photography definition l.jpg
3D Photography : Definition

  • Sometimes called “3D Scanning”

  • Use cameras and light to capture the shape & appearance of real objects

  • Shape == geometry (point sampling + surface reconstruction + fairing)

  • Appearance == surface attributes (color/texture, material properties, reflectance)

  • Final result = richly detailed model

Deepak Bandyopadhyay / 258 / 3D Photography


Applications in industry l.jpg
Applications in Industry

  • Human body / head / face scans

    • Avatar creation for virtual worlds

    • 3d conferencing

    • medical applications

    • product design

    • Platforms:

      • Cyberware RD3030

      • Others (Geomagic, Metacreations, Cyrax, Geometrix…)

Deepak Bandyopadhyay / 258 / 3D Photography


More applications l.jpg
More applications

  • Historical preservation, dissemination of museum artifacts (Digital Michelangelo, Monticello, …)

  • CAD/CAM (eg. Legacy motorcycle parts scanned by Geomagic for Harley-Davidson).

  • Marketing (models of products on the web)

  • 3D games & simulation

  • Reverse engineering

Deepak Bandyopadhyay / 258 / 3D Photography


Technology overview l.jpg
Technology Overview

  • The Imaging Pipeline

    • Real World

    • Optics

    • Recorder

    • Digitizer

    • Vision & Graphics

Deepak Bandyopadhyay / 258 / 3D Photography


Quick notes on optics l.jpg
Quick Notes on Optics

  • Model lenses with all their properties - aberration, distortion, flare, vignetting etc.

  • We correct for some of these effects (eg. distortion) in the calibration, ignore others.

  • CCD (charged coupled devices) are the most popular recording media.

Deepak Bandyopadhyay / 258 / 3D Photography


Theory passive methods l.jpg
Theory : Passive Methods

  • Stereo pair matching

  • Structure from motion

  • Shape from shading

  • Photometric stereo

Deepak Bandyopadhyay / 258 / 3D Photography


Stereo matching l.jpg
Stereo Matching

  • Stereo Matching Basics

    • Needs two images, like stereoscopy

    • Given correspondence betweenpoints in 2 views, we can find depth by triangulation

    • But correspondence is hard prob!

    • A lot of literature on solving it…

  • Stereo Matching output

    • 3D point cloud

    • Remove outliers and pass through surface reconstructor

Deepak Bandyopadhyay / 258 / 3D Photography


Structure from motion l.jpg
Structure from Motion

  • Camera moving, objects static

  • Compute camera motion and object geometry from motion of image points

  • Assumption -orthographic projn (use telephoto)

  • If: world origin = 3D centroid camera origin = 2D centroidThen: camera translation drops out

Deepak Bandyopadhyay / 258 / 3D Photography


Structure from motion12 l.jpg
Structure from Motion

  • Camera moving, objects static

  • Compute camera motion and object geometry from motion of image points

Deepak Bandyopadhyay / 258 / 3D Photography


Structure from motion13 l.jpg
Structure from Motion

  • Factorization [Tomasi & Kanade, 92]

  • Find M, S using Singular Value Decomposition of W.

SVD gives:

S’S modulo linear transform A. Solve for A using constraints on M.

Deepak Bandyopadhyay / 258 / 3D Photography


More methods l.jpg
More methods

  • Shape from shading, [Horn]

    • Invert Lambert’s Law (L=I k cos )knowing the intensity at image pointto solve for normal

  • Photometric stereo [Woodham]

    • An extension of the above

    • Two or more images under different illumination conditions.

    • Each image provides one normal

    • Three images provide unique solution for a pixel.

Deepak Bandyopadhyay / 258 / 3D Photography


Active sensing l.jpg
Active Sensing

  • Passive methods (eg. stereo matching) suffer from ambiguities - many similar regions in an image correspond to a point in the other.

  • Project known / regular pattern (“structured light”) into scene to disambiguate

  • get precise reconstruction by combining views

    • Laser rangefinder

    • Projectors and imperceptible structured light

Deepak Bandyopadhyay / 258 / 3D Photography


Desktop 3d photography jean yves bouguet pietro perona l.jpg
Desktop 3D PhotographyJean-Yves Bouguet, Pietro Perona

  • An active sensing technique using “weak structured lighting”

  • Need: camera, lamp, chessboard, pencil, stick

  • Idea:

    • Light object with lamp & aim camera at it

    • Move stick around & capture shadow sequence

    • Use image of deformed shadow to calc 3D shape

Deepak Bandyopadhyay / 258 / 3D Photography


Desktop 3d photography jean yves bouguet pietro perona17 l.jpg
Desktop 3D PhotographyJean-Yves Bouguet, Pietro Perona

  • Computation of 3d position from the plane of light source, stick and shadow

Deepak Bandyopadhyay / 258 / 3D Photography


Volumetric methods chevette project debevec 1991 l.jpg
Volumetric MethodsChevette Project, Debevec, 1991

Deepak Bandyopadhyay / 258 / 3D Photography


Voxel models from images l.jpg
Voxel Models from Images

  • When there are 2 colors in the image - use volume intersection [Szeliski 1993]

    • Back-project silhouettes from camera views & intersect

Deepak Bandyopadhyay / 258 / 3D Photography


Voxel models from images20 l.jpg
Voxel Models from Images

  • With more colors but constrained viewpoints, we use voxel coloring [Seitz & Dyer, 1997]

    • Choose a voxel & project to it from all views

    • Color if enough matches

    • Prob - determining visibilityof a point from a view

    • Solution - depth orderedtraversal using a “view indep.d.o.” (dist from separating plane)

Deepak Bandyopadhyay / 258 / 3D Photography


Voxel models from images21 l.jpg
Voxel Models from Images

  • A view-independent depth order may not exist (for some configuration of viewpoints / scene geometry).

  • Use Space Carving [Kutulakos & Seitz, 1998]

    • Computes 3D (voxel) shape from multiple color photos

    • Computes “maximally photo-consistent shape”

      • maximal superset of all 3D shapes that produce the given photos

Deepak Bandyopadhyay / 258 / 3D Photography


Space carving l.jpg
Space Carving

  • Algorithm:

    a) Initialize V to volume containing true scene

    b) For each voxel,

    • check if photo-consistent

    • if not, remove (“carve”) it.

  • Can be shown to converge to maximal photo-consistent scene (union of all photo-consistent scenes).

  • Deepak Bandyopadhyay / 258 / 3D Photography


    Space carving results l.jpg
    Space Carving : Results

    • House walkthru - 24 rendered input views

    • Results best as seen from one of the original views

    Deepak Bandyopadhyay / 258 / 3D Photography


    Modeling from a single view criminisi et al 1999 l.jpg
    Modeling from a single view(Criminisi et al, 1999)

    • Compute 3D affine measurements of the scene from single perspective image

    • Use minimal geom info

      • vanishing line for a pencil ofplanes || to reference plane

      • vanishing point of parallellines along a directionoutside reference plane

    Deepak Bandyopadhyay / 258 / 3D Photography


    Modeling from a single view criminisi et al 199925 l.jpg
    Modeling from a single view(Criminisi et al, 1999)

    • Compute “ratio of parallel distances”

    • Creating a 3D model from a photograph

      • horizontal lines used to compute vanishing line

      • parallel vertical lines used to compute vanishing point

    • Can generate geometrically correct model from a Renaissance painting (with correct perspective)

    Deepak Bandyopadhyay / 258 / 3D Photography


    Extracting color reflectance l.jpg
    Extracting color, reflectance

    • Photographs have lighting/shading effects that we estimate (reflectance function) and compensate for (specular highlight removal) or change (relighting)

    • Work of Paul Debevec & others at Berkeley (acquiring reflectance field)

    • Wood et al at U. Washington (surface light lield for 3D photography)

    Deepak Bandyopadhyay / 258 / 3D Photography


    Surface light field wood et al 2000 l.jpg
    Surface Light Field[Wood et al, 2000]

    • A 4D function on the surface - at surface parameter (u,v), for every direction (,), stores the color.

    • Fixed illumination conditions.

    • Photographs taken from a lot of different directions sample the surface light field.

    • Continuous function (piecewise linear over ,) estimated by pointwise fairing.

    Deepak Bandyopadhyay / 258 / 3D Photography


    Reflectance from photographs yu debevec et al 1999 l.jpg
    Reflectance from Photographs (Yu, Debevec et al, 1999)

    • Estimating reflectance for entire scenes

      • Too general a problem, parameterize thus:

        • Assume surface can be divided into patches

        • Diffuse reflectance function (albedo), varies across a patch

        • Specular reflectance function taken as const across a region

    • Assume known lighting, calib, geometry known

    • Approach - Inverse Global Illumination

      • Estimate BRDF for direct illumination - f(u,v,,)

    Deepak Bandyopadhyay / 258 / 3D Photography


    Reflectance from photographs yu debevec et al 199929 l.jpg
    Reflectance from Photographs (Yu, Debevec et al, 1999)

    • Inverse Global Illumination

      • Known Li (measure), Ii (calc fm known light sources) at every pixel

      • Estimate BRDF for direct illumination - f(u,v,i,i,r,r)

        • Write BRDF as a constant diffuse term and a specular term which is a function of incoming & outgoing  and roughness.

        • Solve for the constants(d, s,)

        • For indirect illumination - estimate the parameters (and indirect illumination coeffs with other patches) iteratively

    Deepak Bandyopadhyay / 258 / 3D Photography


    Case study fa ade debevec taylor malik 1996 l.jpg
    Case study - FaçadeDebevec, Taylor & Malik, 1996

    • Modeling architectural scenes from photographs

    • Not fully automatic (user inputs blocky 3D model)

      • Using blocks leads to fewer params in architectural models

    • User marks corresponding features on photo

    • Computer solves for block size, scale, camera rotation by minimizing error of corresponding features

    • Reprojects textures from the photographs onto the reconstructed model

    Deepak Bandyopadhyay / 258 / 3D Photography


    Arches and surfaces of revolution l.jpg
    Arches andSurfaces of Revolution

    Taj Mahal

    modeled from

    one photograph

    Deepak Bandyopadhyay / 258 / 3D Photography


    Case study digital michelangelo project l.jpg
    Case study - Digital Michelangelo Project

    • 3D scanning of large statues (SIGGRAPH 00)

    • Separate geometry and color scans

      • custom rig : laser scanner & camera mounted concurrently

    • Range scan post-processing

      • Combine range scans from different positions

        • Use volumetric modeling methods (Curless, Levoy 1996)

      • Fill holes using space carving

    Deepak Bandyopadhyay / 258 / 3D Photography


    Case study digital michelangelo project33 l.jpg
    Case study - Digital Michelangelo Project

    • Color scan processing

      • Compensate for ambient lighting

        • subtract image with & without spotlight

      • Subtract out shadows & specularities

      • find surface orientation (inverse lighting computation)

      • convert color to RGB reflectance (acquire light field)

        • Using estimated BRDF of marble

        • modeling subsurface scattering

    Deepak Bandyopadhyay / 258 / 3D Photography


    Digital michelangelo scanning a large object l.jpg

    calibrated motions

    pitch (yellow)

    pan (blue)

    horizontal translation (orange)

    uncalibrated motions

    vertical translation

    remounting the scan head

    moving the entire gantry

    Digital MichelangeloScanning a large object

    Deepak Bandyopadhyay / 258 / 3D Photography


    References l.jpg
    References

    • [Bouguet98] Bouguet, J.-Y., P. Perona. 3D Photography on your Desk. In Proc. ICCV 1998

    • [Bouguet00] Bouguet, J.-Y. Presentation on Desktop 3D Photography, in SIGGRAPH course notes on 3D Photography, 2000

    • [Criminisi99] Criminisi, A., I. Reid and A. Zisserman. Single View Metrology. In Proc. ICCV, pp 434-442, September 1999

    • [Curless96] Curless, B. and M. Levoy. A Volumetric Method for Building Complex Models from Range Images. In Proc. SIGGRAPH 1996

    • [Debevec96] Debevec, P., C. Taylor and J. Malik. Façade - Modeling and Rendering Architectural Scenes from Photographs. In Proc. SIGGRAPH 1996

    • [Debevec00a] Debevec, P. Presentation on the Façade, from SIGGRAPH course notes on 3D Photography, 1999, 2000.

    • [Debevec00b] Debevec, P., T. Hawkins, C. Tchou, H.P.Duiker, W. Sarokin and M. Sagar. Acquiring the Reflectance Field of a Human Face. In Proc. SIGGRAPH 2000.

    Deepak Bandyopadhyay / 258 / 3D Photography


    More references l.jpg
    More References

    • [Horn70] Horn, B.K.P. Shape from Shading : A Method for Obtaining the Shape of a Smooth Opaque Object from One View. Ph.D. Thesis, Dept of EE, MIT, 1970.

    • [Kutulakos98] Kutulakos, K. N. and S. Seitz. A Theory of Shape by Space Carving. URCS TR#692, May 1998, appeared in Proc. ICCV 1999.

    • [Levoy96] Levoy, M. and P. Hanrahan. Light Field Rendering. In Proc. SIGGRAPH 1996.

    • [Levoy00a] Levoy, M., Pulli, K., Curless, B. et al. The Digital Michelangelo Project - 3D Scanning of Large Statues. In Proc. SIGGRAPH 2000.

    • [Levoy00b] Levoy, M. Presentation on the Digital Michelangelo Project, in SIGGRAPH course notes on 3D Photography, 2000.

    • [Seitz97] Seitz & Dyer. Photorealistic Scene Reconstruction by Voxel Coloring. In Proc. CVPR 1997, pp. 1067-1073.

    Deepak Bandyopadhyay / 258 / 3D Photography


    Still more references l.jpg
    Still More References

    • [Seitz00] Seitz, S. SIGGRAPH course notes on 3D photography, 1999, 2000.

    • [Szeliski93] Szeliski, R. Rapid Octree Construction from Image Sequences. CGVIP : Image Understanding, vol. 58, no. 1, pp 23-32, 1993.

    • [Wood00] Wood, D., D. I. Azuma, K. Aldinger, B. Curless, T. Duchamp, D.H. Salesin and W. Stuetzle. Surface Light Fields for 3D Photography. In Proc. SIGGRAPH 2000.

    • [Woodham80] Woodham, R. Photometric Stereo for Determining Surface Orientation from Multiple Images. Journal of Optical Engineering, vol. 19, no. 1, pp 138-144, 1980.

    • [Yu99] Yu, Y., P. Debevec, J. Malik and T. Hawkins. Inverse Global Illumination - Recovering Reflectance Models of Real Scenes from Photographs.

    Deepak Bandyopadhyay / 258 / 3D Photography


    ad