3d photography image based model acquisition
Download
Skip this Video
Download Presentation
3D Photography (Image-based Model Acquisition)

Loading in 2 Seconds...

play fullscreen
1 / 37

3D Photography - PowerPoint PPT Presentation


  • 432 Views
  • Uploaded on

3D Photography (Image-based Model Acquisition). Funky Image Goes Here. “Analog” 3D photography !. “3D stereoscopic imaging” been around as long as cameras have Use camera with 2 or more lenses (or stereo attachment) Use stereo viewer to create impression of 3D. Motivation .

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '3D Photography' - mike_john


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
3d photography image based model acquisition

3D Photography(Image-based Model Acquisition)

Funky Image Goes Here

Deepak Bandyopadhyay / UNC Chapel Hill

analog 3d photography
“Analog” 3D photography !
  • “3D stereoscopic imaging”
    • been around as long as cameras have
    • Use camera with 2 or more lenses (or stereo attachment)
    • Use stereo viewer to create impression of 3D

Deepak Bandyopadhyay / 258 / 3D Photography

motivation
Motivation
  • Digitizing real world objects
  • Getting realistic models

places

objects

humans

Deepak Bandyopadhyay / 258 / 3D Photography

3d photography definition
3D Photography : Definition
  • Sometimes called “3D Scanning”
  • Use cameras and light to capture the shape & appearance of real objects
  • Shape == geometry (point sampling + surface reconstruction + fairing)
  • Appearance == surface attributes (color/texture, material properties, reflectance)
  • Final result = richly detailed model

Deepak Bandyopadhyay / 258 / 3D Photography

applications in industry
Applications in Industry
  • Human body / head / face scans
    • Avatar creation for virtual worlds
    • 3d conferencing
    • medical applications
    • product design
    • Platforms:
      • Cyberware RD3030
      • Others (Geomagic, Metacreations, Cyrax, Geometrix…)

Deepak Bandyopadhyay / 258 / 3D Photography

more applications
More applications
  • Historical preservation, dissemination of museum artifacts (Digital Michelangelo, Monticello, …)
  • CAD/CAM (eg. Legacy motorcycle parts scanned by Geomagic for Harley-Davidson).
  • Marketing (models of products on the web)
  • 3D games & simulation
  • Reverse engineering

Deepak Bandyopadhyay / 258 / 3D Photography

technology overview
Technology Overview
  • The Imaging Pipeline
    • Real World
    • Optics
    • Recorder
    • Digitizer
    • Vision & Graphics

Deepak Bandyopadhyay / 258 / 3D Photography

quick notes on optics
Quick Notes on Optics
  • Model lenses with all their properties - aberration, distortion, flare, vignetting etc.
  • We correct for some of these effects (eg. distortion) in the calibration, ignore others.
  • CCD (charged coupled devices) are the most popular recording media.

Deepak Bandyopadhyay / 258 / 3D Photography

theory passive methods
Theory : Passive Methods
  • Stereo pair matching
  • Structure from motion
  • Shape from shading
  • Photometric stereo

Deepak Bandyopadhyay / 258 / 3D Photography

stereo matching
Stereo Matching
  • Stereo Matching Basics
    • Needs two images, like stereoscopy
    • Given correspondence betweenpoints in 2 views, we can find depth by triangulation
    • But correspondence is hard prob!
    • A lot of literature on solving it…
  • Stereo Matching output
    • 3D point cloud
    • Remove outliers and pass through surface reconstructor

Deepak Bandyopadhyay / 258 / 3D Photography

structure from motion
Structure from Motion
  • Camera moving, objects static
  • Compute camera motion and object geometry from motion of image points
  • Assumption -orthographic projn (use telephoto)
  • If: world origin = 3D centroid camera origin = 2D centroidThen: camera translation drops out

Deepak Bandyopadhyay / 258 / 3D Photography

structure from motion12
Structure from Motion
  • Camera moving, objects static
  • Compute camera motion and object geometry from motion of image points

Deepak Bandyopadhyay / 258 / 3D Photography

structure from motion13
Structure from Motion
  • Factorization [Tomasi & Kanade, 92]
  • Find M, S using Singular Value Decomposition of W.

SVD gives:

S’S modulo linear transform A. Solve for A using constraints on M.

Deepak Bandyopadhyay / 258 / 3D Photography

more methods
More methods
  • Shape from shading, [Horn]
    • Invert Lambert’s Law (L=I k cos )knowing the intensity at image pointto solve for normal
  • Photometric stereo [Woodham]
    • An extension of the above
    • Two or more images under different illumination conditions.
    • Each image provides one normal
    • Three images provide unique solution for a pixel.

Deepak Bandyopadhyay / 258 / 3D Photography

active sensing
Active Sensing
  • Passive methods (eg. stereo matching) suffer from ambiguities - many similar regions in an image correspond to a point in the other.
  • Project known / regular pattern (“structured light”) into scene to disambiguate
  • get precise reconstruction by combining views
    • Laser rangefinder
    • Projectors and imperceptible structured light

Deepak Bandyopadhyay / 258 / 3D Photography

desktop 3d photography jean yves bouguet pietro perona
Desktop 3D PhotographyJean-Yves Bouguet, Pietro Perona
  • An active sensing technique using “weak structured lighting”
  • Need: camera, lamp, chessboard, pencil, stick
  • Idea:
    • Light object with lamp & aim camera at it
    • Move stick around & capture shadow sequence
    • Use image of deformed shadow to calc 3D shape

Deepak Bandyopadhyay / 258 / 3D Photography

desktop 3d photography jean yves bouguet pietro perona17
Desktop 3D PhotographyJean-Yves Bouguet, Pietro Perona
  • Computation of 3d position from the plane of light source, stick and shadow

Deepak Bandyopadhyay / 258 / 3D Photography

volumetric methods chevette project debevec 1991
Volumetric MethodsChevette Project, Debevec, 1991

Deepak Bandyopadhyay / 258 / 3D Photography

voxel models from images
Voxel Models from Images
  • When there are 2 colors in the image - use volume intersection [Szeliski 1993]
    • Back-project silhouettes from camera views & intersect

Deepak Bandyopadhyay / 258 / 3D Photography

voxel models from images20
Voxel Models from Images
  • With more colors but constrained viewpoints, we use voxel coloring [Seitz & Dyer, 1997]
    • Choose a voxel & project to it from all views
    • Color if enough matches
    • Prob - determining visibilityof a point from a view
    • Solution - depth orderedtraversal using a “view indep.d.o.” (dist from separating plane)

Deepak Bandyopadhyay / 258 / 3D Photography

voxel models from images21
Voxel Models from Images
  • A view-independent depth order may not exist (for some configuration of viewpoints / scene geometry).
  • Use Space Carving [Kutulakos & Seitz, 1998]
    • Computes 3D (voxel) shape from multiple color photos
    • Computes “maximally photo-consistent shape”
      • maximal superset of all 3D shapes that produce the given photos

Deepak Bandyopadhyay / 258 / 3D Photography

space carving
Space Carving
  • Algorithm:

a) Initialize V to volume containing true scene

b) For each voxel,

      • check if photo-consistent
      • if not, remove (“carve”) it.
  • Can be shown to converge to maximal photo-consistent scene (union of all photo-consistent scenes).

Deepak Bandyopadhyay / 258 / 3D Photography

space carving results
Space Carving : Results
  • House walkthru - 24 rendered input views
  • Results best as seen from one of the original views

Deepak Bandyopadhyay / 258 / 3D Photography

modeling from a single view criminisi et al 1999
Modeling from a single view(Criminisi et al, 1999)
  • Compute 3D affine measurements of the scene from single perspective image
  • Use minimal geom info
    • vanishing line for a pencil ofplanes || to reference plane
    • vanishing point of parallellines along a directionoutside reference plane

Deepak Bandyopadhyay / 258 / 3D Photography

modeling from a single view criminisi et al 199925
Modeling from a single view(Criminisi et al, 1999)
  • Compute “ratio of parallel distances”
  • Creating a 3D model from a photograph
    • horizontal lines used to compute vanishing line
    • parallel vertical lines used to compute vanishing point
  • Can generate geometrically correct model from a Renaissance painting (with correct perspective)

Deepak Bandyopadhyay / 258 / 3D Photography

extracting color reflectance
Extracting color, reflectance
  • Photographs have lighting/shading effects that we estimate (reflectance function) and compensate for (specular highlight removal) or change (relighting)
  • Work of Paul Debevec & others at Berkeley (acquiring reflectance field)
  • Wood et al at U. Washington (surface light lield for 3D photography)

Deepak Bandyopadhyay / 258 / 3D Photography

surface light field wood et al 2000
Surface Light Field[Wood et al, 2000]
  • A 4D function on the surface - at surface parameter (u,v), for every direction (,), stores the color.
  • Fixed illumination conditions.
  • Photographs taken from a lot of different directions sample the surface light field.
  • Continuous function (piecewise linear over ,) estimated by pointwise fairing.

Deepak Bandyopadhyay / 258 / 3D Photography

reflectance from photographs yu debevec et al 1999
Reflectance from Photographs (Yu, Debevec et al, 1999)
  • Estimating reflectance for entire scenes
    • Too general a problem, parameterize thus:
      • Assume surface can be divided into patches
      • Diffuse reflectance function (albedo), varies across a patch
      • Specular reflectance function taken as const across a region
  • Assume known lighting, calib, geometry known
  • Approach - Inverse Global Illumination
    • Estimate BRDF for direct illumination - f(u,v,,)

Deepak Bandyopadhyay / 258 / 3D Photography

reflectance from photographs yu debevec et al 199929
Reflectance from Photographs (Yu, Debevec et al, 1999)
  • Inverse Global Illumination
    • Known Li (measure), Ii (calc fm known light sources) at every pixel
    • Estimate BRDF for direct illumination - f(u,v,i,i,r,r)
      • Write BRDF as a constant diffuse term and a specular term which is a function of incoming & outgoing  and roughness.
      • Solve for the constants(d, s,)
      • For indirect illumination - estimate the parameters (and indirect illumination coeffs with other patches) iteratively

Deepak Bandyopadhyay / 258 / 3D Photography

case study fa ade debevec taylor malik 1996
Case study - FaçadeDebevec, Taylor & Malik, 1996
  • Modeling architectural scenes from photographs
  • Not fully automatic (user inputs blocky 3D model)
    • Using blocks leads to fewer params in architectural models
  • User marks corresponding features on photo
  • Computer solves for block size, scale, camera rotation by minimizing error of corresponding features
  • Reprojects textures from the photographs onto the reconstructed model

Deepak Bandyopadhyay / 258 / 3D Photography

arches and surfaces of revolution
Arches andSurfaces of Revolution

Taj Mahal

modeled from

one photograph

Deepak Bandyopadhyay / 258 / 3D Photography

case study digital michelangelo project
Case study - Digital Michelangelo Project
  • 3D scanning of large statues (SIGGRAPH 00)
  • Separate geometry and color scans
    • custom rig : laser scanner & camera mounted concurrently
  • Range scan post-processing
    • Combine range scans from different positions
      • Use volumetric modeling methods (Curless, Levoy 1996)
    • Fill holes using space carving

Deepak Bandyopadhyay / 258 / 3D Photography

case study digital michelangelo project33
Case study - Digital Michelangelo Project
  • Color scan processing
    • Compensate for ambient lighting
      • subtract image with & without spotlight
    • Subtract out shadows & specularities
    • find surface orientation (inverse lighting computation)
    • convert color to RGB reflectance (acquire light field)
      • Using estimated BRDF of marble
      • modeling subsurface scattering

Deepak Bandyopadhyay / 258 / 3D Photography

digital michelangelo scanning a large object
calibrated motions

pitch (yellow)

pan (blue)

horizontal translation (orange)

uncalibrated motions

vertical translation

remounting the scan head

moving the entire gantry

Digital MichelangeloScanning a large object

Deepak Bandyopadhyay / 258 / 3D Photography

references
References
  • [Bouguet98] Bouguet, J.-Y., P. Perona. 3D Photography on your Desk. In Proc. ICCV 1998
  • [Bouguet00] Bouguet, J.-Y. Presentation on Desktop 3D Photography, in SIGGRAPH course notes on 3D Photography, 2000
  • [Criminisi99] Criminisi, A., I. Reid and A. Zisserman. Single View Metrology. In Proc. ICCV, pp 434-442, September 1999
  • [Curless96] Curless, B. and M. Levoy. A Volumetric Method for Building Complex Models from Range Images. In Proc. SIGGRAPH 1996
  • [Debevec96] Debevec, P., C. Taylor and J. Malik. Façade - Modeling and Rendering Architectural Scenes from Photographs. In Proc. SIGGRAPH 1996
  • [Debevec00a] Debevec, P. Presentation on the Façade, from SIGGRAPH course notes on 3D Photography, 1999, 2000.
  • [Debevec00b] Debevec, P., T. Hawkins, C. Tchou, H.P.Duiker, W. Sarokin and M. Sagar. Acquiring the Reflectance Field of a Human Face. In Proc. SIGGRAPH 2000.

Deepak Bandyopadhyay / 258 / 3D Photography

more references
More References
  • [Horn70] Horn, B.K.P. Shape from Shading : A Method for Obtaining the Shape of a Smooth Opaque Object from One View. Ph.D. Thesis, Dept of EE, MIT, 1970.
  • [Kutulakos98] Kutulakos, K. N. and S. Seitz. A Theory of Shape by Space Carving. URCS TR#692, May 1998, appeared in Proc. ICCV 1999.
  • [Levoy96] Levoy, M. and P. Hanrahan. Light Field Rendering. In Proc. SIGGRAPH 1996.
  • [Levoy00a] Levoy, M., Pulli, K., Curless, B. et al. The Digital Michelangelo Project - 3D Scanning of Large Statues. In Proc. SIGGRAPH 2000.
  • [Levoy00b] Levoy, M. Presentation on the Digital Michelangelo Project, in SIGGRAPH course notes on 3D Photography, 2000.
  • [Seitz97] Seitz & Dyer. Photorealistic Scene Reconstruction by Voxel Coloring. In Proc. CVPR 1997, pp. 1067-1073.

Deepak Bandyopadhyay / 258 / 3D Photography

still more references
Still More References
  • [Seitz00] Seitz, S. SIGGRAPH course notes on 3D photography, 1999, 2000.
  • [Szeliski93] Szeliski, R. Rapid Octree Construction from Image Sequences. CGVIP : Image Understanding, vol. 58, no. 1, pp 23-32, 1993.
  • [Wood00] Wood, D., D. I. Azuma, K. Aldinger, B. Curless, T. Duchamp, D.H. Salesin and W. Stuetzle. Surface Light Fields for 3D Photography. In Proc. SIGGRAPH 2000.
  • [Woodham80] Woodham, R. Photometric Stereo for Determining Surface Orientation from Multiple Images. Journal of Optical Engineering, vol. 19, no. 1, pp 138-144, 1980.
  • [Yu99] Yu, Y., P. Debevec, J. Malik and T. Hawkins. Inverse Global Illumination - Recovering Reflectance Models of Real Scenes from Photographs.

Deepak Bandyopadhyay / 258 / 3D Photography

ad