Texture-Mapping Real Scenes from Photographs - PowerPoint PPT Presentation

Texture mapping real scenes from photographs
1 / 23

  • Uploaded on
  • Presentation posted in: General

SIGGRAPH 2000 Course on Image-Based Surface Details. Texture-Mapping Real Scenes from Photographs. Yizhou Yu Computer Science Division University of California at Berkeley. Basic Steps. Acquire Photographs Recover Geometry Align Photographs with Geometry Map Photographs onto Geometry.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Texture-Mapping Real Scenes from Photographs

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Texture mapping real scenes from photographs

SIGGRAPH 2000 Course on

Image-Based Surface Details

Texture-Mapping Real Scenes from Photographs

Yizhou Yu

Computer Science Division

University of California at Berkeley

Basic steps

Basic Steps

  • Acquire Photographs

  • Recover Geometry

  • Align Photographs with Geometry

  • Map Photographs onto Geometry

Camera pose estimation

Camera Pose Estimation

  • Input

    • Known geometry recovered from photographs or laser range scanners

    • A set of photographs taken with a camera

  • Output

    • For each photograph, 3 rotation and 3 translation parameters of the camera with respect to the geometry

  • Requirement

    • 4 correspondences between each photograph and the geometry

Recover camera pose with known correspondences

Recover Camera Pose with Known Correspondences

  • Least-squares solution

    • Needs good initial estimation from human interaction



Recover rotation parameters only from known correspondences

Recover Rotation Parameters only from Known Correspondences

  • Constraints

  • Least-squares solution



Obtaining correspondences

Obtaining Correspondences

  • Feature Detection in 3D geometry and 2D images

  • Human interaction

    • Interactively pick corresponding points in photographs and 3D geometry

  • Automatic Search

    • Combinatorial search

Automatic search for correspondences

Automatic Search for Correspondences

  • Pose estimation using calibration targets

  • Combinatorial search for the best match

    • 4 correspondences each image

3D Targets

Camera pose results

Camera Pose Results

  • Accuracy: consistently within 2 pixels

Texture-mapping a single image

Texture mapping

Texture Mapping

  • Conventional texture-mapping with texture coordinates

  • Projective texture-mapping

Texture map synthesis i

Texture Map Synthesis I

  • Conventional Texture-Mapping with Texture Coordinates

    • Create a triangular texture patch for each triangle

    • The texture patch is a weighted average of the image patches from multiple photographs

    • Pixels that are close to image boundaries or viewed from a grazing angle obtain smaller weights


3D Triangle

Texture Map

Texture map synthesis ii

Texture Map Synthesis II

  • Allocate space for texture patches from texture maps

    • Generalization of memory allocation to 2D

    • Quantize edge length to a power of 2

    • Sort texture patches into decreasing order and use First-Fit strategy to allocate space


A texture map packed with triangular texture patches

A Texture Map Packed with Triangular Texture Patches

Texture mapping and object manipulation

Texture-Mapping and Object Manipulation

Original Configuration

Novel Configuration

Texture map compression i

Texture Map Compression I

  • The size of each texture patch is determined by the amount of color variations on its corresponding triangles in photographs.

  • An edge detector (the derivative of the Gaussian) is used as a metric for variations.

Texture map compression ii

Texture Map Compression II

  • Reuse texture patches

    • Map the same patch to multiple 3D triangles with similar color variations

  • K-means clustering to generate texture patch representatives

  • Larger penalty along triange edges to reduce Mach Band effect

3D Triangles

Texture Map

Synthetic images with compressed and uncompressed texture maps

Synthetic Images with Compressed and Uncompressed Texture Maps


5 texture maps


20 texture maps

20 texture maps

5 texture maps

Projective texture mapping

Projective Texture-Mapping

  • Can directly use the original photographs in texture-mapping

  • Visibility processing is more complicated

  • Projective texture-mapping has been implemented in hardware, therefore, real-time rendering becomes possible

  • View-dependent effects can be added by effectively using hardware accumulation buffer

Motivation for visibility processing artifacts caused by hardware

Motivation for Visibility Processing: Artifacts Caused by Hardware




Texture gets projected onto occluded and

backfacing polygons

Visibility algorithms

Visibility Algorithms

  • Image-space algorithms

    • Shadow buffer

    • Ray casting

  • Object-space algorithms

    • Weiler-Atherton

A hybrid visibility algorithm

A Hybrid Visibility Algorithm

  • Occlusion testing in image-space using Z-buffer hardware

    • Render polygons with their identifiers as colors

    • Retrieve occluding polygons’ ids from color buffer

  • Object-space shallow clipping to generate fewer polygons

Input photographs and recovered geometry from facade

Input Photographs and Recovered Geometry from Facade

Visibility processing results

Visibility Processing Results

The tower

The rest of the campus

Synthetic renderings

Synthetic Renderings

  • Login