2d texture synthesis
Skip this Video
Download Presentation
2D Texture Synthesis

Loading in 2 Seconds...

play fullscreen
1 / 30

2D Texture Synthesis - PowerPoint PPT Presentation

  • Uploaded on

2D Texture Synthesis. Instructor: Yizhou Yu. Texture synthesis. Goal: increase texture resolution yet keep local texture variation. Synthesis by global statistics. Idea: Obtain statistics of input image, match with output image Histograms: normalized graph of intensity frequencies.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' 2D Texture Synthesis' - yoshi-mcfarland

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
2d texture synthesis

2D Texture Synthesis

Instructor: Yizhou Yu

texture synthesis
Texture synthesis
  • Goal: increase texture resolution yet keep local texture variation
synthesis by global statistics
Synthesis by global statistics
  • Idea: Obtain statistics of input image, match with output image
  • Histograms: normalized graph of intensity frequencies
synthesis by global statistics1
Synthesis by global statistics
  • Consider extreme case:
  • Histogram matching can generate two images:
using higher order global stats
Using higher-order global stats
  • Histogram matching does not take into account spatial information.
  • Need additional info:
    • For every pixel, examine local neighborhood
      • Gives local but not global features (e.g. – veins)
  • Use higher order stats:
    • Correlation
      • average of the product of the intensities =
      • tells how interdependent pixel values are
general procedure
General procedure
  • 1.) Define error metric:
    • What do you want to match?
  • 2.) Match statistics
    • Histogram matching:
      • Get the density value of a pixel in the output image. Map this density to a pixel intensity in the input image  Overwrite the pixel in the output image with the mapped pixel intensity in the input image
pyramid based texture analysis synthesis
Pyramid-based Texture Analysis/Synthesis
  • Paper by David J. Heeger and James R. Bergen from SIGGRAPH 1995
  • http://www.cns.nyu.edu/~david/ftp/reprints/heeger-siggraph95.pdf
pyramid based synthesis
Pyramid-based Synthesis
  • Downsample image several times and keep track of only differences in a feature image
  • To reconstruct, upsample small images and add differences
  • Pixels in feature images are close to zero
  • Provides multiple scales with each feature image representing different feature sizes
laplacian pyramid
Laplacian Pyramid
  • 1-D filter: (1/16) (1 4 6 4 1)
  • To generalize to 2D, use tensor product
  • Normalize by dividing by 256
  • To upsample, copy pixels to every other pixel in larger image
  • Fill the rest with zeroes, run low pass filter to fill in values
oriented filters
Oriented Filters
  • Gaussian filters are symmetric, thus edges and contours are not detected
  • Use multiple oriented filters to catch non-symmetric features

Left top series: Oriented filters Right image: Texture

Left bottom series: Filtered textures

pyramids with oriented filters
Pyramids with Oriented Filters
  • Each oriented filter creates its own feature image
  • Thus, for each downsampled image, keep one image for each oriented filter
  • Each oriented filter captures one orientation of lines
matching image features
Matching Image Features
  • Input parameters:
    • noise: initial noisy texture
    • texture: texture to be matched
  • Output texture is stored in noise
matching image features1
Matching Image Features
  • Helper functions:
    • MatchHistogram(noise, texture)
      • Matches histogram, using method described last time
    • MakePyramid(texture)
      • Create pyramid images (base and feature images)
    • CollapsePyramid(pyramid)
      • Constructs high resolution image from base and feature images
matching image features2
Matching Image Features

MatchTexture(noise, texture)


MatchHistogram(noise, texture) // first-order matching

analysis_pyr = MakePyramid(texture) // create pyramid from input texture

for several iterations

synthesis_pyr = MakePyramid(noise) // create pyramid from noise

for each feature image, fi of analysis_pyr

for each feature image, fj of synthesis_pyr of same orientation

MatchHistogram(fi, fj)

end for

end for

noise = CollapsePyramid(synthesis_pyr)

MatchHistogram(noise, texture)

end for


matching image features3
Matching Image Features
  • MatchTexture matches histograms of feature images, not pixel values, thus providing a much better matching
  • Feature images already consider local neighborhoods, resulting in better approximations
  • Good for randomized textures
  • Textures with large scale features or thin and long features not matched well
    • e.g. stripes in wood, lines in coral
texture synthesis using local neighborhoods
Texture Synthesis Using Local Neighborhoods
  • Main goal is to keep local spatial coherence, but not global stats.
  • Randomized method
    • Pick pixel and copy it along with its neighborhood to random parts in synthesized image
    • If two neighborhoods overlap, just blend
    • This can result in features getting cut off if larger than local neighborhood
neighborhood based texture synthesis
Neighborhood-Based Texture Synthesis
  • Patch-Based
    • Patch-based sampling achieves real-time speed. [Liang et. al. 2001]
    • Image quilting: high-quality results and simple implementation [Efros & Freeman 2001]
    • Graph cut provides a powerful and refinable scheme. [Kwantra et. al. 2003]
  • Pixel-Based
    • [Efros & Leung 99], [Wei & Levoy 2000], [Ashikhmin 2001], [Hertzmann et. al. 2001], [Zhang et. al. 2003]
pixel wise synthesis
Pixel-wise Synthesis
  • Grow pixel by pixel
  • Start from an existing patch of the texture (as opposed to noise texture like Pyramid-based Synthesis)
  • Look for regions in input texture most similar to current region in new texture
  • Copy pixels next to best-match region to expand new texture
following raster order
Following Raster Order
  • Blue region: already set by algorithm
  • Green region: compare this region to input image
  • Yellow region: closest match
  • Red region: replace these pixels with magenta region to maintain local integrity
hierarchical synthesis
Hierarchical Synthesis
  • Build a multi-resolution pyramid for the example texture
  • Generate a synthesized pyramid for the output texture
    • At each level, follow the raster order
randomizing synthesis
Randomizing Synthesis
  • Instead of picking pixels from closest matching region, use threshold to pick a few candidate matches
  • Randomly pick one of the candidates
  • Refer to “Texture Synthesis by Non-parametric Sampling”
    • http://www.cs.berkeley.edu/~efros/research/synthesis.html
patch based synthesis
Patch-Based Synthesis
  • Search in a sample texture for neighborhoods most similar to a context region
  • Merge a patch with the partially synthesized output texture
the seam problem
The Seam Problem
  • Feature discontinuities may appear in patch-based synthesis.
the second reason
The Second Reason
  • Didn’t find the smoothest transition between the incoming patch and context region.
  • Solutions: use dynamic programming [Efros and Freeman 2001] or graph cut [Kwatra et al. 2003] to find an optimal cut.



the first reason
The First Reason
  • Rigid template matching (SSD) often employed in the first step does NOT consider geometric similarity



feature map guided texture synthesis
Feature Map Guided Texture Synthesis
  • Basic Steps [Wu and Yu 2004]
    • Maintain an input and output feature map
    • Match and align curvilinear features
    • Integrate feature maps into template-based texture synthesis


Feature Map



Texton Mask

acceleration schemes
Acceleration Schemes
  • Fourier Transform
  • Acceleration techniques for nearest-neighbor search
    • Tree-structured vector quantization
    • Kd-trees
  • Minimizing the candidate set
    • Coherent synthesis, [Ashikhmin 2001]
  • Precomputing candidate set
    • K-coherent search, [Tong et al. 2002]
    • Jump Map, [Zelinka and Garland 2002]
  • GPU with parallel synthesis
    • [Lefebvre and Hoppe 2005]