2d texture synthesis
1 / 30

2D Texture Synthesis - PowerPoint PPT Presentation

  • Uploaded on

2D Texture Synthesis. Instructor: Yizhou Yu. Texture synthesis. Goal: increase texture resolution yet keep local texture variation. Synthesis by global statistics. Idea: Obtain statistics of input image, match with output image Histograms: normalized graph of intensity frequencies.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' 2D Texture Synthesis' - yoshi-mcfarland

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
2d texture synthesis

2D Texture Synthesis

Instructor: Yizhou Yu

Texture synthesis
Texture synthesis

  • Goal: increase texture resolution yet keep local texture variation

Synthesis by global statistics
Synthesis by global statistics

  • Idea: Obtain statistics of input image, match with output image

  • Histograms: normalized graph of intensity frequencies

Synthesis by global statistics1
Synthesis by global statistics

  • Consider extreme case:

  • Histogram matching can generate two images:

Using higher order global stats
Using higher-order global stats

  • Histogram matching does not take into account spatial information.

  • Need additional info:

    • For every pixel, examine local neighborhood

      • Gives local but not global features (e.g. – veins)

  • Use higher order stats:

    • Correlation

      • average of the product of the intensities =

      • tells how interdependent pixel values are

General procedure
General procedure

  • 1.) Define error metric:

    • What do you want to match?

  • 2.) Match statistics

    • Histogram matching:

      • Get the density value of a pixel in the output image. Map this density to a pixel intensity in the input image  Overwrite the pixel in the output image with the mapped pixel intensity in the input image

Pyramid based texture analysis synthesis
Pyramid-based Texture Analysis/Synthesis

  • Paper by David J. Heeger and James R. Bergen from SIGGRAPH 1995

  • http://www.cns.nyu.edu/~david/ftp/reprints/heeger-siggraph95.pdf

Pyramid based synthesis
Pyramid-based Synthesis

  • Downsample image several times and keep track of only differences in a feature image

  • To reconstruct, upsample small images and add differences

  • Pixels in feature images are close to zero

  • Provides multiple scales with each feature image representing different feature sizes

Laplacian pyramid
Laplacian Pyramid

  • 1-D filter: (1/16) (1 4 6 4 1)

  • To generalize to 2D, use tensor product

  • Normalize by dividing by 256


  • To upsample, copy pixels to every other pixel in larger image

  • Fill the rest with zeroes, run low pass filter to fill in values

Oriented filters
Oriented Filters

  • Gaussian filters are symmetric, thus edges and contours are not detected

  • Use multiple oriented filters to catch non-symmetric features

Left top series: Oriented filters Right image: Texture

Left bottom series: Filtered textures

Pyramids with oriented filters
Pyramids with Oriented Filters

  • Each oriented filter creates its own feature image

  • Thus, for each downsampled image, keep one image for each oriented filter

  • Each oriented filter captures one orientation of lines

Matching image features
Matching Image Features

  • Input parameters:

    • noise: initial noisy texture

    • texture: texture to be matched

  • Output texture is stored in noise

Matching image features1
Matching Image Features

  • Helper functions:

    • MatchHistogram(noise, texture)

      • Matches histogram, using method described last time

    • MakePyramid(texture)

      • Create pyramid images (base and feature images)

    • CollapsePyramid(pyramid)

      • Constructs high resolution image from base and feature images

Matching image features2
Matching Image Features

MatchTexture(noise, texture)


MatchHistogram(noise, texture) // first-order matching

analysis_pyr = MakePyramid(texture) // create pyramid from input texture

for several iterations

synthesis_pyr = MakePyramid(noise) // create pyramid from noise

for each feature image, fi of analysis_pyr

for each feature image, fj of synthesis_pyr of same orientation

MatchHistogram(fi, fj)

end for

end for

noise = CollapsePyramid(synthesis_pyr)

MatchHistogram(noise, texture)

end for


Matching image features3
Matching Image Features

  • MatchTexture matches histograms of feature images, not pixel values, thus providing a much better matching

  • Feature images already consider local neighborhoods, resulting in better approximations

  • Good for randomized textures

  • Textures with large scale features or thin and long features not matched well

    • e.g. stripes in wood, lines in coral

Texture synthesis using local neighborhoods
Texture Synthesis Using Local Neighborhoods

  • Main goal is to keep local spatial coherence, but not global stats.

  • Randomized method

    • Pick pixel and copy it along with its neighborhood to random parts in synthesized image

    • If two neighborhoods overlap, just blend

    • This can result in features getting cut off if larger than local neighborhood

Neighborhood based texture synthesis
Neighborhood-Based Texture Synthesis

  • Patch-Based

    • Patch-based sampling achieves real-time speed. [Liang et. al. 2001]

    • Image quilting: high-quality results and simple implementation [Efros & Freeman 2001]

    • Graph cut provides a powerful and refinable scheme. [Kwantra et. al. 2003]

  • Pixel-Based

    • [Efros & Leung 99], [Wei & Levoy 2000], [Ashikhmin 2001], [Hertzmann et. al. 2001], [Zhang et. al. 2003]

Pixel wise synthesis
Pixel-wise Synthesis

  • Grow pixel by pixel

  • Start from an existing patch of the texture (as opposed to noise texture like Pyramid-based Synthesis)

  • Look for regions in input texture most similar to current region in new texture

  • Copy pixels next to best-match region to expand new texture

Following raster order
Following Raster Order

  • Blue region: already set by algorithm

  • Green region: compare this region to input image

  • Yellow region: closest match

  • Red region: replace these pixels with magenta region to maintain local integrity

Hierarchical synthesis
Hierarchical Synthesis

  • Build a multi-resolution pyramid for the example texture

  • Generate a synthesized pyramid for the output texture

    • At each level, follow the raster order

Randomizing synthesis
Randomizing Synthesis

  • Instead of picking pixels from closest matching region, use threshold to pick a few candidate matches

  • Randomly pick one of the candidates

  • Refer to “Texture Synthesis by Non-parametric Sampling”

    • http://www.cs.berkeley.edu/~efros/research/synthesis.html

Patch based synthesis
Patch-Based Synthesis

  • Search in a sample texture for neighborhoods most similar to a context region

  • Merge a patch with the partially synthesized output texture

The seam problem
The Seam Problem

  • Feature discontinuities may appear in patch-based synthesis.

The second reason
The Second Reason

  • Didn’t find the smoothest transition between the incoming patch and context region.

  • Solutions: use dynamic programming [Efros and Freeman 2001] or graph cut [Kwatra et al. 2003] to find an optimal cut.



The first reason
The First Reason

  • Rigid template matching (SSD) often employed in the first step does NOT consider geometric similarity



Feature map guided texture synthesis
Feature Map Guided Texture Synthesis

  • Basic Steps [Wu and Yu 2004]

    • Maintain an input and output feature map

    • Match and align curvilinear features

    • Integrate feature maps into template-based texture synthesis



Feature Map



Texton Mask

Acceleration schemes
Acceleration Schemes

  • Fourier Transform

  • Acceleration techniques for nearest-neighbor search

    • Tree-structured vector quantization

    • Kd-trees

  • Minimizing the candidate set

    • Coherent synthesis, [Ashikhmin 2001]

  • Precomputing candidate set

    • K-coherent search, [Tong et al. 2002]

    • Jump Map, [Zelinka and Garland 2002]

  • GPU with parallel synthesis

    • [Lefebvre and Hoppe 2005]