1 / 30

# 2D Texture Synthesis - PowerPoint PPT Presentation

2D Texture Synthesis. Instructor: Yizhou Yu. Texture synthesis. Goal: increase texture resolution yet keep local texture variation. Synthesis by global statistics. Idea: Obtain statistics of input image, match with output image Histograms: normalized graph of intensity frequencies.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

2D Texture Synthesis

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

## 2D Texture Synthesis

Instructor: Yizhou Yu

### Texture synthesis

• Goal: increase texture resolution yet keep local texture variation

### Synthesis by global statistics

• Idea: Obtain statistics of input image, match with output image

• Histograms: normalized graph of intensity frequencies

### Synthesis by global statistics

• Consider extreme case:

• Histogram matching can generate two images:

### Using higher-order global stats

• Histogram matching does not take into account spatial information.

• For every pixel, examine local neighborhood

• Gives local but not global features (e.g. – veins)

• Use higher order stats:

• Correlation

• average of the product of the intensities =

• tells how interdependent pixel values are

### General procedure

• 1.) Define error metric:

• What do you want to match?

• 2.) Match statistics

• Histogram matching:

• Get the density value of a pixel in the output image. Map this density to a pixel intensity in the input image  Overwrite the pixel in the output image with the mapped pixel intensity in the input image

### Pyramid-based Texture Analysis/Synthesis

• Paper by David J. Heeger and James R. Bergen from SIGGRAPH 1995

• http://www.cns.nyu.edu/~david/ftp/reprints/heeger-siggraph95.pdf

### Pyramid-based Synthesis

• Downsample image several times and keep track of only differences in a feature image

• To reconstruct, upsample small images and add differences

• Pixels in feature images are close to zero

• Provides multiple scales with each feature image representing different feature sizes

### Laplacian Pyramid

• 1-D filter: (1/16) (1 4 6 4 1)

• To generalize to 2D, use tensor product

• Normalize by dividing by 256

### Upsampling

• To upsample, copy pixels to every other pixel in larger image

• Fill the rest with zeroes, run low pass filter to fill in values

### Oriented Filters

• Gaussian filters are symmetric, thus edges and contours are not detected

• Use multiple oriented filters to catch non-symmetric features

Left top series: Oriented filtersRight image: Texture

Left bottom series: Filtered textures

### Pyramids with Oriented Filters

• Each oriented filter creates its own feature image

• Thus, for each downsampled image, keep one image for each oriented filter

• Each oriented filter captures one orientation of lines

### Matching Image Features

• Input parameters:

• noise: initial noisy texture

• texture: texture to be matched

• Output texture is stored in noise

### Matching Image Features

• Helper functions:

• MatchHistogram(noise, texture)

• Matches histogram, using method described last time

• MakePyramid(texture)

• Create pyramid images (base and feature images)

• CollapsePyramid(pyramid)

• Constructs high resolution image from base and feature images

### Matching Image Features

MatchTexture(noise, texture)

{

MatchHistogram(noise, texture) // first-order matching

analysis_pyr = MakePyramid(texture) // create pyramid from input texture

for several iterations

synthesis_pyr = MakePyramid(noise) // create pyramid from noise

for each feature image, fi of analysis_pyr

for each feature image, fj of synthesis_pyr of same orientation

MatchHistogram(fi, fj)

end for

end for

noise = CollapsePyramid(synthesis_pyr)

MatchHistogram(noise, texture)

end for

}

### Matching Image Features

• MatchTexture matches histograms of feature images, not pixel values, thus providing a much better matching

• Feature images already consider local neighborhoods, resulting in better approximations

• Good for randomized textures

• Textures with large scale features or thin and long features not matched well

• e.g. stripes in wood, lines in coral

### Texture Synthesis Using Local Neighborhoods

• Main goal is to keep local spatial coherence, but not global stats.

• Randomized method

• Pick pixel and copy it along with its neighborhood to random parts in synthesized image

• If two neighborhoods overlap, just blend

• This can result in features getting cut off if larger than local neighborhood

### Neighborhood-Based Texture Synthesis

• Patch-Based

• Patch-based sampling achieves real-time speed. [Liang et. al. 2001]

• Image quilting: high-quality results and simple implementation [Efros & Freeman 2001]

• Graph cut provides a powerful and refinable scheme. [Kwantra et. al. 2003]

• Pixel-Based

• [Efros & Leung 99], [Wei & Levoy 2000], [Ashikhmin 2001], [Hertzmann et. al. 2001], [Zhang et. al. 2003]

### Pixel-wise Synthesis

• Grow pixel by pixel

• Start from an existing patch of the texture (as opposed to noise texture like Pyramid-based Synthesis)

• Look for regions in input texture most similar to current region in new texture

• Copy pixels next to best-match region to expand new texture

### Following Raster Order

• Blue region: already set by algorithm

• Green region: compare this region to input image

• Yellow region: closest match

• Red region: replace these pixels with magenta region to maintain local integrity

### Hierarchical Synthesis

• Build a multi-resolution pyramid for the example texture

• Generate a synthesized pyramid for the output texture

• At each level, follow the raster order

### Randomizing Synthesis

• Instead of picking pixels from closest matching region, use threshold to pick a few candidate matches

• Randomly pick one of the candidates

• Refer to “Texture Synthesis by Non-parametric Sampling”

• http://www.cs.berkeley.edu/~efros/research/synthesis.html

### Patch-Based Synthesis

• Search in a sample texture for neighborhoods most similar to a context region

• Merge a patch with the partially synthesized output texture

### The Seam Problem

• Feature discontinuities may appear in patch-based synthesis.

### The Second Reason

• Didn’t find the smoothest transition between the incoming patch and context region.

• Solutions: use dynamic programming [Efros and Freeman 2001] or graph cut [Kwatra et al. 2003] to find an optimal cut.

Original

Warped

### The First Reason

• Rigid template matching (SSD) often employed in the first step does NOT consider geometric similarity

Original

Warped

### Feature Map Guided Texture Synthesis

• Basic Steps [Wu and Yu 2004]

• Maintain an input and output feature map

• Match and align curvilinear features

• Integrate feature maps into template-based texture synthesis

Sample

Feature Map

Graphcut

Quilting

### Acceleration Schemes

• Fourier Transform

• Acceleration techniques for nearest-neighbor search

• Tree-structured vector quantization

• Kd-trees

• Minimizing the candidate set

• Coherent synthesis, [Ashikhmin 2001]

• Precomputing candidate set

• K-coherent search, [Tong et al. 2002]

• Jump Map, [Zelinka and Garland 2002]

• GPU with parallel synthesis

• [Lefebvre and Hoppe 2005]

Original

Warped