2D Texture Synthesis

1 / 30

# 2D Texture Synthesis - PowerPoint PPT Presentation

2D Texture Synthesis. Instructor: Yizhou Yu. Texture synthesis. Goal: increase texture resolution yet keep local texture variation. Synthesis by global statistics. Idea: Obtain statistics of input image, match with output image Histograms: normalized graph of intensity frequencies.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' 2D Texture Synthesis' - yoshi-mcfarland

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### 2D Texture Synthesis

Instructor: Yizhou Yu

Texture synthesis
• Goal: increase texture resolution yet keep local texture variation
Synthesis by global statistics
• Idea: Obtain statistics of input image, match with output image
• Histograms: normalized graph of intensity frequencies
Synthesis by global statistics
• Consider extreme case:
• Histogram matching can generate two images:
Using higher-order global stats
• Histogram matching does not take into account spatial information.
• For every pixel, examine local neighborhood
• Gives local but not global features (e.g. – veins)
• Use higher order stats:
• Correlation
• average of the product of the intensities =
• tells how interdependent pixel values are
General procedure
• 1.) Define error metric:
• What do you want to match?
• 2.) Match statistics
• Histogram matching:
• Get the density value of a pixel in the output image. Map this density to a pixel intensity in the input image  Overwrite the pixel in the output image with the mapped pixel intensity in the input image
Pyramid-based Texture Analysis/Synthesis
• Paper by David J. Heeger and James R. Bergen from SIGGRAPH 1995
• http://www.cns.nyu.edu/~david/ftp/reprints/heeger-siggraph95.pdf
Pyramid-based Synthesis
• Downsample image several times and keep track of only differences in a feature image
• To reconstruct, upsample small images and add differences
• Pixels in feature images are close to zero
• Provides multiple scales with each feature image representing different feature sizes
Laplacian Pyramid
• 1-D filter: (1/16) (1 4 6 4 1)
• To generalize to 2D, use tensor product
• Normalize by dividing by 256
Upsampling
• To upsample, copy pixels to every other pixel in larger image
• Fill the rest with zeroes, run low pass filter to fill in values
Oriented Filters
• Gaussian filters are symmetric, thus edges and contours are not detected
• Use multiple oriented filters to catch non-symmetric features

Left top series: Oriented filters Right image: Texture

Left bottom series: Filtered textures

Pyramids with Oriented Filters
• Each oriented filter creates its own feature image
• Thus, for each downsampled image, keep one image for each oriented filter
• Each oriented filter captures one orientation of lines
Matching Image Features
• Input parameters:
• noise: initial noisy texture
• texture: texture to be matched
• Output texture is stored in noise
Matching Image Features
• Helper functions:
• MatchHistogram(noise, texture)
• Matches histogram, using method described last time
• MakePyramid(texture)
• Create pyramid images (base and feature images)
• CollapsePyramid(pyramid)
• Constructs high resolution image from base and feature images
Matching Image Features

MatchTexture(noise, texture)

{

MatchHistogram(noise, texture) // first-order matching

analysis_pyr = MakePyramid(texture) // create pyramid from input texture

for several iterations

synthesis_pyr = MakePyramid(noise) // create pyramid from noise

for each feature image, fi of analysis_pyr

for each feature image, fj of synthesis_pyr of same orientation

MatchHistogram(fi, fj)

end for

end for

noise = CollapsePyramid(synthesis_pyr)

MatchHistogram(noise, texture)

end for

}

Matching Image Features
• MatchTexture matches histograms of feature images, not pixel values, thus providing a much better matching
• Feature images already consider local neighborhoods, resulting in better approximations
• Good for randomized textures
• Textures with large scale features or thin and long features not matched well
• e.g. stripes in wood, lines in coral
Texture Synthesis Using Local Neighborhoods
• Main goal is to keep local spatial coherence, but not global stats.
• Randomized method
• Pick pixel and copy it along with its neighborhood to random parts in synthesized image
• If two neighborhoods overlap, just blend
• This can result in features getting cut off if larger than local neighborhood
Neighborhood-Based Texture Synthesis
• Patch-Based
• Patch-based sampling achieves real-time speed. [Liang et. al. 2001]
• Image quilting: high-quality results and simple implementation [Efros & Freeman 2001]
• Graph cut provides a powerful and refinable scheme. [Kwantra et. al. 2003]
• Pixel-Based
• [Efros & Leung 99], [Wei & Levoy 2000], [Ashikhmin 2001], [Hertzmann et. al. 2001], [Zhang et. al. 2003]
Pixel-wise Synthesis
• Grow pixel by pixel
• Start from an existing patch of the texture (as opposed to noise texture like Pyramid-based Synthesis)
• Look for regions in input texture most similar to current region in new texture
• Copy pixels next to best-match region to expand new texture
Following Raster Order
• Blue region: already set by algorithm
• Green region: compare this region to input image
• Yellow region: closest match
• Red region: replace these pixels with magenta region to maintain local integrity
Hierarchical Synthesis
• Build a multi-resolution pyramid for the example texture
• Generate a synthesized pyramid for the output texture
• At each level, follow the raster order
Randomizing Synthesis
• Instead of picking pixels from closest matching region, use threshold to pick a few candidate matches
• Randomly pick one of the candidates
• Refer to “Texture Synthesis by Non-parametric Sampling”
• http://www.cs.berkeley.edu/~efros/research/synthesis.html
Patch-Based Synthesis
• Search in a sample texture for neighborhoods most similar to a context region
• Merge a patch with the partially synthesized output texture
The Seam Problem
• Feature discontinuities may appear in patch-based synthesis.
The Second Reason
• Didn’t find the smoothest transition between the incoming patch and context region.
• Solutions: use dynamic programming [Efros and Freeman 2001] or graph cut [Kwatra et al. 2003] to find an optimal cut.

Original

Warped

The First Reason
• Rigid template matching (SSD) often employed in the first step does NOT consider geometric similarity

Original

Warped

Feature Map Guided Texture Synthesis
• Basic Steps [Wu and Yu 2004]
• Maintain an input and output feature map
• Match and align curvilinear features
• Integrate feature maps into template-based texture synthesis
Comparisons

Sample

Feature Map

Graphcut

Quilting

Acceleration Schemes
• Fourier Transform
• Acceleration techniques for nearest-neighbor search
• Tree-structured vector quantization
• Kd-trees
• Minimizing the candidate set
• Coherent synthesis, [Ashikhmin 2001]
• Precomputing candidate set
• K-coherent search, [Tong et al. 2002]
• Jump Map, [Zelinka and Garland 2002]
• GPU with parallel synthesis
• [Lefebvre and Hoppe 2005]

Original

Warped