an introduction to texture synthesis n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
An Introduction to Texture Synthesis PowerPoint Presentation
Download Presentation
An Introduction to Texture Synthesis

Loading in 2 Seconds...

play fullscreen
1 / 36

An Introduction to Texture Synthesis - PowerPoint PPT Presentation


  • 66 Views
  • Uploaded on

An Introduction to Texture Synthesis. 2008/05/29. Desirable Properties. Result looks like the input Efficient General Easy to use Extensible. The Challenge. Need to model the whole spectrum: from repeated to stochastic texture. repeated. stochastic. Both?.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'An Introduction to Texture Synthesis' - enrico


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
desirable properties
Desirable Properties

Result looks like the input

Efficient

General

Easy to use

Extensible

the challenge
The Challenge

Need to model the whole spectrum: from repeated to stochastic texture

repeated

stochastic

Both?

slide4
[Shannon,’48] proposed a way to generate English-looking text using N-grams:
  • Assume a generalized Markov model
  • Use a large text to compute prob. distributions of each letter given N-1 previous letters
  • Starting from a seed repeatedly sample this Markov chain to generate new letters
  • Also works for whole words
slide5

p

non-parametric

sampling

Assuming Markov property, compute P(p|N(p))

  • Building explicit probability tables infeasible
  • Instead, let’s search the input image for all similar neighborhoods — that’s our histogram for p

To synthesize p, just pick one match at random

Input image

Synthesizing a pixel

basic algorithm
Basic Algorithm

Exhaustively search neighborhoods

neighborhood
Neighborhood

Neighborhood size determines the quality & cost

33

55

77

739 s

423 s

528 s

1111

4141

99

24350 s

1020 s

1445 s

efros leung 99
Efros & Leung ’99

The algorithm

  • Very simple
  • Surprisingly good results
  • Synthesis is easier than analysis!
  • …but very slow

Optimizations and Improvements

  • Multi-resolution Pyramid
  • Jump Map
  • Chaos Mosaic
multi resolution pyramid
Multi-resolution Pyramid

High resolution

Low resolution

multi resolution pyramid1
Multi-resolution Pyramid

Better image quality & faster computation

1 level

55

1 level

1111

3 levels

55

what is a jump map
What is a Jump Map?

Same size as input

what is a jump map1
What is a Jump Map?

Same size as input

Set of jumps per pixel

what is a jump map2
What is a Jump Map?

0.4

0.2

0.3

  • Same size as input
  • Set of jumps per pixel
  • Jumps are weighted according to similarity
    • Need not sum to 1
chaos mosaic xu guo shum 00
Chaos Mosaic [Xu, Guo & Shum, ‘00]

Process: 1) tile input image; 2) pick random blocks and place them in random locations 3) Smooth edges

input

idea

result

Used in Lapped Textures [Praun et.al,’00]

chaos mosaic xu guo shum 001
Chaos Mosaic [Xu, Guo & Shum, ‘00]

Of course, doesn’t work for structured textures

input

result

image quilting
Image Quilting

Idea:

  • let’s combine random block placement of Chaos Mosaic with spatial constraints of Efros & Leung
efros leung 99 extended
Efros & Leung ’99 extended

B

p

Synthesizing a block

  • Idea: unit of synthesis = block
    • Exactly the same but now we want P(B|N(B))
    • Much faster: synthesize all pixels in a block at once
    • Not the same as multi-scale!

non-parametric

sampling

Observation: neighbor pixels are highly correlated

Input image

slide20

B1

B1

B2

B2

Neighboring blocks

constrained by overlap

Minimal error

boundary cut

block

Input texture

B1

B2

Random placement

of blocks

minimal error boundary
Minimal error boundary

2

_

=

overlap error

min. error boundary

overlapping blocks

vertical boundary

algorithm
Algorithm
  • Pick size of block and size of overlap
  • Synthesize blocks in raster order
  • Search input texture for block that satisfies overlap constraints (above and left)
    • Easy to optimize using NN search
  • Paste new block into resulting texture
    • use dynamic programming to compute minimal error boundary cut
slide27

Failures

(Chernobyl

Harvest)

texture transfer
Texture Transfer

Take the texture from one object and “paint” it onto another object

  • This requires separating texture and shape
  • That’s HARD, but we can cheat
  • Assume we can capture shape by boundary and rough shading

Then, just add another constraint when sampling: similarity to underlying image at that spot

slide30

parmesan

+

=

rice

+

=

slide31

+

=

slide32

Source

correspondence

image

Target

correspondence image

Source

texture

Target image

slide33

+

=

slide34

Portilla & Simoncelli

Xu, Guo & Shum

input image

Wei & Levoy

Image Quilting

slide35

Portilla & Simoncelli

Xu, Guo & Shum

input image

Wei & Levoy

Image Quilting

slide36

Homage to Shannon!

Portilla & Simoncelli

Xu, Guo & Shum

input image

Wei & Levoy

Image Quilting