Inverse texture synthesis
This presentation is the property of its rightful owner.
Sponsored Links
1 / 29

Inverse Texture Synthesis PowerPoint PPT Presentation


  • 98 Views
  • Uploaded on
  • Presentation posted in: General

Inverse Texture Synthesis. Li-Yi Wei 1 Jianwei Han 2 Kun Zhou 1 , 2 Hujun Bao 2 Baining Guo 1 Harry Shum 1 1 Microsoft 2 Zhejiang University. Example-based texture synthesis. For a small input texture produce an arbitrarily large output with similar look

Download Presentation

Inverse Texture Synthesis

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Inverse Texture Synthesis

Li-Yi Wei1 Jianwei Han2 Kun Zhou1,2

Hujun Bao2 Baining Guo1 Harry Shum1

1Microsoft2Zhejiang University


Example-based texture synthesis

  • For a small input texture

    • produce an arbitrarily large output with similar look

  • Why? may not possible to obtain large input

texture synthesis

input

output


Inverse texture synthesis

  • From a large input texture

    • produce a small output that best summarizes input

inverse texture synthesis

output

input


Why?

  • Textures are getting large

    • Advances in scanning technology

    • High dimensionality: time-varying, BRDF

    • Expensive to store, transmit, compute

Yale University

MSR Asia

Columbia University


Overview

inverse texture synthesis

input

(large)

output

(small)

texturing

(fast)

texturing

(slow)

similar quality


Related work: image compression

pixel-wise

identical

compress

decompress

inverse synth

texture synth

input

perceptual

similar


Related work: epitome

  • Epitome [Jojic et al. 2003]

  • Jigsaw [Kannan et al. 2007]

    • Major source of inspiration for us

    • For general images, not just textures

    • We provide better quality

  • Bidirectional similarity [Simakov et al. 2008]

  • Factoring repeated content [Wang et al. 2008]


Related work: manual crop

stationary

globally

varying

original

manual crop

our result


Globally-varying textures

  • Markov Random Field (MRF) textures

    • local & stationary

  • Globally-varying textures

    • local, but not necessarily stationary

MRF

globally varying


Globally varying texturesPrevious work

MRF input → globally varying output

texture-by-numbers in Image analogies [Hertzmann et al. 2001]

progressively variant textures [Zhang et al. 2003]

texture design and morphing [Matusik et al. 2005]

Globally varying input

appearance manifold [Wang et al. 2006]

spatially & time varying BRDF [Gu et al. 2006]

context-aware texture [Lu et al. 2007]


Globally varying texturesDefinition

texture + control maps

Examples of control maps

user-specified colors [Hertzmann et al. 2001]

spatially-varying parameters [Gu et al. 2006]

weathering degree-map [Wang et al. 2006]

context information [Lu et al. 2007]

texture (paint crack)

control map (paint thickness)


Globally varying textures

Including time-varying textures as well

Large data size!

time-varying BRDF

[Gu et al. 2006]

512 x 512 x 33, 288 MB

context-aware texture

[Lu et al. 2007]

1226 x 978 x 50, 35 MB


Inverse texture synthesis

Compacting globally varying textures

including both texture + control map

inverse synthesis

texture

control

texture

control map

input

output compaction


Compaction as summary of original

  • Re-synthesis with user control map

faster

slower

forward

synthesis

+

compaction

user control

re-synthesis

from

compaction

re-synthesis

from

original


inverse term (New!)

forward term [Kwatra et al. 2005]

Basic formulation

  • Inspired by texture optimization [Kwatra et al. 2005]

xp

Zp

best match

zq

best match

Z (output)

xq

X (input)


Energy plot

energy

original

compaction size


Why both terms?

inverse

forward

  • inverse term preserves all input features

  • forward term avoids artifacts in compaction

both

both

f-only

missing feature

i-only

garbage

both

i-only

discontinuity


Comparing with epitome [Jojic et al. 2003]

  • Similar to our method but only inverse term

    • blur, discontinuity

epitome

epitome

our

our

original

original


Comparing with epitome [Jojic et al. 2003]Re-synthesis

epitome

epitome

our

our

original

original


Solver

  • How to solve this?

    • Texture optimization [Kwatra et al. 2005]

    • Discrete solver [Han et al. 2006]


NO inverse term

forward term [Kwatra et al. 2005]

zq

xq

Optimization [Kwatra et al. 2005]

  • E-step

    • fix xq

    • argminz E(x,z)

    • least square

  • M-step

    • fix Z

    • argminxq |xq-zq|2

    • search

fix xq

xq

Zq

Z

argminxq |xq-zq|2

X


inverse term

forward term [Kwatra et al. 2005]

zp

zq

xp

xq

Our solver

  • E-step

    • fix xq

    • argminz E(x,z)

    • least square

  • M-step (forward)

    • fix Z

    • argminxq |xq-zq|2

    • search

xp

xq

Zq

discrete solver [Han et al. 2006]

  • M-step (inverse)

    • fix xp

    • argminzp |xp-zp|2

    • discrete solver

Z

argminxq |xq-zq|2

discrete solver

X


Results


GPU synthesis – small texture betterExtension from [Lefebvre & Hoppe 2005]

3 fps, original

6 fps, compact

cheese

mold

1214 x 1212

1282

3.5 fps, original

7.0 fps, compact

dirt

271x481

1282

original

compaction


Limitation:Correlation between texture & control

texture

control

original

reconstruction

compaction


Orientation field for anisotropic textures

  • Orientation field w as part of energy function

    • E(x, z) → E(x, z; w)

  • Good orientation field yields better solution

comp.

no w

comp.

with w

original

orientation field


Future work

  • Higher dimensional textures

    • e.g. video

  • General images, not just textures

    • Bidirectional similarity [Simakov et al. CVPR 2008]

  • Image compression


Acknowledgements

  • Yale graphics group

  • Columbia graphics group

  • Sylvain Lefebvre

  • Hughes Hoppe

  • Matusik et al. 2005

  • Mayang.com

  • Jiaping Wang

  • Xin Tong

  • Jian Sun

  • Frank Yu

  • Bennett Wilburn

  • Eric Stollnitz

  • Dwight Daniels

  • Reviewers

  • Dinesh Manocha

  • Ming Lin

  • Chas Boyd

  • Brandon Lloyd

  • Avneesh Sud

  • Billy Chen


Thank You!


  • Login