Inverse texture synthesis
This presentation is the property of its rightful owner.
Sponsored Links
1 / 29

Inverse Texture Synthesis PowerPoint PPT Presentation


  • 93 Views
  • Uploaded on
  • Presentation posted in: General

Inverse Texture Synthesis. Li-Yi Wei 1 Jianwei Han 2 Kun Zhou 1 , 2 Hujun Bao 2 Baining Guo 1 Harry Shum 1 1 Microsoft 2 Zhejiang University. Example-based texture synthesis. For a small input texture produce an arbitrarily large output with similar look

Download Presentation

Inverse Texture Synthesis

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Inverse texture synthesis

Inverse Texture Synthesis

Li-Yi Wei1 Jianwei Han2 Kun Zhou1,2

Hujun Bao2 Baining Guo1 Harry Shum1

1Microsoft2Zhejiang University


Example based texture synthesis

Example-based texture synthesis

  • For a small input texture

    • produce an arbitrarily large output with similar look

  • Why? may not possible to obtain large input

texture synthesis

input

output


Inverse texture synthesis1

Inverse texture synthesis

  • From a large input texture

    • produce a small output that best summarizes input

inverse texture synthesis

output

input


Inverse texture synthesis

Why?

  • Textures are getting large

    • Advances in scanning technology

    • High dimensionality: time-varying, BRDF

    • Expensive to store, transmit, compute

Yale University

MSR Asia

Columbia University


Overview

Overview

inverse texture synthesis

input

(large)

output

(small)

texturing

(fast)

texturing

(slow)

similar quality


Related work image compression

Related work: image compression

pixel-wise

identical

compress

decompress

inverse synth

texture synth

input

perceptual

similar


Related work epitome

Related work: epitome

  • Epitome [Jojic et al. 2003]

  • Jigsaw [Kannan et al. 2007]

    • Major source of inspiration for us

    • For general images, not just textures

    • We provide better quality

  • Bidirectional similarity [Simakov et al. 2008]

  • Factoring repeated content [Wang et al. 2008]


Related work manual crop

Related work: manual crop

stationary

globally

varying

original

manual crop

our result


Globally varying textures

Globally-varying textures

  • Markov Random Field (MRF) textures

    • local & stationary

  • Globally-varying textures

    • local, but not necessarily stationary

MRF

globally varying


Globally varying textures previous work

Globally varying texturesPrevious work

MRF input → globally varying output

texture-by-numbers in Image analogies [Hertzmann et al. 2001]

progressively variant textures [Zhang et al. 2003]

texture design and morphing [Matusik et al. 2005]

Globally varying input

appearance manifold [Wang et al. 2006]

spatially & time varying BRDF [Gu et al. 2006]

context-aware texture [Lu et al. 2007]


Globally varying textures definition

Globally varying texturesDefinition

texture + control maps

Examples of control maps

user-specified colors [Hertzmann et al. 2001]

spatially-varying parameters [Gu et al. 2006]

weathering degree-map [Wang et al. 2006]

context information [Lu et al. 2007]

texture (paint crack)

control map (paint thickness)


Globally varying textures1

Globally varying textures

Including time-varying textures as well

Large data size!

time-varying BRDF

[Gu et al. 2006]

512 x 512 x 33, 288 MB

context-aware texture

[Lu et al. 2007]

1226 x 978 x 50, 35 MB


Inverse texture synthesis2

Inverse texture synthesis

Compacting globally varying textures

including both texture + control map

inverse synthesis

texture

control

texture

control map

input

output compaction


Compaction as summary of original

Compaction as summary of original

  • Re-synthesis with user control map

faster

slower

forward

synthesis

+

compaction

user control

re-synthesis

from

compaction

re-synthesis

from

original


Basic formulation

inverse term (New!)

forward term [Kwatra et al. 2005]

Basic formulation

  • Inspired by texture optimization [Kwatra et al. 2005]

xp

Zp

best match

zq

best match

Z (output)

xq

X (input)


Energy plot

Energy plot

energy

original

compaction size


Why both terms

Why both terms?

inverse

forward

  • inverse term preserves all input features

  • forward term avoids artifacts in compaction

both

both

f-only

missing feature

i-only

garbage

both

i-only

discontinuity


Comparing with epitome jojic et al 2003

Comparing with epitome [Jojic et al. 2003]

  • Similar to our method but only inverse term

    • blur, discontinuity

epitome

epitome

our

our

original

original


Comparing with epitome jojic et al 2003 re synthesis

Comparing with epitome [Jojic et al. 2003]Re-synthesis

epitome

epitome

our

our

original

original


Solver

Solver

  • How to solve this?

    • Texture optimization [Kwatra et al. 2005]

    • Discrete solver [Han et al. 2006]


Optimization kwatra et al 2005

NO inverse term

forward term [Kwatra et al. 2005]

zq

xq

Optimization [Kwatra et al. 2005]

  • E-step

    • fix xq

    • argminz E(x,z)

    • least square

  • M-step

    • fix Z

    • argminxq |xq-zq|2

    • search

fix xq

xq

Zq

Z

argminxq |xq-zq|2

X


Our solver

inverse term

forward term [Kwatra et al. 2005]

zp

zq

xp

xq

Our solver

  • E-step

    • fix xq

    • argminz E(x,z)

    • least square

  • M-step (forward)

    • fix Z

    • argminxq |xq-zq|2

    • search

xp

xq

Zq

discrete solver [Han et al. 2006]

  • M-step (inverse)

    • fix xp

    • argminzp |xp-zp|2

    • discrete solver

Z

argminxq |xq-zq|2

discrete solver

X


Results

Results


Gpu synthesis small texture better extension from lefebvre hoppe 2005

GPU synthesis – small texture betterExtension from [Lefebvre & Hoppe 2005]

3 fps, original

6 fps, compact

cheese

mold

1214 x 1212

1282

3.5 fps, original

7.0 fps, compact

dirt

271x481

1282

original

compaction


Limitation correlation between texture control

Limitation:Correlation between texture & control

texture

control

original

reconstruction

compaction


Orientation field for anisotropic textures

Orientation field for anisotropic textures

  • Orientation field w as part of energy function

    • E(x, z) → E(x, z; w)

  • Good orientation field yields better solution

comp.

no w

comp.

with w

original

orientation field


Future work

Future work

  • Higher dimensional textures

    • e.g. video

  • General images, not just textures

    • Bidirectional similarity [Simakov et al. CVPR 2008]

  • Image compression


Acknowledgements

Acknowledgements

  • Yale graphics group

  • Columbia graphics group

  • Sylvain Lefebvre

  • Hughes Hoppe

  • Matusik et al. 2005

  • Mayang.com

  • Jiaping Wang

  • Xin Tong

  • Jian Sun

  • Frank Yu

  • Bennett Wilburn

  • Eric Stollnitz

  • Dwight Daniels

  • Reviewers

  • Dinesh Manocha

  • Ming Lin

  • Chas Boyd

  • Brandon Lloyd

  • Avneesh Sud

  • Billy Chen


Thank you

Thank You!


  • Login