Primal Sketch
This presentation is the property of its rightful owner.
Sponsored Links
1 / 51

Primal Sketch Integrating Structure and Texture PowerPoint PPT Presentation


  • 93 Views
  • Uploaded on
  • Presentation posted in: General

Primal Sketch Integrating Structure and Texture. Ying Nian Wu UCLA Department of Statistics Keck Meeting April 28, 2006. Guo, Zhu, Wu (ICCV, 2003; GMBV, 2004; CVIU, 2006). A Generative Model for Natural Images. texture regions. input image. sketch graph. +. =. synthesized image.

Download Presentation

Primal Sketch Integrating Structure and Texture

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Primal sketch integrating structure and texture

Primal Sketch

Integrating Structure and Texture

Ying Nian Wu

UCLA Department of Statistics

Keck Meeting

April 28, 2006

Guo, Zhu, Wu (ICCV, 2003; GMBV, 2004; CVIU, 2006)


Primal sketch integrating structure and texture

A Generative Model for Natural Images

texture regions

input image

sketch graph

+

=

synthesized image

sketchable image

synthesized textures


Primal sketch integrating structure and texture

Outline

Sparse coding

Markov random field

Primal sketch model

Sketch pursuit algorithm


Primal sketch integrating structure and texture

Sparse Coding

Olshausen and Field (1996)


Primal sketch integrating structure and texture

500

bases

800

bases

Matching Pursuit

Mallat and Zhang (1993)

matching

pursuit

input

image


Primal sketch integrating structure and texture

Symbolic representation

of 300 bases

Reconstructed image

Primak sketch


Primal sketch integrating structure and texture

Markov Random Fields

Markov Property:

MRF model = Gibbs distribution

Besag (1974)

Geman and Geman (1984)

Cross and Jain (1983)

One example of neighborhood


Primal sketch integrating structure and texture

MRF model & Image ensemble

MRF model (Zhu, Wu & Mumford, 1997)

Image ensemble (Wu, Zhu & Liu, 2000)


Primal sketch integrating structure and texture

Feature statistics: histograms of filter responses

(Heeger and Bergen, 1995)

Filtering – convolution

original image I

filter responses J

of the “dy” filter

a set of filters F


Primal sketch integrating structure and texture

Histogram of Filter Responses


Primal sketch integrating structure and texture

Average histogram error


Primal sketch integrating structure and texture

800 bases

A sample of image ensemble

with 5*13=65 parameters

50*70 patch


Primal sketch integrating structure and texture

observed image

sampled image from image ensemble

Primak sketch


Primal sketch integrating structure and texture

Sparse Coding vs. MRF

Sparse Coding models target low complexity patterns.

const: related to the dictionary

MRF models target high complexity patterns.

p*: fitted MRF

q: any distribution


Primal sketch integrating structure and texture

Primal Sketch Model

Image pixels = Sketchable & Non-sketchable

Sketchable: sparse coding using image primitives

Non-sketchable: feature statistics/Markov random fields

  • Integration:

  • Non-sketchable interpolates sketchable

  • Non-sketchable recycles failed sketch detections


Primal sketch integrating structure and texture

Sketches

Elder and Zucker, 1998


Primal sketch integrating structure and texture

Sketch Graph

Sketch graph

Vertices:

1,2,3 – corners

4,5,6,7 – end points

8,9,10 – junctions, etc


Primal sketch integrating structure and texture

Image Primitives

b) Photometric

a) Geometric


Primal sketch integrating structure and texture

Sketch Graph Model

Geometric

Photometric sketch image


Primal sketch integrating structure and texture

Sketchs = Gabor clusters

Alignment across spatial and frequency domains


Primal sketch integrating structure and texture

Integrating structure and texture

Sketch Graph

Sketches

Alignment

Gabor filters

Non-alignment

Textures

Pool marginal histograms


Primal sketch integrating structure and texture

Model fitting

First: Sketch pursuit aided by Gabors

Second: Non-sketchable texturing

Sketchability test


Primal sketch integrating structure and texture

Sketch pursuit objective

Approximated model


Primal sketch integrating structure and texture

Sketch Pursuit Phase I

input image

edge/ridge strength

Edge/ridge map

Proposals: a set of sketches as candidates.

Select the sketches in the order of likelihood gain.


Primal sketch integrating structure and texture

Sketch Pursuit Phase IIRefinement

Refinement

Initialization

Evolve the sketch graph by graph operators.


Primal sketch integrating structure and texture

Graph Operators


Primal sketch integrating structure and texture

A

B

Graph Editing

A

Phase I

B

Phase II


Primal sketch integrating structure and texture

Phase II Algorithm

  • Randomly choose a local sub-graph (S0)

  • Try all 10 pair of graph operators 1~ 5 steps, to generate a set of new graph candidates (S4,S2,S3)

  • Compare all new graph candidates

  • Select the one with the largest posterior gain (e.g. S4), accept the new graph. If no positive gain, no update.

  • Repeat 1~4 until no update

S0

G1

G3

S1

S2

S3

G4

S4


Primal sketch integrating structure and texture

texture regions

synthesized textures

K-mean clustering

Histograms in 7x7 window

7 filters x 7 bins


Primal sketch integrating structure and texture

Primal Sketch Model Result

input image

sketch graph

sketchable image

reconstructed image


Primal sketch integrating structure and texture

input image

sketch graph

reconstructed image

sketchable image


Primal sketch integrating structure and texture

input image

sketch graph

reconstructed image


Primal sketch integrating structure and texture

input image

reconstructed image

sketch graph


Primal sketch integrating structure and texture

Lossy Image Coding

codes for the vertices:

152*2*9 = 2,736 bits

codes for the strokes:

275*2*4.7 = 2, 585 bits

sketch graph

codes for the profiles:

275*(2.4*8+1.4*2) = 6,050 bits

Total codes for structures (18,185 pixels)

11,371 bits = 1,421 bytes

sketchable image


Primal sketch integrating structure and texture

codes for the region boundaries:

3659*3 = 10,977 bits

texture regions

codes for the texture histograms:

7*5*13*4.5 =2,048 bits

Total codes for textures (41,815 pixels)

13,025 bits = 1,628 bytes

Total codes for whole image

(72,000 pixels), 3,049 bytes

synthesized textures


Primal sketch integrating structure and texture

Scaling


Primal sketch integrating structure and texture

Scaling


Primal sketch integrating structure and texture

Scaling

Wu, Zhu, Bahrami, Li (2006)


  • Login