wedgelets a multiscale geometric representation for images
Download
Skip this Video
Download Presentation
Wedgelets: A Multiscale Geometric Representation for Images

Loading in 2 Seconds...

play fullscreen
1 / 52

wedgelets: a multiscale geometric representation for images - PowerPoint PPT Presentation


  • 317 Views
  • Uploaded on

Wedgelets: A Multiscale Geometric Representation for Images. Michael Wakin Rice University dsp.rice.edu Joint work with Richard Baraniuk, Hyeokho Choi, Justin Romberg. Computational Harmonic Analysis. Representation

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'wedgelets: a multiscale geometric representation for images' - Sophia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
wedgelets a multiscale geometric representation for images

Wedgelets: AMultiscaleGeometricRepresentation for Images

Michael Wakin

Rice University

dsp.rice.edu

Joint work with Richard Baraniuk, Hyeokho Choi, Justin Romberg

computational harmonic analysis
Computational Harmonic Analysis
  • Representation
  • Analysis study through structure of should extract features of interest
  • Approximation exploit sparsity of

coefficients

basis, frame

nonlinear approximation
Nonlinear Approximation

few big

sorted index

many small

nonlinear approximation4
Nonlinear Approximation

few big

sorted index

  • -term approximation: use largestindependently
  • Greedy / thresholding
error approximation rates
Error Approximation Rates
  • Optimize asymptotic error decay rate

as

1 d piecewise smooth signals
1-D Piecewise Smooth Signals
  • smooth except for singularities at a finite number of 0-D points

Fourier sinusoids: suboptimal greedy approximation and extraction

wavelets: optimal greedy approximation

extract singularity structure

2 d piecewise smooth signals
2-D Piecewise Smooth Signals
  • smooth except for singularities along a finite number of smooth 1-D curves
  • Challenge: analyze/approximategeometric structure

geometry

texture

texture

wavelet based image processing
Wavelet-based Image Processing
  • Standard 2-D tensor product wavelet transform
wavelet challenges
Wavelet Challenges
  • Current wavelet methods suboptimal for 2-d piecewise smooth functions
  • Geometrical info not explicit
  • Inefficient -large number of significant WCs cluster around edge contours, no matter how smooth
wavelets and cartoons
Wavelets and Cartoons

13

26

52

  • Even for a smooth C2 contour, which straightens at fine scales…
2 d wavelets poor approximation
2-D Wavelets: Poor Approximation

13

26

52

  • Even for a smooth C2 contour, which straightens at fine scales…
  • Too many wavelets required!

-term wavelet approximation

not

slide12

Response: A Pursuit of Sparsity

I. Anisotropic, Directional Frames

  • curvelets
  • contourlets
  • bandelets
  • etc…
slide13

Response: A Pursuit of Sparsity

I. Anisotropic, Directional Frames

  • curvelets
  • contourlets
  • bandelets
  • etc…

II. Geometric Tilings / Dyadic Partitions

  • wedgelets
  • beamlets
  • platelets
  • etc…
geometry model for cartoons
Geometry Model for Cartoons

C2 boundary

  • Toy model: flat regions separated by smooth contours
  • Goal: representation that is
    • sparse
    • simply modeled
    • efficiently computed
    • extensible to texture regions (RichB -> Saturday)
2 d dyadic partition
2-D Dyadic Partition
  • Multiscale analysis
  • Zoom in by factor of 2 each scale
  • Represent image on each block
  • Partition/tiling

not a basis/frame

2 d dyadic partition quadtree
2-D Dyadic Partition = Quadtree
  • Each parent node has 4 children at next finer scale
  • Pruning reflects partitioning
  • Leaf nodes decorated with atoms
wedgelet representation donoho
Wedgelet Representation [Donoho]

r

  • Build a cartoon using wedgelets on dyadic squares
wedgelet representation
Wedgelet Representation

r

  • Build a cartoon using wedgelets on dyadic squares
  • Quad-tree structuredeeper in tree finer curve approximation
  • Decorate leaves with (r,q)

parameters

wedgelet representation19
Wedgelet Representation
  • Prune wedgelet quadtree to approximate local geometry (adaptive)
  • New problem:

How to find the proper approximation?

wedgelet inference
Wedgelet Inference
  • Find representation / prune tree to balance a fidelity vs. complexity trade-off
wedgelet inference21
Wedgelet Inference
  • Find representation / prune tree to balance a fidelity vs. complexity trade-off
  • For Comp(W) –measure the complexity of the wedgelet representation(quadtree + (r,q))
  • Donoho: Comp(W) = #leaves

NLA-like solution

wedgelet inference22
Wedgelet Inference
  • Find representation / prune tree to balance a fidelity vs. complexity trade-off
  • Exponential computational cost to solve brute force
  • Dynamic programming to the rescue!
slide23
distortion

complexity

slide24
distortion

Each leaf isindependent!

cost

top down greedy
Top-Down: Greedy

3

3

1

1

2

1

1

3

4

4

1

  • Label each node of the tree with
  • Move down from tree root and prune when

15

15

6

4

6

4

3

bottom up dynamic programming
Bottom-Up: Dynamic Programming

1

1

3

2

1

3

1

4

1

4

3

2

3

2

3

1

1

1

  • Label each node of the tree with
  • Move up from the leaves and prune whenelse

15

15

6

4

6

4

3

3

bottom up dynamic programming28
Bottom-Up: Dynamic Programming

1

3

1

1

3

1

4

1

3

4

1

1

3

2

2

  • Label each node of the tree with
  • Move up from the leaves and prune whenelse

15

15

6

4

5

4

3

bottom up dynamic programming29
Bottom-Up: Dynamic Programming

1

3

1

1

3

1

4

1

3

4

1

1

3

2

2

  • Label each node of the tree with
  • Move up from the leaves and prune whenelse

15

9

6

4

5

4

3

summary dynamic programming
Summary: Dynamic Programming
  • Simple recursive bottom-up algorithm
  • Efficient: O(N) for N-node tree
  • Finds optimal solution provided cost is additivewith independent components
wedgelet inference31
Wedgelet Inference
  • Find representation / prune tree to balance a fidelity vs. complexity trade-off
  • Optimal approximation
  • Near optimal rate distortion decay
  • Near optimal estimation [Donoho]
leaves complexity penalty
#Leaves Complexity Penalty
  • Accounts for wedgelet partition size, but not wedgelet orientation

#leaves

#leaves

“smooth”

“simple”

“rough”

“complex”

multiscale geometry model mgm34
Multiscale Geometry Model (MGM)
  • Decorate each tree node with orientation (r,q) and then model dependencies thru scale
multiscale geometry model mgm35
Multiscale Geometry Model (MGM)
  • Decorate each tree node with orientation (r,q) and then model dependencies thru scale
  • Insight: Smooth curve Geometric innovations small at fine scales
  • Model: Favor small innovations over large innovations (statistically)
multiscale geometry model mgm36
Multiscale Geometry Model (MGM)
  • Wavelet-like geometry model: coarse-to-fine prediction
    • model parent-to-child transitions of orientations

small

innovations

large

slide37
MGM
  • Wavelet-like geometry model: coarse-to-fine prediction
    • model parent-to-child transitions of orientations
  • Markov-1 statistical model
    • state = (r,q) orientation of wedgelet
    • parent-to-child state transition matrix
slide38
MGM
  • Markov-1 statistical model
  • Joint wedgelet Markov probability model:
  • Complexity = Shannon codelength == number of bits to encode
mgm and edge smoothness
MGM and Edge Smoothness

“smooth”

“simple”

“rough”

“complex”

mgm inference
MGM Inference
  • Find representation / prune tree to balance the fidelity vs. complexity trade-off
  • Efficient solution via dynamic programming
wedgelet coding of cartoon images
Wedgelet Coding of Cartoon Images
  • Choosing wedgelets = rate-distortion optimization

Shannon code length

to encode

predictive wedgelet coding
Predictive Wedgelet Coding

C2contour (1-d)

Optimal rate-distortion performance

compared to

for leaf-only encoding

slide43

Proof: Start with leaf-encoding Consider a wedgelet at scale j Block size is 2-jx2-j

O(2-2j)

2-j

C2 curve contained in strip with width ~ length2

slide46

A discrete wedgelet exists that deviates from curve by only O(2-2j); L2 distortion on this block = O(2-3j)

O(2-2j)

2-j

Since curve is C2, there are O(2j) such blocks

Total distortion D~O(2-2j)

slide47

Number of bits (bitrate) to encode tick marks = O(j) bits per block; thus, total rate R ~ O(j2j)

O(2-2j)

2-j

Combining: for leaf-only encoding

slide48

Can reduce bitrate by predicting wedgelets down through scale (MGM); consider scale j+1…

O(2-2(j+1))

2-(j+1)

2-(j+1)

slide49

Finer scale wedgelets within O(2-2(j+1)) of curve

Coarse scale wedgelet within O(2-2j) of curve

O(2-2(j+1))

2-(j+1)

2-(j+1)

Given the parent, there are only O(1) possible wedgelets on each finer block. Thus, rate is O(1) bits per block, or R~O(2j) bits total. This gives the optimal D(R)~R-2.

summary wedgelets
Summary: Wedgelets
  • Adaptive dyadic partition
  • Fit to data using fidelity/complexity optimization
  • Improve using MGM statistical model for parent-to-child wedgelet orientation correlations
  • MGM parent-to-child modeling akin to “wavelets” for wedgelets
  • Optimal approximation and R/D rates

[Dynamic programming as a useful tool]

  • Only suited for cartoon images
n d surflet representation vc mw db rb
n-D Surflet Representation [VC,MW,DB,RB]
  • Generalize wedgeletconcept to
    • arbitrary dimension n
    • arbitrary (n-1)-D manifold smoothness k
    • requires >linear elements
joint texture geometry modeling
Joint Texture/Geometry Modeling
  • Dictionary D = {wavelets} U {wedgelets}
  • Representation tradeoff: texture vs.geometry
  • Test case: approximation / compression
ad