Wedgelets a multiscale geometric representation for images
Download
1 / 52

wedgelets: a multiscale geometric representation for images - PowerPoint PPT Presentation


  • 317 Views
  • Updated On :

Wedgelets: A Multiscale Geometric Representation for Images. Michael Wakin Rice University dsp.rice.edu Joint work with Richard Baraniuk, Hyeokho Choi, Justin Romberg. Computational Harmonic Analysis. Representation

Related searches for wedgelets: a multiscale geometric representation for images

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'wedgelets: a multiscale geometric representation for images' - Sophia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Wedgelets a multiscale geometric representation for images l.jpg

Wedgelets: AMultiscaleGeometricRepresentation for Images

Michael Wakin

Rice University

dsp.rice.edu

Joint work with Richard Baraniuk, Hyeokho Choi, Justin Romberg


Computational harmonic analysis l.jpg
Computational Harmonic Analysis

  • Representation

  • Analysis study through structure of should extract features of interest

  • Approximation exploit sparsity of

coefficients

basis, frame


Nonlinear approximation l.jpg
Nonlinear Approximation

few big

sorted index

many small


Nonlinear approximation4 l.jpg
Nonlinear Approximation

few big

sorted index

  • -term approximation: use largestindependently

  • Greedy / thresholding


Error approximation rates l.jpg
Error Approximation Rates

  • Optimize asymptotic error decay rate

as


1 d piecewise smooth signals l.jpg
1-D Piecewise Smooth Signals

  • smooth except for singularities at a finite number of 0-D points

    Fourier sinusoids: suboptimal greedy approximation and extraction

    wavelets: optimal greedy approximation

    extract singularity structure


2 d piecewise smooth signals l.jpg
2-D Piecewise Smooth Signals

  • smooth except for singularities along a finite number of smooth 1-D curves

  • Challenge: analyze/approximategeometric structure

geometry

texture

texture


Wavelet based image processing l.jpg
Wavelet-based Image Processing

  • Standard 2-D tensor product wavelet transform


Wavelet challenges l.jpg
Wavelet Challenges

  • Current wavelet methods suboptimal for 2-d piecewise smooth functions

  • Geometrical info not explicit

  • Inefficient -large number of significant WCs cluster around edge contours, no matter how smooth


Wavelets and cartoons l.jpg
Wavelets and Cartoons

13

26

52

  • Even for a smooth C2 contour, which straightens at fine scales…


2 d wavelets poor approximation l.jpg
2-D Wavelets: Poor Approximation

13

26

52

  • Even for a smooth C2 contour, which straightens at fine scales…

  • Too many wavelets required!

-term wavelet approximation

not


Slide12 l.jpg

Response: A Pursuit of Sparsity

I. Anisotropic, Directional Frames

  • curvelets

  • contourlets

  • bandelets

  • etc…


Slide13 l.jpg

Response: A Pursuit of Sparsity

I. Anisotropic, Directional Frames

  • curvelets

  • contourlets

  • bandelets

  • etc…

    II. Geometric Tilings / Dyadic Partitions

  • wedgelets

  • beamlets

  • platelets

  • etc…


Geometry model for cartoons l.jpg
Geometry Model for Cartoons

C2 boundary

  • Toy model: flat regions separated by smooth contours

  • Goal: representation that is

    • sparse

    • simply modeled

    • efficiently computed

    • extensible to texture regions (RichB -> Saturday)


2 d dyadic partition l.jpg
2-D Dyadic Partition

  • Multiscale analysis

  • Zoom in by factor of 2 each scale

  • Represent image on each block

  • Partition/tiling

    not a basis/frame


2 d dyadic partition quadtree l.jpg
2-D Dyadic Partition = Quadtree

  • Each parent node has 4 children at next finer scale

  • Pruning reflects partitioning

  • Leaf nodes decorated with atoms


Wedgelet representation donoho l.jpg
Wedgelet Representation [Donoho]

r

  • Build a cartoon using wedgelets on dyadic squares


Wedgelet representation l.jpg
Wedgelet Representation

r

  • Build a cartoon using wedgelets on dyadic squares

  • Quad-tree structuredeeper in tree finer curve approximation

  • Decorate leaves with (r,q)

    parameters


Wedgelet representation19 l.jpg
Wedgelet Representation

  • Prune wedgelet quadtree to approximate local geometry (adaptive)

  • New problem:

    How to find the proper approximation?


Wedgelet inference l.jpg
Wedgelet Inference

  • Find representation / prune tree to balance a fidelity vs. complexity trade-off


Wedgelet inference21 l.jpg
Wedgelet Inference

  • Find representation / prune tree to balance a fidelity vs. complexity trade-off

  • For Comp(W) –measure the complexity of the wedgelet representation(quadtree + (r,q))

  • Donoho: Comp(W) = #leaves

    NLA-like solution


Wedgelet inference22 l.jpg
Wedgelet Inference

  • Find representation / prune tree to balance a fidelity vs. complexity trade-off

  • Exponential computational cost to solve brute force

  • Dynamic programming to the rescue!


Slide23 l.jpg

distortion

complexity


Slide24 l.jpg

distortion

Each leaf isindependent!

cost



Top down greedy l.jpg
Top-Down: Greedy

3

3

1

1

2

1

1

3

4

4

1

  • Label each node of the tree with

  • Move down from tree root and prune when

15

15

6

4

6

4

3


Bottom up dynamic programming l.jpg
Bottom-Up: Dynamic Programming

1

1

3

2

1

3

1

4

1

4

3

2

3

2

3

1

1

1

  • Label each node of the tree with

  • Move up from the leaves and prune whenelse

15

15

6

4

6

4

3

3


Bottom up dynamic programming28 l.jpg
Bottom-Up: Dynamic Programming

1

3

1

1

3

1

4

1

3

4

1

1

3

2

2

  • Label each node of the tree with

  • Move up from the leaves and prune whenelse

15

15

6

4

5

4

3


Bottom up dynamic programming29 l.jpg
Bottom-Up: Dynamic Programming

1

3

1

1

3

1

4

1

3

4

1

1

3

2

2

  • Label each node of the tree with

  • Move up from the leaves and prune whenelse

15

9

6

4

5

4

3


Summary dynamic programming l.jpg
Summary: Dynamic Programming

  • Simple recursive bottom-up algorithm

  • Efficient: O(N) for N-node tree

  • Finds optimal solution provided cost is additivewith independent components


Wedgelet inference31 l.jpg
Wedgelet Inference

  • Find representation / prune tree to balance a fidelity vs. complexity trade-off

  • Optimal approximation

  • Near optimal rate distortion decay

  • Near optimal estimation [Donoho]


Leaves complexity penalty l.jpg
#Leaves Complexity Penalty

  • Accounts for wedgelet partition size, but not wedgelet orientation

#leaves

#leaves

“smooth”

“simple”

“rough”

“complex”



Multiscale geometry model mgm34 l.jpg
Multiscale Geometry Model (MGM)

  • Decorate each tree node with orientation (r,q) and then model dependencies thru scale


Multiscale geometry model mgm35 l.jpg
Multiscale Geometry Model (MGM)

  • Decorate each tree node with orientation (r,q) and then model dependencies thru scale

  • Insight: Smooth curve Geometric innovations small at fine scales

  • Model: Favor small innovations over large innovations (statistically)


Multiscale geometry model mgm36 l.jpg
Multiscale Geometry Model (MGM)

  • Wavelet-like geometry model: coarse-to-fine prediction

    • model parent-to-child transitions of orientations

small

innovations

large


Slide37 l.jpg
MGM

  • Wavelet-like geometry model: coarse-to-fine prediction

    • model parent-to-child transitions of orientations

  • Markov-1 statistical model

    • state = (r,q) orientation of wedgelet

    • parent-to-child state transition matrix


Slide38 l.jpg
MGM

  • Markov-1 statistical model

  • Joint wedgelet Markov probability model:

  • Complexity = Shannon codelength == number of bits to encode


Mgm and edge smoothness l.jpg
MGM and Edge Smoothness

“smooth”

“simple”

“rough”

“complex”


Mgm inference l.jpg
MGM Inference

  • Find representation / prune tree to balance the fidelity vs. complexity trade-off

  • Efficient solution via dynamic programming


Wedgelet coding of cartoon images l.jpg
Wedgelet Coding of Cartoon Images

  • Choosing wedgelets = rate-distortion optimization

Shannon code length

to encode


Predictive wedgelet coding l.jpg
Predictive Wedgelet Coding

C2contour (1-d)

Optimal rate-distortion performance

compared to

for leaf-only encoding


Slide43 l.jpg

Proof: Start with leaf-encoding Consider a wedgelet at scale j Block size is 2-jx2-j

O(2-2j)

2-j

C2 curve contained in strip with width ~ length2




Slide46 l.jpg

A discrete wedgelet exists that deviates from curve by only spacing O(2-2j); L2 distortion on this block = O(2-3j)

O(2-2j)

2-j

Since curve is C2, there are O(2j) such blocks

Total distortion D~O(2-2j)


Slide47 l.jpg

Number of bits (bitrate) to encode tick marks = spacing O(j) bits per block; thus, total rate R ~ O(j2j)

O(2-2j)

2-j

Combining: for leaf-only encoding


Slide48 l.jpg

Can reduce bitrate by predicting wedgelets down through scale (MGM); consider scale j+1…

O(2-2(j+1))

2-(j+1)

2-(j+1)


Slide49 l.jpg

Finer scale wedgelets within scale (MGM); consider scale O(2-2(j+1)) of curve

Coarse scale wedgelet within O(2-2j) of curve

O(2-2(j+1))

2-(j+1)

2-(j+1)

Given the parent, there are only O(1) possible wedgelets on each finer block. Thus, rate is O(1) bits per block, or R~O(2j) bits total. This gives the optimal D(R)~R-2.


Summary wedgelets l.jpg
Summary: Wedgelets scale (MGM); consider scale

  • Adaptive dyadic partition

  • Fit to data using fidelity/complexity optimization

  • Improve using MGM statistical model for parent-to-child wedgelet orientation correlations

  • MGM parent-to-child modeling akin to “wavelets” for wedgelets

  • Optimal approximation and R/D rates

    [Dynamic programming as a useful tool]

  • Only suited for cartoon images


N d surflet representation vc mw db rb l.jpg
n scale (MGM); consider scale -D Surflet Representation [VC,MW,DB,RB]

  • Generalize wedgeletconcept to

    • arbitrary dimension n

    • arbitrary (n-1)-D manifold smoothness k

    • requires >linear elements


Joint texture geometry modeling l.jpg
Joint Texture/Geometry Modeling scale (MGM); consider scale

  • Dictionary D = {wavelets} U {wedgelets}

  • Representation tradeoff: texture vs.geometry

  • Test case: approximation / compression


ad