Higher order clique reduction in binary graph cut
Download
1 / 34

Higher-Order Clique Reduction in Binary Graph Cut - PowerPoint PPT Presentation


  • 216 Views
  • Uploaded on

Higher-Order Clique Reduction in Binary Graph Cut. Hiroshi Ishikawa. Nagoya City University Department of Information and Biological Sciences. Contribution of this work. Reduce any higher-order binary MRF into first order Adds variables

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Higher-Order Clique Reduction in Binary Graph Cut' - koen


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Higher order clique reduction in binary graph cut l.jpg

Higher-Order Clique Reduction in Binary Graph Cut

Hiroshi Ishikawa

Nagoya City University

Department of Information and Biological Sciences

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Contribution of this work l.jpg
Contribution of this work

Reduce any higher-order binary MRF

into first order

Adds variables

Can also be used for multi-label energy, with the Fusion Move technique

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Energy minimization l.jpg

FindX

Given Y

Close toY

Smooth

All pixels

Neighboring

pixels

AssignsXv (= 0 or 1) to each pixel v

Energy Minimization

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Energy minimization4 l.jpg

Better (Lower Energy)

Worse (Higher Energy)

A

B

C

D

Energy Minimization

Good (Low Energy)

Bad (High Energy)

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Energy minimization5 l.jpg
Energy Minimization

Good (Low Energy)

Bad (High Energy)

12 Bad

12 Bad

40 Good

40 Good

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Energy minimization6 l.jpg

Better (Lower Energy)

Worse (Higher Energy)

Energy Minimization

A

B

C

D

10 As

10 As

4 Bs

8 Bs

7 Cs

3 Cs

0 Ds

0 Ds

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Higher order energy l.jpg

Clique

Clique

Higher-Order Energy

Third Order (Clique up to 4 pixels)

First Order (Clique up to 2 pixels)

Third Order (Clique up to 4 pixels)

Clique

Clique

General Order

C: a set of cliques

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


First order mrf minimization l.jpg
First-Order MRF Minimization

Graph cuts

Greig et al. ’89

Boykov et al. CVPR’98, PAMI2001(-exp.)

Kolmogorov & Zabih. PAMI2004

Belief propagation

Felzenszwalb & Huttenlocher. IJCV2006

Meltzer et al. ICCV2005

Tree-reweighted message passing

Kolmogorov. PAMI2006

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Higher order mrf minimization l.jpg
Higher-Order MRF Minimization

Graph cuts

Kolmogorov & Zabih. PAMI2004

Freedman & Drineas. CVPR2005

Woodford et al. CVPR2008

Kohli et al. PAMI’08, Cremers&Grady ECCV’06

Rother et al. CVPR2009

Komodakis & Paragios. CVPR2009

Belief propagation

Lan et al. ECCV2006

Potetz. CVPR2008

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Higher order mrf minimization10 l.jpg
Higher-Order MRF Minimization

Graph cuts

Kolmogorov & Zabih. PAMI2004

Freedman & Drineas. CVPR2005

Woodford et al. CVPR2008

Kohli et al. PAMI’08, Cremers&Grady ECCV’06

Rother et al. CVPR2009

Komodakis & Paragios. CVPR2009

Belief propagation

Lan et al. ECCV2006

Potetz. CVPR2008

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Functions of binary variables l.jpg
Functions of Binary Variables

Pseudo-Boolean function (PBF)

Function of binary (0 or 1) variables

Can always write uniquely as a polynomial

One variable x : E0 (1x) + E1 x

Two variables x, y :

E00(1x) (1y) + E01(1x) y + E10x (1y) + E11x y

Three variables x, y, z :

E000(1x) (1y) (1z) + E001(1x) (1y) z +…+ E111x y z

nth order binary MRF = (n+1)th degree PBF

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


2 nd order cubic case l.jpg
2nd-Order (Cubic) Case

Kolmogorov & Zabih. PAMI2004

Freedman & Drineas. CVPR2005

Reduce cubic PBF into quadratic one using

B={0,1}

xy z

0 0 0

0 0 1

0 1 1

1 1 1

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


2 nd order cubic case13 l.jpg
2nd-Order (Cubic) Case

If a < 0

Thus

So, in a minimization problem, we can substitute

by

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Higher order case l.jpg
Higher-Order Case

ifa < 0

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Higher order case15 l.jpg
Higher-Order Case

For a > 0 and d > 3, nothing similar is known

→ our contribution

Imagine such a formula:

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Higher order case16 l.jpg
Higher-Order Case

For a > 0 and d > 3, nothing similar is known

→ our contribution

Imagine such a formula:

Notice LHS is symmetric

i.e., if we swap the value of two variables,

LHS is unchanged

So RHS must be symmetric, too.

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Symmetric polynomial l.jpg
Symmetric Polynomial

Fact

Any symmetric polynomial can be written

as a polynomial in terms of elementary

symmetric polynomials.

If f(x, y,z,t) is quadratic symmetric, it can be

written with a polynomial P(u,v) :

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Quartic degree 4 case l.jpg
Quartic (Degree 4) Case

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Quartic degree 4 case19 l.jpg
Quartic (Degree 4) Case

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Quartic degree 4 case20 l.jpg
Quartic (Degree 4) Case

An exhaustive search fora, b, c, d, e yields

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Quintic degree 5 case l.jpg
Quintic (Degree 5) Case

Similarly,

and so on, until one can guess…

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


General case l.jpg
General Case

where

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


General case23 l.jpg
General Case

For each monomial, the number of new

variable is:

For instance, general quintic looks like:

So the number is exponential in degree

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Multiple labels fusion move l.jpg
Multiple labels: Fusion Move

Assume labels

Labeling Y assigns a label Yv to each v

Fusion Move

Iteratively update Y :

1. Generate a proposed labeling P

Lempitsky et al. ICCV2007

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Multiple labels fusion move25 l.jpg
Multiple labels: Fusion Move

Assume labels

Labeling Y assigns a label Yv to each v

Fusion Move

Iteratively update Y :

1. Generate a proposed labeling P

2. MergeY and P

The merge defines a binary problem:

“For each v, changeYv to Pvor not”

Lempitsky et al. ICCV2007

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Multiple labels fusion move26 l.jpg

0

1

0

0

1

0

1

0

0

1

0

1

1

1

0

0

Multiple labels: Fusion Move

Fusion Move

Iteratively update Y :

1. Generate a proposed labeling P

2. MergeY and P

The merge defines a binary problem:

“For each v, changeYv to Pvor not”

Y

P

X

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Multiple labels fusion move27 l.jpg

0

0

0

1

0

0

0

1

1

0

0

0

0

1

0

0

1

0

0

1

0

0

1

1

1

1

1

1

1

0

1

0

Multiple labels: Fusion Move

Fusion Move

Iteratively update Y :

1. Generate a proposed labeling P

2. MergeY and P

The merge defines a binary problem:

“For each v, changeYv to Pvor not”

Y

P

X

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Fusion move with qpbo l.jpg
Fusion Move with QPBO

QPBO (Roof duality)

Minimizes submodular E globally.

For non-submodular E, assigns each pixel

0, 1, or unlabeled

With fusion move, by not changing

unlabeled pixels to P, E doesn’t increase

Hammer et al. 1984, Boros et al. 1991, 2006

Kolmogorov & Rother PAMI2007, Rother et al. CVPR2007

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Experiment denoising by foe l.jpg
Experiment: Denoising by FoE

FoE (Fields of Experts) Roth & Black CVPR2005

A higher-order prior for natural images

C: a set of cliques

C :

C :

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Experiment denoising by foe30 l.jpg
Experiment: Denoising by FoE

Original

Noise-added

3rd order

1st order

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Experiment denoising by foe31 l.jpg
Experiment: Denoising by FoE

Energy (smaller the better)

PSNR (larger the better)

32

45000

40000

31

35000

30

30000

29

25000

28

20000

27

15000

26

10000

25

5000

24

0

Lan et al.

Potetz

This work

Lan et al.

Potetz

This work

  • Lan et al. ECCV2006 ~8 hours

  • Potetz. CVPR2008 ~30 mins

  • This work ~10 mins

 = 10

 = 20

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Experiment denoising by foe32 l.jpg

12

0

10

0

8

0

6

0

4

0

2

0

0

5

0

10

0

15

0

20

0

25

0

Experiment: Denoising by FoE

Energy & PSNR

Two proposal generation strategies

E

(×1000)

PSNR

2

7

blur & random

2

6

expansion

2

5

expansion

2

4

blur & random

2

3

2

2

0

5

0

10

0

15

0

20

0

25

0

time (sec.)

time (sec.)

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Summary l.jpg
Summary

Reduce any higher-order binary MRF

into first order

Adds variables

Number exponential in order

For multi-label, can be used with Fusion Move with QPBO

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.


Slide34 l.jpg

Thank you!

Code available at

http://www.nsc.nagoya-cu.ac.jp/~hi/

Acknowledgements

Stefan Roth,Brian Potetz, and

Vladimir Kolmogorov

CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.