generalized belief propagation for gaussian graphical model in probabilistic image processing
Download
Skip this Video
Download Presentation
Generalized Belief Propagation for Gaussian Graphical Model in probabilistic image processing

Loading in 2 Seconds...

play fullscreen
1 / 42

Generalized Belief Propagation for Gaussian Graphical Model in probabilistic image processing - PowerPoint PPT Presentation


  • 94 Views
  • Uploaded on

Generalized Belief Propagation for Gaussian Graphical Model in probabilistic image processing. Kazuyuki Tanaka Graduate School of Information Sciences, Tohoku University, Japan http://www.smapip.is.tohoku.ac.jp/~kazu/. Reference K. Tanaka, H. Shouno, M. Okada and D. M. Titterington:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Generalized Belief Propagation for Gaussian Graphical Model in probabilistic image processing' - stasia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
generalized belief propagation for gaussian graphical model in probabilistic image processing

Generalized Belief Propagation for Gaussian Graphical Model in probabilistic image processing

Kazuyuki Tanaka

Graduate School of Information Sciences,

Tohoku University, Japan

http://www.smapip.is.tohoku.ac.jp/~kazu/

Reference

K. Tanaka, H. Shouno, M. Okada and D. M. Titterington:

Accuracy of the Bethe Approximation for Hyperparameter Estimation in Probabilistic Image Processing, J. Phys. A: Math & Gen., 37, 8675 (2004).

SPDSA2005 (Roma)

contents

Contents

Introduction

Loopy Belief Propagation

Generalized Belief Propagation

Probabilistic Image Processing

Concluding Remarks

SPDSA2005 (Roma)

bayesian image analysis

Noise

Transmission

Original Image

Degraded Image

Bayesian Image Analysis

Graphical Model with Loops

=Spin System on Square Lattice

Bayesian Image Analysis + Belief Propagation

→ Probabilistic Image Processing

SPDSA2005 (Roma)

belief propagation

Probabilistic model with no loop

Belief Propagation

Belief Propagation

= Transfer Matrix

(Lauritzen, Pearl)

Probabilistic model with some loops

Approximation→Loopy Belief Propagation

(Yedidia, Freeman, Weiss)

Generalized Belief Propagation

Loopy Belief Propagation (LBP)= Bethe Approximation

Generalized Belief Propagation (GBP)

= Cluster Variation Method

  • How is the accuracy of LBP and GBP?

SPDSA2005 (Roma)

gaussian graphical model
Gaussian Graphical Model

SPDSA2005 (Roma)

probabilistic image processing by gaussian graphical model and generalized belief propagation
Probabilistic Image Processing by Gaussian Graphical Model and Generalized Belief Propagation
  • How can we construct a probabilistic image processing algorithm by using Loopy Belief Propagation and Generalized Belief Propagation?
  • How is the accuracy of Loopy Belief Propagation and Generalized Belief Propagation?
  • In order to clarify both questions, we assume the Gaussian graphical model as a posterior probabilistic model

SPDSA2005 (Roma)

contents1

Contents

Introduction

Loopy Belief Propagation

Generalized Belief Propagation

Probabilistic Image Processing

Concluding Remarks

SPDSA2005 (Roma)

loopy belief propagation
Loopy Belief Propagation

Trial Function

Tractable Form

SPDSA2005 (Roma)

loopy belief propagation1
Loopy Belief Propagation

Trial Function

Marginal Distribution of GGM is also GGM

SPDSA2005 (Roma)

loopy belief propagation2
Loopy Belief Propagation

Bethe Free Energy in GGM

SPDSA2005 (Roma)

loopy belief propagation3
Loopy Belief Propagation

m is exact

SPDSA2005 (Roma)

loopy belief propagation and tap free energy
Loopy Belief Propagation and TAP Free Energy

Loopy Belief Propagation

TAP Free

Energy

Mean Field Free Energy

SPDSA2005 (Roma)

contents2

Contents

Introduction

Loopy Belief Propagation

Generalized Belief Propagation

Probabilistic Image Processing

Concluding Remarks

SPDSA2005 (Roma)

generalized belief propagation

1

2

1

2

1

2

3

4

3

4

3

4

Generalized Belief Propagation

Cluster: Set of nodes

Every subcluster of the element of B does not belong to B.

Example: System consisting of 4 nodes

SPDSA2005 (Roma)

selection of b in lbp and gbp

1

2

3

4

5

6

7

8

9

5

2

1

4

5

3

6

2

2

1

2

3

4

1

3

6

5

2

6

5

8

9

8

7

4

5

5

4

7

8

9

6

5

8

7

4

8

6

5

9

Selection of B in LBP and GBP

LBP

(Bethe Approx.)

GBP

(Square Approx.

in CVM)

SPDSA2005 (Roma)

selection of b and c in loopy belief propagation
Selection of B and C in Loopy Belief Propagation

LBP

(Bethe Approx.)

The set of Basic Clusters

The Set of Basic Clusters and Their Subclusters

SPDSA2005 (Roma)

selection of b and c in generalized belief propagation
Selection of B and C in Generalized Belief Propagation

GBP

(Square Approximation in CVM)

The set of Basic Clusters

The Set of Basic Clusters and Their Subclusters

SPDSA2005 (Roma)

generalized belief propagation1
Generalized Belief Propagation

Trial Function

Marginal Distribution of GGM is also GGM

SPDSA2005 (Roma)

generalized belief propagation2
Generalized Belief Propagation

m is exact

SPDSA2005 (Roma)

contents3

Contents

Introduction

Loopy Belief Propagation

Generalized Belief Propagation

Probabilistic Image Processing

Concluding Remarks

SPDSA2005 (Roma)

bayesian image analysis1

Noise

Transmission

Bayesian Image Analysis

Original Image

Degraded Image

SPDSA2005 (Roma)

bayesian image analysis2

Degradation Process

Bayesian Image Analysis

Additive White Gaussian Noise

Transmission

Original Image

Degraded Image

SPDSA2005 (Roma)

bayesian image analysis3

Standard Images

A Priori Probability

Bayesian Image Analysis

Generate

Similar?

SPDSA2005 (Roma)

bayesian image analysis4

Original Image f

Degraded Image g

Bayesian Image Analysis

A Posteriori Probability

Gaussian Graphical Model

SPDSA2005 (Roma)

bayesian image analysis5
Bayesian Image Analysis

Degraded Image

A Priori Probability

Degraded Image

Original Image

Pixels

A Posteriori

Probability

SPDSA2005 (Roma)

hyperparameter determination by maximization of marginal likelihood

Hyperparameter Determination by Maximization of Marginal Likelihood

Marginalization

Degraded Image

Original Image

Marginal Likelihood

SPDSA2005 (Roma)

maximization of marginal likelihood by em expectation maximization algorithm

Marginal Likelihood

Maximization of Marginal Likelihood by EM (Expectation Maximization) Algorithm

Q-Function

Incomplete Data

Equivalent

SPDSA2005 (Roma)

maximization of marginal likelihood by em expectation maximization algorithm1

EM Algorithm

Iterate the following EM-steps until convergence:

Marginal Likelihood

Maximization of Marginal Likelihood by EM (Expectation Maximization) Algorithm

Q-Function

A. P. Dempster, N. M. Laird and D. B. Rubin, “Maximum likelihood from incomplete data

via the EM algorithm,” J. Roy. Stat. Soc. B, 39 (1977).

SPDSA2005 (Roma)

image restoration
Image Restoration
  • The original image is generated from the prior probability. (Hyperparameters: Maximization of Marginal Likelihood)

Degraded Image

Original Image

Loopy Belief Propagation

Mean-Field Method

Exact Result

SPDSA2005 (Roma)

numerical experiments of logarithm of marginal likelihood
Numerical Experiments of Logarithm of Marginal Likelihood
  • The original image is generated from the prior probability. (Hyperparameters: Maximization of Marginal Likelihood)

Degraded Image

Original Image

MFA

-5.0

-5.0

MFA

LPB

-5.5

Exact

LPB

Exact

-5.5

-6.0

10

20

30

40

50

60

0

0.0010

0.0020

Mean-Field Method

Loopy Belief Propagation

Exact Result

SPDSA2005 (Roma)

numerical experiments of logarithm of marginal likelihood1

0.002

Exact

LBP

MF

0.001

LPB

Exact

MFA

0

100

0

50

Numerical Experiments of Logarithm of Marginal Likelihood

EM Algorithm with Belief Propagation

Original Image

Degraded Image

SPDSA2005 (Roma)

image restoration by gaussian graphical model
Image Restoration by Gaussian Graphical Model

EM Algorithm with Belief Propagation

Original Image

Degraded Image

MSE: 1512

MSE: 1529

SPDSA2005 (Roma)

image restoration by gaussian graphical model1
Image Restoration by Gaussian Graphical Model

Original Image

Degraded Image

Mean Field Method

MSE:611

MSE: 1512

LBP

TAP

GBP

Exact Solution

MSE:327

MSE:320

MSE: 315

MSE:315

SPDSA2005 (Roma)

image restoration by gaussian graphical model2
Image Restoration by Gaussian Graphical Model

Original Image

Degraded Image

Mean Field Method

MSE: 1529

MSE: 565

LBP

TAP

GBP

Exact Solution

MSE:260

MSE:248

MSE:236

MSE:236

SPDSA2005 (Roma)

image restoration by gaussian graphical model and conventional filters
Image Restoration by Gaussian Graphical Model and Conventional Filters

GBP

(3x3) Lowpass

(5x5) Median

(5x5) Wiener

SPDSA2005 (Roma)

image restoration by gaussian graphical model and conventional filters1
Image Restoration by Gaussian Graphical Model and Conventional Filters

GBP

(5x5) Lowpass

(5x5) Median

(5x5) Wiener

SPDSA2005 (Roma)

contents4

Contents

Introduction

Loopy Belief Propagation

Generalized Belief Propagation

Probabilistic Image Processing

Concluding Remarks

SPDSA2005 (Roma)

summary
Summary
  • Generalized Belief Propagation for Gaussian Graphical Model
  • Accuracy of Generalized Belief Propagation
  • Derivation of TAP Free Energy for Gaussian Graphical Model by Perturbation Expansion of Bethe Approximation

SPDSA2005 (Roma)

future problem
Future Problem
  • Hyperparameter Estimation by TAP Free Energy is better than by Loopy Belief Propagation.
  • Effectiveness of Higher Order Terms of TAP Free Energy for Hyperparameter Estimation by means of Marginal Likelihood in Bayesian Image Analysis.

Mean Field Free Energy

TAP Free

Energy

SPDSA2005 (Roma)

ad