Loading in 5 sec....

Generalized Belief Propagation for Gaussian Graphical Model in probabilistic image processingPowerPoint Presentation

Generalized Belief Propagation for Gaussian Graphical Model in probabilistic image processing

- 92 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' Generalized Belief Propagation for Gaussian Graphical Model in probabilistic image processing' - stasia

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### Generalized Belief Propagation for Gaussian Graphical Model in probabilistic image processing

### Contents

### Bayesian Image Analysis

### Belief Propagation

### Contents and Generalized Belief Propagation

### Iteration Procedure

### Contents and Generalized Belief Propagation

### Contents and Generalized Belief Propagation

### Bayesian Image Analysis

### Bayesian Image Analysis

### Bayesian Image Analysis

### Bayesian Image Analysis

### Hyperparameter Determination by Maximization of Marginal Likelihood

### Maximization of Marginal Likelihood by EM (Expectation Maximization) Algorithm

### Maximization of Marginal Likelihood by EM (Expectation Maximization) Algorithm

### Contents Conventional Filters

Kazuyuki Tanaka

Graduate School of Information Sciences,

Tohoku University, Japan

http://www.smapip.is.tohoku.ac.jp/~kazu/

Reference

K. Tanaka, H. Shouno, M. Okada and D. M. Titterington:

Accuracy of the Bethe Approximation for Hyperparameter Estimation in Probabilistic Image Processing, J. Phys. A: Math & Gen., 37, 8675 (2004).

SPDSA2005 (Roma)

Introduction

Loopy Belief Propagation

Generalized Belief Propagation

Probabilistic Image Processing

Concluding Remarks

SPDSA2005 (Roma)

Transmission

Original Image

Degraded Image

Graphical Model with Loops

=Spin System on Square Lattice

Bayesian Image Analysis + Belief Propagation

→ Probabilistic Image Processing

SPDSA2005 (Roma)

Probabilistic model with no loop

Belief Propagation

= Transfer Matrix

(Lauritzen, Pearl)

Probabilistic model with some loops

Approximation→Loopy Belief Propagation

(Yedidia, Freeman, Weiss)

Generalized Belief Propagation

Loopy Belief Propagation (LBP)= Bethe Approximation

Generalized Belief Propagation (GBP)

= Cluster Variation Method

- How is the accuracy of LBP and GBP?

SPDSA2005 (Roma)

Gaussian Graphical Model

SPDSA2005 (Roma)

Probabilistic Image Processing by Gaussian Graphical Model and Generalized Belief Propagation

- How can we construct a probabilistic image processing algorithm by using Loopy Belief Propagation and Generalized Belief Propagation?
- How is the accuracy of Loopy Belief Propagation and Generalized Belief Propagation?
- In order to clarify both questions, we assume the Gaussian graphical model as a posterior probabilistic model

SPDSA2005 (Roma)

Introduction

Loopy Belief Propagation

Generalized Belief Propagation

Probabilistic Image Processing

Concluding Remarks

SPDSA2005 (Roma)

Kullback-Leibler Divergence of Gaussian Graphical Model and Generalized Belief Propagation

Entropy Term

SPDSA2005 (Roma)

Loopy Belief Propagation and Generalized Belief Propagation

Trial Function

Tractable Form

SPDSA2005 (Roma)

Loopy Belief Propagation and Generalized Belief Propagation

Trial Function

Marginal Distribution of GGM is also GGM

SPDSA2005 (Roma)

Loopy Belief Propagation and Generalized Belief Propagation

Bethe Free Energy in GGM

SPDSA2005 (Roma)

Fixed Point Equation and Generalized Belief Propagation

Iteration

SPDSA2005 (Roma)

Loopy Belief Propagation and TAP Free Energy and Generalized Belief Propagation

Loopy Belief Propagation

TAP Free

Energy

Mean Field Free Energy

SPDSA2005 (Roma)

Introduction

Loopy Belief Propagation

Generalized Belief Propagation

Probabilistic Image Processing

Concluding Remarks

SPDSA2005 (Roma)

1 and Generalized Belief Propagation

2

1

2

1

2

3

4

3

4

3

4

Generalized Belief PropagationCluster: Set of nodes

Every subcluster of the element of B does not belong to B.

Example: System consisting of 4 nodes

SPDSA2005 (Roma)

1 and Generalized Belief Propagation

2

3

4

5

6

7

8

9

5

2

1

4

5

3

6

2

2

1

2

3

4

1

3

6

5

2

6

5

8

9

8

7

4

5

5

4

7

8

9

6

5

8

7

4

8

6

5

9

Selection of B in LBP and GBPLBP

(Bethe Approx.）

GBP

(Square Approx.

in CVM)

SPDSA2005 (Roma)

Selection of and Generalized Belief PropagationB and C in Loopy Belief Propagation

LBP

(Bethe Approx.）

The set of Basic Clusters

The Set of Basic Clusters and Their Subclusters

SPDSA2005 (Roma)

Selection of and Generalized Belief PropagationB and C in Generalized Belief Propagation

GBP

(Square Approximation in CVM）

The set of Basic Clusters

The Set of Basic Clusters and Their Subclusters

SPDSA2005 (Roma)

Generalized Belief Propagation and Generalized Belief Propagation

Trial Function

Marginal Distribution of GGM is also GGM

SPDSA2005 (Roma)

Introduction

Loopy Belief Propagation

Generalized Belief Propagation

Probabilistic Image Processing

Concluding Remarks

SPDSA2005 (Roma)

Noise and Generalized Belief Propagation

Transmission

Original Image

Degraded Image

SPDSA2005 (Roma)

Degradation Process and Generalized Belief Propagation

Additive White Gaussian Noise

Transmission

Original Image

Degraded Image

SPDSA2005 (Roma)

Standard Images and Generalized Belief Propagation

A Priori Probability

Generate

Similar?

SPDSA2005 (Roma)

Original Image and Generalized Belief Propagationf

Degraded Image g

A Posteriori Probability

Gaussian Graphical Model

SPDSA2005 (Roma)

Bayesian Image Analysis and Generalized Belief Propagation

Degraded Image

A Priori Probability

Degraded Image

Original Image

Pixels

A Posteriori

Probability

SPDSA2005 (Roma)

Marginalization

Degraded Image

Original Image

Marginal Likelihood

SPDSA2005 (Roma)

Marginal Likelihood Likelihood

Q-Function

Incomplete Data

Equivalent

SPDSA2005 (Roma)

EM Algorithm Likelihood

Iterate the following EM-steps until convergence:

Marginal Likelihood

Q-Function

A. P. Dempster, N. M. Laird and D. B. Rubin, “Maximum likelihood from incomplete data

via the EM algorithm,” J. Roy. Stat. Soc. B, 39 (1977).

SPDSA2005 (Roma)

Image Restoration Likelihood

- The original image is generated from the prior probability. (Hyperparameters: Maximization of Marginal Likelihood)

Degraded Image

Original Image

Loopy Belief Propagation

Mean-Field Method

Exact Result

SPDSA2005 (Roma)

Numerical Experiments Likelihoodof Logarithm of Marginal Likelihood

- The original image is generated from the prior probability. (Hyperparameters: Maximization of Marginal Likelihood)

Degraded Image

Original Image

MFA

-5.0

-5.0

MFA

LPB

-5.5

Exact

LPB

Exact

-5.5

-6.0

10

20

30

40

50

60

0

0.0010

0.0020

Mean-Field Method

Loopy Belief Propagation

Exact Result

SPDSA2005 (Roma)

0.002 Likelihood

Exact

LBP

MF

0.001

LPB

Exact

MFA

0

100

0

50

Numerical Experiments of Logarithm of Marginal LikelihoodEM Algorithm with Belief Propagation

Original Image

Degraded Image

SPDSA2005 (Roma)

Image Restoration by Gaussian Graphical Model Likelihood

EM Algorithm with Belief Propagation

Original Image

Degraded Image

MSE: 1512

MSE: 1529

SPDSA2005 (Roma)

Image Restoration by Gaussian Graphical Model Likelihood

Original Image

Degraded Image

Mean Field Method

MSE:611

MSE: 1512

LBP

TAP

GBP

Exact Solution

MSE:327

MSE:320

MSE: 315

MSE:315

SPDSA2005 (Roma)

Image Restoration by Gaussian Graphical Model Likelihood

Original Image

Degraded Image

Mean Field Method

MSE: 1529

MSE: 565

LBP

TAP

GBP

Exact Solution

MSE:260

MSE:248

MSE:236

MSE:236

SPDSA2005 (Roma)

Image Restoration by Gaussian Graphical LikelihoodModel

SPDSA2005 (Roma)

Image Restoration by Gaussian Graphical Model and Conventional Filters

GBP

(3x3) Lowpass

(5x5) Median

(5x5) Wiener

SPDSA2005 (Roma)

Image Restoration by Gaussian Graphical Model and Conventional Filters

GBP

(5x5) Lowpass

(5x5) Median

(5x5) Wiener

SPDSA2005 (Roma)

Introduction

Loopy Belief Propagation

Generalized Belief Propagation

Probabilistic Image Processing

Concluding Remarks

SPDSA2005 (Roma)

Summary Conventional Filters

- Generalized Belief Propagation for Gaussian Graphical Model
- Accuracy of Generalized Belief Propagation
- Derivation of TAP Free Energy for Gaussian Graphical Model by Perturbation Expansion of Bethe Approximation

SPDSA2005 (Roma)

Future Problem Conventional Filters

- Hyperparameter Estimation by TAP Free Energy is better than by Loopy Belief Propagation.

- Effectiveness of Higher Order Terms of TAP Free Energy for Hyperparameter Estimation by means of Marginal Likelihood in Bayesian Image Analysis.

Mean Field Free Energy

TAP Free

Energy

SPDSA2005 (Roma)

Download Presentation

Connecting to Server..