Loading in 2 Seconds...

Generalized Belief Propagation for Gaussian Graphical Model in probabilistic image processing

Loading in 2 Seconds...

- 94 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' Generalized Belief Propagation for Gaussian Graphical Model in probabilistic image processing' - stasia

**An Image/Link below is provided (as is) to download presentation**
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

Presentation Transcript

### Generalized Belief Propagation for Gaussian Graphical Model in probabilistic image processing

### Contents

### Bayesian Image Analysis

### Belief Propagation

### Contents

### Contents

### Contents

### Bayesian Image Analysis

### Bayesian Image Analysis

### Hyperparameter Determination by Maximization of Marginal Likelihood

### Maximization of Marginal Likelihood by EM (Expectation Maximization) Algorithm

### Maximization of Marginal Likelihood by EM (Expectation Maximization) Algorithm

### Contents

Kazuyuki Tanaka

Graduate School of Information Sciences,

Tohoku University, Japan

http://www.smapip.is.tohoku.ac.jp/~kazu/

Reference

K. Tanaka, H. Shouno, M. Okada and D. M. Titterington:

Accuracy of the Bethe Approximation for Hyperparameter Estimation in Probabilistic Image Processing, J. Phys. A: Math & Gen., 37, 8675 (2004).

SPDSA2005 (Roma)

Introduction

Loopy Belief Propagation

Generalized Belief Propagation

Probabilistic Image Processing

Concluding Remarks

SPDSA2005 (Roma)

Transmission

Original Image

Degraded Image

Graphical Model with Loops

=Spin System on Square Lattice

Bayesian Image Analysis + Belief Propagation

→ Probabilistic Image Processing

SPDSA2005 (Roma)

Probabilistic model with no loop

Belief Propagation

= Transfer Matrix

(Lauritzen, Pearl)

Probabilistic model with some loops

Approximation→Loopy Belief Propagation

(Yedidia, Freeman, Weiss)

Generalized Belief Propagation

Loopy Belief Propagation (LBP)= Bethe Approximation

Generalized Belief Propagation (GBP)

= Cluster Variation Method

- How is the accuracy of LBP and GBP?

SPDSA2005 (Roma)

Gaussian Graphical Model

SPDSA2005 (Roma)

Probabilistic Image Processing by Gaussian Graphical Model and Generalized Belief Propagation

- How can we construct a probabilistic image processing algorithm by using Loopy Belief Propagation and Generalized Belief Propagation?
- How is the accuracy of Loopy Belief Propagation and Generalized Belief Propagation?
- In order to clarify both questions, we assume the Gaussian graphical model as a posterior probabilistic model

SPDSA2005 (Roma)

Introduction

Loopy Belief Propagation

Generalized Belief Propagation

Probabilistic Image Processing

Concluding Remarks

SPDSA2005 (Roma)

Loopy Belief Propagation and TAP Free Energy

Loopy Belief Propagation

TAP Free

Energy

Mean Field Free Energy

SPDSA2005 (Roma)

Introduction

Loopy Belief Propagation

Generalized Belief Propagation

Probabilistic Image Processing

Concluding Remarks

SPDSA2005 (Roma)

2

1

2

1

2

3

4

3

4

3

4

Generalized Belief PropagationCluster: Set of nodes

Every subcluster of the element of B does not belong to B.

Example: System consisting of 4 nodes

SPDSA2005 (Roma)

2

3

4

5

6

7

8

9

5

2

1

4

5

3

6

2

2

1

2

3

4

1

3

6

5

2

6

5

8

9

8

7

4

5

5

4

7

8

9

6

5

8

7

4

8

6

5

9

Selection of B in LBP and GBPLBP

(Bethe Approx.）

GBP

(Square Approx.

in CVM)

SPDSA2005 (Roma)

Selection of B and C in Loopy Belief Propagation

LBP

(Bethe Approx.）

The set of Basic Clusters

The Set of Basic Clusters and Their Subclusters

SPDSA2005 (Roma)

Selection of B and C in Generalized Belief Propagation

GBP

(Square Approximation in CVM）

The set of Basic Clusters

The Set of Basic Clusters and Their Subclusters

SPDSA2005 (Roma)

Generalized Belief Propagation

Trial Function

Marginal Distribution of GGM is also GGM

SPDSA2005 (Roma)

Introduction

Loopy Belief Propagation

Generalized Belief Propagation

Probabilistic Image Processing

Concluding Remarks

SPDSA2005 (Roma)

Additive White Gaussian Noise

Transmission

Original Image

Degraded Image

SPDSA2005 (Roma)

Degraded Image g

A Posteriori Probability

Gaussian Graphical Model

SPDSA2005 (Roma)

Bayesian Image Analysis

Degraded Image

A Priori Probability

Degraded Image

Original Image

Pixels

A Posteriori

Probability

SPDSA2005 (Roma)

Marginalization

Degraded Image

Original Image

Marginal Likelihood

SPDSA2005 (Roma)

Q-Function

Incomplete Data

Equivalent

SPDSA2005 (Roma)

Iterate the following EM-steps until convergence:

Marginal Likelihood

Q-Function

A. P. Dempster, N. M. Laird and D. B. Rubin, “Maximum likelihood from incomplete data

via the EM algorithm,” J. Roy. Stat. Soc. B, 39 (1977).

SPDSA2005 (Roma)

Image Restoration

- The original image is generated from the prior probability. (Hyperparameters: Maximization of Marginal Likelihood)

Degraded Image

Original Image

Loopy Belief Propagation

Mean-Field Method

Exact Result

SPDSA2005 (Roma)

Numerical Experiments of Logarithm of Marginal Likelihood

- The original image is generated from the prior probability. (Hyperparameters: Maximization of Marginal Likelihood)

Degraded Image

Original Image

MFA

-5.0

-5.0

MFA

LPB

-5.5

Exact

LPB

Exact

-5.5

-6.0

10

20

30

40

50

60

0

0.0010

0.0020

Mean-Field Method

Loopy Belief Propagation

Exact Result

SPDSA2005 (Roma)

Exact

LBP

MF

0.001

LPB

Exact

MFA

0

100

0

50

Numerical Experiments of Logarithm of Marginal LikelihoodEM Algorithm with Belief Propagation

Original Image

Degraded Image

SPDSA2005 (Roma)

Image Restoration by Gaussian Graphical Model

EM Algorithm with Belief Propagation

Original Image

Degraded Image

MSE: 1512

MSE: 1529

SPDSA2005 (Roma)

Image Restoration by Gaussian Graphical Model

Original Image

Degraded Image

Mean Field Method

MSE:611

MSE: 1512

LBP

TAP

GBP

Exact Solution

MSE:327

MSE:320

MSE: 315

MSE:315

SPDSA2005 (Roma)

Image Restoration by Gaussian Graphical Model

Original Image

Degraded Image

Mean Field Method

MSE: 1529

MSE: 565

LBP

TAP

GBP

Exact Solution

MSE:260

MSE:248

MSE:236

MSE:236

SPDSA2005 (Roma)

Image Restoration by Gaussian Graphical Model

SPDSA2005 (Roma)

Image Restoration by Gaussian Graphical Model and Conventional Filters

GBP

(3x3) Lowpass

(5x5) Median

(5x5) Wiener

SPDSA2005 (Roma)

Image Restoration by Gaussian Graphical Model and Conventional Filters

GBP

(5x5) Lowpass

(5x5) Median

(5x5) Wiener

SPDSA2005 (Roma)

Introduction

Loopy Belief Propagation

Generalized Belief Propagation

Probabilistic Image Processing

Concluding Remarks

SPDSA2005 (Roma)

Summary

- Generalized Belief Propagation for Gaussian Graphical Model
- Accuracy of Generalized Belief Propagation
- Derivation of TAP Free Energy for Gaussian Graphical Model by Perturbation Expansion of Bethe Approximation

SPDSA2005 (Roma)

Future Problem

- Hyperparameter Estimation by TAP Free Energy is better than by Loopy Belief Propagation.

- Effectiveness of Higher Order Terms of TAP Free Energy for Hyperparameter Estimation by means of Marginal Likelihood in Bayesian Image Analysis.

Mean Field Free Energy

TAP Free

Energy

SPDSA2005 (Roma)

Download Presentation

Connecting to Server..