On the Power of Belief Propagation: A Constraint Propagation Perspective

Download Presentation

On the Power of Belief Propagation: A Constraint Propagation Perspective

Loading in 2 Seconds...

- 102 Views
- Uploaded on
- Presentation posted in: General

On the Power of Belief Propagation: A Constraint Propagation Perspective

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

On the Power of Belief Propagation:A Constraint Propagation Perspective

Rina Dechter

BozhenaBidyuk

RobertMateescu

EmmaRollon

4

3

2

1

1

2

3

4

5

5

5

5

5

How many people?

Causal support

Diagnostic support

- Pearl (1988): use of BP to loopy networks
- McEliece, et. Al 1988: IBP’s success on coding networks
- Lots of research into convergence … and accuracy (?), but:
- Why IBP works well for coding networks
- Can we characterize other good problem classes
- Can we have any guarantees on accuracy (even if converges)

- Sound
- Incomplete
- Always converges
(polynomial)

A

B

1

1

2

2

<

3

3

A

B

<

=

D

C

D

C

<

1

1

2

2

3

3

1

1

A

A

A

A

2

3

A

A

2

3

AB

AC

AB

AC

B

A

B

C

B

A

B

C

4

5

4

5

ABD

BCF

ABD

BCF

6

F

D

6

DFG

F

D

DFG

Flat constraint network

Belief network

1

A

A

A

2

3

AB

AC

A

B

C

B

5

4

ABD

BCF

6

F

D

DFG

Updated relation:

Updated belief:

1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

- Theorem:
Iterative BP performs arc-consistency on the flat network.

- Soundness:
- Inference of zero beliefs by IBP converges
- All the inferred zero beliefs are correct

- Incompleteness:
- Iterative BP is as weak and as strong as arc-consistency

- Continuity Hypothesis: IBP is sound for zero - IBP is accurate for extreme beliefs? Tested empirically

Algorithms:

IBP

IJGP

We investigated empirically if the results for zero beliefs extend to ε-small beliefs (ε > 0)

- Measures:
- Exact/IJGP histogram
- Recall absolute error
- Precision absolute error

- Network types:
- Coding
- Linkage analysis*
- Grids*
- Two-layer noisy-OR*
- CPCS54, CPCS360

YES

Have determinism?

NO

* Instances from the UAI08 competition

N=200, 1000 instances, w*=15

w* = 24

w* = 27

w* = 26

Exact Histogram IJGP Histogram Recall Abs. Error Precision Abs. Error

pedigree1, w* = 21

Absolute Error

Percentage

pedigree37, w* = 30

Absolute Error

Percentage

i-bound = 3

i-bound = 7

- Observed variables break the flow of inference
- IBP is exact when evidence variables form a cycle-cutset

- Unobserved variables without observed descendents send zero-information to the parent variables – it is irrelevant
- In a network without evidence, IBP converges in one iteration top-down

X

Y

X

Observed variables with xtreme priors or xtreme support can nearly-cut information flow:

A

B1

C1

EB

EC

B2

C2

D

EC

EB

ED

Average Error vs. Priors

- IBP converges & sound for zero beliefs
- IBP’s power to infer zeros is as weak or as strong as arc-consistency
- However: inference of extreme beliefs can be wrong.
- Cutset property (Bidyuk and Dechter, 2000):
- Evidence and inferred singleton act like cutset
- If zeros are cycle-cutset, all beliefs are exact
- Extensions to epsilon-cutset were supported empirically.