On the power of belief propagation a constraint propagation perspective
Download
1 / 23

On the Power of Belief Propagation: A Constraint Propagation Perspective - PowerPoint PPT Presentation


  • 114 Views
  • Uploaded on
  • Presentation posted in: General

On the Power of Belief Propagation: A Constraint Propagation Perspective. Rina Dechter. Bozhena Bidyuk. Robert Mateescu. Emma Rollon. Distributed Belief Propagation. 4. 3. 2. 1. 1. 2. 3. 4. Distributed Belief Propagation. 5. 5. 5. 5. 5. How many people?.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

On the Power of Belief Propagation: A Constraint Propagation Perspective

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


On the Power of Belief Propagation:A Constraint Propagation Perspective

Rina Dechter

BozhenaBidyuk

RobertMateescu

EmmaRollon


Distributed Belief Propagation


4

3

2

1

1

2

3

4

Distributed Belief Propagation

5

5

5

5

5

How many people?


Distributed Belief Propagation

Causal support

Diagnostic support


Belief Propagation in Polytrees


BP on Loopy Graphs

  • Pearl (1988): use of BP to loopy networks

  • McEliece, et. Al 1988: IBP’s success on coding networks

  • Lots of research into convergence … and accuracy (?), but:

    • Why IBP works well for coding networks

    • Can we characterize other good problem classes

    • Can we have any guarantees on accuracy (even if converges)


Arc-consistency

  • Sound

  • Incomplete

  • Always converges

    (polynomial)

A

B

1

1

2

2

<

3

3

A

B

<

=

D

C

D

C

<

1

1

2

2

3

3


1

1

A

A

A

A

2

3

A

A

2

3

AB

AC

AB

AC

B

A

B

C

B

A

B

C

4

5

4

5

ABD

BCF

ABD

BCF

6

F

D

6

DFG

F

D

DFG

Flattening the Bayesian Network

Flat constraint network

Belief network


1

A

A

A

2

3

AB

AC

A

B

C

B

5

4

ABD

BCF

6

F

D

DFG

Belief Zero Propagation = Arc-Consistency

Updated relation:

Updated belief:


1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

Flat Network - Example


1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

IBP Example – Iteration 1


1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

IBP Example – Iteration 2


1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

IBP Example – Iteration 3


1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

IBP Example – Iteration 4


1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

IBP Example – Iteration 5


IBP – Inference Power for Zero Beliefs

  • Theorem:

    Iterative BP performs arc-consistency on the flat network.

  • Soundness:

    • Inference of zero beliefs by IBP converges

    • All the inferred zero beliefs are correct

  • Incompleteness:

    • Iterative BP is as weak and as strong as arc-consistency

  • Continuity Hypothesis: IBP is sound for zero - IBP is accurate for extreme beliefs? Tested empirically


Algorithms:

IBP

IJGP

Experimental Results

We investigated empirically if the results for zero beliefs extend to ε-small beliefs (ε > 0)

  • Measures:

    • Exact/IJGP histogram

    • Recall absolute error

    • Precision absolute error

  • Network types:

    • Coding

    • Linkage analysis*

    • Grids*

    • Two-layer noisy-OR*

    • CPCS54, CPCS360

YES

Have determinism?

NO

* Instances from the UAI08 competition


Networks withDeterminism: Coding

N=200, 1000 instances, w*=15


Nets w/o Determinism: bn2o

w* = 24

w* = 27

w* = 26


NetswithDeterminism: Linkage

Exact Histogram IJGP Histogram Recall Abs. Error Precision Abs. Error

pedigree1, w* = 21

Absolute Error

Percentage

pedigree37, w* = 30

Absolute Error

Percentage

i-bound = 3

i-bound = 7


The Cutset Phenomena & irrelevant nodes

  • Observed variables break the flow of inference

    • IBP is exact when evidence variables form a cycle-cutset

  • Unobserved variables without observed descendents send zero-information to the parent variables – it is irrelevant

    • In a network without evidence, IBP converges in one iteration top-down

X

Y

X


Nodes with extreme support

Observed variables with xtreme priors or xtreme support can nearly-cut information flow:

A

B1

C1

EB

EC

B2

C2

D

EC

EB

ED

Average Error vs. Priors


Conclusion: For Networks with Determinism

  • IBP converges & sound for zero beliefs

  • IBP’s power to infer zeros is as weak or as strong as arc-consistency

  • However: inference of extreme beliefs can be wrong.

  • Cutset property (Bidyuk and Dechter, 2000):

    • Evidence and inferred singleton act like cutset

    • If zeros are cycle-cutset, all beliefs are exact

    • Extensions to epsilon-cutset were supported empirically.


ad
  • Login