On the power of belief propagation a constraint propagation perspective
This presentation is the property of its rightful owner.
Sponsored Links
1 / 23

On the Power of Belief Propagation: A Constraint Propagation Perspective PowerPoint PPT Presentation


  • 87 Views
  • Uploaded on
  • Presentation posted in: General

On the Power of Belief Propagation: A Constraint Propagation Perspective. Rina Dechter. Bozhena Bidyuk. Robert Mateescu. Emma Rollon. Distributed Belief Propagation. 4. 3. 2. 1. 1. 2. 3. 4. Distributed Belief Propagation. 5. 5. 5. 5. 5. How many people?.

Download Presentation

On the Power of Belief Propagation: A Constraint Propagation Perspective

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


On the power of belief propagation a constraint propagation perspective

On the Power of Belief Propagation:A Constraint Propagation Perspective

Rina Dechter

BozhenaBidyuk

RobertMateescu

EmmaRollon


Distributed belief propagation

Distributed Belief Propagation


Distributed belief propagation1

4

3

2

1

1

2

3

4

Distributed Belief Propagation

5

5

5

5

5

How many people?


Distributed belief propagation2

Distributed Belief Propagation

Causal support

Diagnostic support


Belief propagation in polytrees

Belief Propagation in Polytrees


Bp on loopy graphs

BP on Loopy Graphs

  • Pearl (1988): use of BP to loopy networks

  • McEliece, et. Al 1988: IBP’s success on coding networks

  • Lots of research into convergence … and accuracy (?), but:

    • Why IBP works well for coding networks

    • Can we characterize other good problem classes

    • Can we have any guarantees on accuracy (even if converges)


Arc consistency

Arc-consistency

  • Sound

  • Incomplete

  • Always converges

    (polynomial)

A

B

1

1

2

2

<

3

3

A

B

<

=

D

C

D

C

<

1

1

2

2

3

3


Flattening the bayesian network

1

1

A

A

A

A

2

3

A

A

2

3

AB

AC

AB

AC

B

A

B

C

B

A

B

C

4

5

4

5

ABD

BCF

ABD

BCF

6

F

D

6

DFG

F

D

DFG

Flattening the Bayesian Network

Flat constraint network

Belief network


Belief zero propagation arc consistency

1

A

A

A

2

3

AB

AC

A

B

C

B

5

4

ABD

BCF

6

F

D

DFG

Belief Zero Propagation = Arc-Consistency

Updated relation:

Updated belief:


Flat network example

1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

Flat Network - Example


Ibp example iteration 1

1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

IBP Example – Iteration 1


Ibp example iteration 2

1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

IBP Example – Iteration 2


Ibp example iteration 3

1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

IBP Example – Iteration 3


Ibp example iteration 4

1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

IBP Example – Iteration 4


Ibp example iteration 5

1

A

A

3

A

2

AB

AC

B

A

C

B

5

4

ABD

BCF

6

F

D

DFG

IBP Example – Iteration 5


Ibp inference power for zero beliefs

IBP – Inference Power for Zero Beliefs

  • Theorem:

    Iterative BP performs arc-consistency on the flat network.

  • Soundness:

    • Inference of zero beliefs by IBP converges

    • All the inferred zero beliefs are correct

  • Incompleteness:

    • Iterative BP is as weak and as strong as arc-consistency

  • Continuity Hypothesis: IBP is sound for zero - IBP is accurate for extreme beliefs? Tested empirically


Experimental results

Algorithms:

IBP

IJGP

Experimental Results

We investigated empirically if the results for zero beliefs extend to ε-small beliefs (ε > 0)

  • Measures:

    • Exact/IJGP histogram

    • Recall absolute error

    • Precision absolute error

  • Network types:

    • Coding

    • Linkage analysis*

    • Grids*

    • Two-layer noisy-OR*

    • CPCS54, CPCS360

YES

Have determinism?

NO

* Instances from the UAI08 competition


Networks with determinism coding

Networks withDeterminism: Coding

N=200, 1000 instances, w*=15


Nets w o determinism bn2o

Nets w/o Determinism: bn2o

w* = 24

w* = 27

w* = 26


Nets with determinism linkage

NetswithDeterminism: Linkage

Exact Histogram IJGP Histogram Recall Abs. Error Precision Abs. Error

pedigree1, w* = 21

Absolute Error

Percentage

pedigree37, w* = 30

Absolute Error

Percentage

i-bound = 3

i-bound = 7


The cutset phenomena irrelevant nodes

The Cutset Phenomena & irrelevant nodes

  • Observed variables break the flow of inference

    • IBP is exact when evidence variables form a cycle-cutset

  • Unobserved variables without observed descendents send zero-information to the parent variables – it is irrelevant

    • In a network without evidence, IBP converges in one iteration top-down

X

Y

X


Nodes with extreme support

Nodes with extreme support

Observed variables with xtreme priors or xtreme support can nearly-cut information flow:

A

B1

C1

EB

EC

B2

C2

D

EC

EB

ED

Average Error vs. Priors


Conclusion for networks with determinism

Conclusion: For Networks with Determinism

  • IBP converges & sound for zero beliefs

  • IBP’s power to infer zeros is as weak or as strong as arc-consistency

  • However: inference of extreme beliefs can be wrong.

  • Cutset property (Bidyuk and Dechter, 2000):

    • Evidence and inferred singleton act like cutset

    • If zeros are cycle-cutset, all beliefs are exact

    • Extensions to epsilon-cutset were supported empirically.


  • Login