Introduction to belief propagation and its generalizations l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 15

Introduction to Belief Propagation and its Generalizations. PowerPoint PPT Presentation


  • 191 Views
  • Updated On :
  • Presentation posted in: General

Introduction to Belief Propagation and its Generalizations. Max Welling Donald Bren School of Information and Computer and Science University of California Irvine. Graphical Models. A ‘marriage’ between probability theory and graph theory. Why probabilities?

Related searches for Introduction to Belief Propagation and its Generalizations.

Download Presentation

Introduction to Belief Propagation and its Generalizations.

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Introduction to belief propagation and its generalizations l.jpg

Introduction to Belief Propagation and its Generalizations.

Max Welling

Donald Bren School

of Information and Computer and Science

University of California Irvine


Graphical models l.jpg

Graphical Models

A ‘marriage’ between probability theory and graph theory

  • Why probabilities?

  • Reasoning with uncertainties, confidence levels

  • Many processes are inherently ‘noisy’ robustness issues

  • Why graphs?

  • Provide necessary structure in large models:

  • - Designing new probabilistic models.

  • - Reading out (conditional) independencies.

  • Inference & optimization:

  • - Dynamical programming

  • - Belief Propagation


Types of graphical model l.jpg

Types of Graphical Model

i

Parents(i)

j

i

Undirected graph

(Markov random field)

Directed graph

(Bayesian network)

factor graphs

interactions

variables


Example 1 undirected graph l.jpg

?

air or water ?

?

high

information

regions

low

information

regions

neighborhood

information

Example 1: Undirected Graph


Undirected graphs cont ed l.jpg

Undirected Graphs (cont’ed)

Nodes encode hidden information

(patch-identity).

They receive local information from the image (brightness, color).

Information is propagated though the graph over its edges.

Edges encode ‘compatibility’ between nodes.


Example 2 directed graphs l.jpg

Example 2: Directed Graphs

computers

TOPICS

war

animals

Iraqi

the

Matlab


Inference in graphical models l.jpg

Inference in Graphical Models

  • Inference:

  • Answer queries about unobserved random variables, given values

  • of observed random variables.

  • More general: compute their joint posterior distribution:

  • Why do we need it?

  • Answer queries : -Given past purchases, in what genre books is a client interested?

  • -Given a noisy image, what was the original image?

  • Learning probabilistic models from examples

  • (expectation maximization, iterative scaling )

  • Optimization problems: min-cut, max-flow, Viterbi, …

learning

inference

Example: P( = sea | image) ?


Approximate inference l.jpg

Approximate Inference

Inference is computationally intractable for large graphs (with cycles).

  • Approximate methods:

  • Markov Chain Monte Carlo sampling.

  • Mean field and more structured variational techniques.

  • Belief Propagation algorithms.


Belief propagation on trees l.jpg

external evidence

message

Compatibilities (interactions)

belief (approximate marginal probability)

Belief Propagation on trees

k

k

Mki

i

k

k

k

j

i

k

k


Belief propagation on loopy graphs l.jpg

external evidence

message

Compatibilities (interactions)

belief (approximate marginal probability)

Belief Propagation on loopy graphs

k

k

Mki

i

k

k

k

j

i

k

k


Some facts about bp l.jpg

Some facts about BP

  • BP is exact on trees.

  • If BP converges it has reached a local minimum of an objective function

  • (the Bethe free energy Yedidia et.al ‘00 , Heskes ’02)often good approximation

  • If it converges, convergence is fast near the fixed point.

  • Many exciting applications:

  • - error correcting decoding (MacKay, Yedidia, McEliece, Frey)

  • - vision (Freeman, Weiss)

  • - bioinformatics (Weiss)

  • - constraint satisfaction problems (Dechter)

  • - game theory (Kearns)

  • - …


Bp related algorithms l.jpg

BP Related Algorithms

  • Convergent alternatives (Welling,Teh’02, Yuille’02, Heskes’03)

  • Expectation Propagation (Minka’01)

  • Convex alternatives (Wainwright’02, Wiegerinck,Heskes’02)

  • Linear Response Propagation (Welling,Teh’02)

  • Generalized Belief Propagation(Yedidia,Freeman,Weiss’01)

  • Survey Propagation (Braunstein,Mezard,Weigt,Zecchina’03)


Generalized belief propagation l.jpg

Generalized Belief Propagation

Idea: To guess the distribution of one of your neighbors, you ask your other neighbors to guess your distribution. Opinions get combined multiplicatively.

GBP

BP


Marginal consistency l.jpg

Marginal Consistency

Solve inference problem

separately on each “patch”,

then stitch them together

using “marginal consistency”.


Region graphs yedidia freeman weiss 02 l.jpg

Region Graphs (Yedidia, Freeman, Weiss ’02)

Stitching together solutions on local clusters by enforcing

“marginal consistency” on their intersections.

C=1

C=1

C=1

C=1

C=…

C=…

C=…

C=…

C=…

C=…

C=…

C=…

C=…

Region: collection of interactions & variables.


  • Login