Bayesian networks bucket elimination algorithm
Sponsored Links
This presentation is the property of its rightful owner.
1 / 57

Bayesian Networks Bucket Elimination Algorithm PowerPoint PPT Presentation


  • 185 Views
  • Uploaded on
  • Presentation posted in: General

Bayesian Networks Bucket Elimination Algorithm. 主講人:虞台文 大同大學資工所 智慧型多媒體研究室. Content. Basic Concept Belief Updating Most Probable Explanation (MPE) Maximum A Posteriori (MAP). Bayesian Networks Bucket Elimination Algorithm. Basic Concept 大同大學資工所 智慧型多媒體研究室. Satisfiability.

Download Presentation

Bayesian Networks Bucket Elimination Algorithm

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Bayesian NetworksBucket Elimination Algorithm

主講人:虞台文

大同大學資工所

智慧型多媒體研究室


Content

  • Basic Concept

  • Belief Updating

  • Most Probable Explanation (MPE)

  • Maximum A Posteriori (MAP)


Bayesian NetworksBucket Elimination Algorithm

Basic Concept

大同大學資工所

智慧型多媒體研究室


Satisfiability

Given a statement of clauses (in disjunction normal form), the satisfiability problem is to determine whether there exists a truth assignment to make the statement true.

Examples:

Satisfiable

A=True, B=True, C=False, D=False

Satisfiable?


Resolution

can be true if and only if

can be true.

unsatisfiable


BucketA

BucketB

BucketC

BucketD

Direct Resolution

Example:

Given a set of clauses

and an order d=ABCD

Set initial buckets as follows:


BucketA

BucketB

BucketC

BucketD

Direct Resolution

Because no empty clause () is resulted, the statement is satisfiable.

How to get a truth assignment?


BucketA

BucketB

BucketC

BucketD

Direct Resolution


Direct Resolution


Queries on Bayesian Networks

  • Belief updating

  • Finding the most probable explanation (mpe)

    • Given evidence, finding a maximum probability assignment to the rest of variables.

  • Maximizing a posteriori hypothesis (map)

    • Given evidence, finding an assignment to a subset of hypothesis variables that maximize their probability.

  • Maximizing the expected utility of the problem (meu)

    • Given evidence and utility function, finding a subset of decision variables that maximize the expected utility.


Bucket Elimination

  • The algorithm will be used as a framework for various probabilistic inferences on Bayesian Networks.


Preliminary – Elimination Functions

Given a function h defined over subset of variables S, where X S,

Eliminate parameterX fromh

Defined overU = S– {X}.


Preliminary – Elimination Functions

Given a function h defined over subset of variables S, where X S,


Preliminary – Elimination Functions

Given function h1,…, hn defined over subset of variables S1,…, Sn, respectively,

Defined over


Preliminary – Elimination Functions

Given function h1,…, hn defined over subset of variables S1,…, Sn, respectively,


Bayesian NetworksBucket Elimination Algorithm

Belief Updating

大同大學資工所

智慧型多媒體研究室


Goal

Normalization

Factor


A

C

B

F

D

G

Basic Concept of Variable Elimination

Example:


Basic Concept of Variable Elimination

Example:


Basic Concept of Variable Elimination

G(f)

D(a, b)

F(b, c)

B(a, c)

C(a)


Basic Concept of Variable Elimination

BucketG

BucketD

BucketF

BucketB

BucketC

BucketA


Basic Concept of Variable Elimination

BucketG

BucketD

BucketF

BucketB

BucketC

BucketA


Basic Concept of Variable Elimination


Basic Concept of Variable Elimination


0.7

0.1

0.7

0.1

0.7

0.1

0.7

0.1

Basic Concept of Variable Elimination


Basic Concept of Variable Elimination


Basic Concept of Variable Elimination


Basic Concept of Variable Elimination


Bucket Elimination Algorithm


Complexity

  • The BuckElim Algorithm can be applied to any ordering.

  • The arity of the function recorded in a bucket

    • the numbers of variables appearing in the processed bucked, excluding the bucket’s variable.

  • Time and Space complexity is exponentially grow with a function of arity r.

  • The arity is dependent on the ordering.

  • How many possible orderings for BN’s variables?


A

C

B

F

D

G

Consider the ordering AFDCBG.

Determination of the Arity

BucketG

BucketB

1

G

4

BucketC

B

1

,3

C

BucketD

0

,2

D

BucketF

,1

0

F

BucketA

0

A


A

C

B

1

1

F

G

D

4

4

B

G

3

1

C

2

0

D

1

0

F

0

0

A

d

Given the ordering, e.g., AFDCBG.

Determination of the Arity

The width of a graph is the maximum width of its nodes.

w(d) = 4

w*(d) = 4

w(d): width of initial graph

for ordering d.

w*(d): width of induced graph

for ordering d.

Width of node

Width of node

G

B

C

Induced

Graph

D

Initial

Graph

F

A


Definition of Tree-Width

Goal: Finding an ordering with smallest induced width.

Greedy heuristic and Approximation methods

Are available.

NP-Hard


Summary

  • The complexity of BuckElim algorithm is dominated by the time and space needed to process a bucket.

  • It is time and space is exponential in number of bucket variables.

  • Induced width bounds the arity of bucket functions.


A

C

B

F

D

G

Exercises

  • Use BuckElim to evaluate P(a|b=1) with the following two ordering:

  • d1=ACBFDG

  • d2=AFDCBG

Give the details and make some conclusion.

How to improve the algorithm?


Bayesian NetworksBucket Elimination Algorithm

Most Probable Explanation (MPE)

大同大學資工所

智慧型多媒體研究室


MPE

Goal:

evidence


MPE

Goal:


xi

Notations


MPE

Let


Xn

MPE

Some terms involve xn,

some terms not.

Xn is conditioned by its parents.

Xnconditions its children.


Xn

MPE

xnappears in these CPT’s

Not conditioned by xn

Conditioned by xn

Itself


MPE

Process the next bucket recursively.

Eliminate variable xnatBucketn.


A

C

B

F

D

G

Example


A

C

B

F

D

G

Example

Consider ordering ACBFDG

BucketG

BucketD

BucketF

BucketB

BucketC

BucketA


Bucket Elimination Algorithm


Exercise

Consider ordering ACBFDG


Bayesian NetworksBucket Elimination Algorithm

Maximum

A Posteriori (MAP)

大同大學資工所

智慧型多媒體研究室


MAP

Given a belief network, a subset of hypothesized variablesA=(A1, …, Ak), and evidence E=e, the goal is to determine


A

C

B

F

D

G

Example

Hypothesis (Decision)

Variables

g = 1


MAP

Ordering

Some of them may be observed


MAP


MAP


MAP

Bucket Elimination for MPE

Bucket Elimination for belief updating


Bucket Elimination Algorithm


A

C

B

F

g = 1

D

G

Consider orderingCBAFDG

Example

BucketG

BucketD

BucketF

BucketA

BucketB

BucketC


A

C

B

F

g = 1

D

G

Consider orderingCBAFDG

Exercise

BucketG

BucketD

BucketF

BucketA

Give the detail

BucketB

BucketC


  • Login