Bayesian networks bucket elimination algorithm
This presentation is the property of its rightful owner.
Sponsored Links
1 / 57

Bayesian Networks Bucket Elimination Algorithm PowerPoint PPT Presentation


  • 168 Views
  • Uploaded on
  • Presentation posted in: General

Bayesian Networks Bucket Elimination Algorithm. 主講人:虞台文 大同大學資工所 智慧型多媒體研究室. Content. Basic Concept Belief Updating Most Probable Explanation (MPE) Maximum A Posteriori (MAP). Bayesian Networks Bucket Elimination Algorithm. Basic Concept 大同大學資工所 智慧型多媒體研究室. Satisfiability.

Download Presentation

Bayesian Networks Bucket Elimination Algorithm

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Bayesian networks bucket elimination algorithm

Bayesian NetworksBucket Elimination Algorithm

主講人:虞台文

大同大學資工所

智慧型多媒體研究室


Content

Content

  • Basic Concept

  • Belief Updating

  • Most Probable Explanation (MPE)

  • Maximum A Posteriori (MAP)


Bayesian networks bucket elimination algorithm1

Bayesian NetworksBucket Elimination Algorithm

Basic Concept

大同大學資工所

智慧型多媒體研究室


Satisfiability

Satisfiability

Given a statement of clauses (in disjunction normal form), the satisfiability problem is to determine whether there exists a truth assignment to make the statement true.

Examples:

Satisfiable

A=True, B=True, C=False, D=False

Satisfiable?


Resolution

Resolution

can be true if and only if

can be true.

unsatisfiable


Direct resolution

BucketA

BucketB

BucketC

BucketD

Direct Resolution

Example:

Given a set of clauses

and an order d=ABCD

Set initial buckets as follows:


Direct resolution1

BucketA

BucketB

BucketC

BucketD

Direct Resolution

Because no empty clause () is resulted, the statement is satisfiable.

How to get a truth assignment?


Direct resolution2

BucketA

BucketB

BucketC

BucketD

Direct Resolution


Direct resolution3

Direct Resolution


Queries on bayesian networks

Queries on Bayesian Networks

  • Belief updating

  • Finding the most probable explanation (mpe)

    • Given evidence, finding a maximum probability assignment to the rest of variables.

  • Maximizing a posteriori hypothesis (map)

    • Given evidence, finding an assignment to a subset of hypothesis variables that maximize their probability.

  • Maximizing the expected utility of the problem (meu)

    • Given evidence and utility function, finding a subset of decision variables that maximize the expected utility.


Bucket elimination

Bucket Elimination

  • The algorithm will be used as a framework for various probabilistic inferences on Bayesian Networks.


Preliminary elimination functions

Preliminary – Elimination Functions

Given a function h defined over subset of variables S, where X S,

Eliminate parameterX fromh

Defined overU = S– {X}.


Preliminary elimination functions1

Preliminary – Elimination Functions

Given a function h defined over subset of variables S, where X S,


Preliminary elimination functions2

Preliminary – Elimination Functions

Given function h1,…, hn defined over subset of variables S1,…, Sn, respectively,

Defined over


Preliminary elimination functions3

Preliminary – Elimination Functions

Given function h1,…, hn defined over subset of variables S1,…, Sn, respectively,


Bayesian networks bucket elimination algorithm2

Bayesian NetworksBucket Elimination Algorithm

Belief Updating

大同大學資工所

智慧型多媒體研究室


Bayesian networks bucket elimination algorithm

Goal

Normalization

Factor


Basic concept of variable elimination

A

C

B

F

D

G

Basic Concept of Variable Elimination

Example:


Basic concept of variable elimination1

Basic Concept of Variable Elimination

Example:


Basic concept of variable elimination2

Basic Concept of Variable Elimination

G(f)

D(a, b)

F(b, c)

B(a, c)

C(a)


Basic concept of variable elimination3

Basic Concept of Variable Elimination

BucketG

BucketD

BucketF

BucketB

BucketC

BucketA


Basic concept of variable elimination4

Basic Concept of Variable Elimination

BucketG

BucketD

BucketF

BucketB

BucketC

BucketA


Basic concept of variable elimination5

Basic Concept of Variable Elimination


Basic concept of variable elimination6

Basic Concept of Variable Elimination


Basic concept of variable elimination7

0.7

0.1

0.7

0.1

0.7

0.1

0.7

0.1

Basic Concept of Variable Elimination


Basic concept of variable elimination8

Basic Concept of Variable Elimination


Basic concept of variable elimination9

Basic Concept of Variable Elimination


Basic concept of variable elimination10

Basic Concept of Variable Elimination


Bucket elimination algorithm

Bucket Elimination Algorithm


Complexity

Complexity

  • The BuckElim Algorithm can be applied to any ordering.

  • The arity of the function recorded in a bucket

    • the numbers of variables appearing in the processed bucked, excluding the bucket’s variable.

  • Time and Space complexity is exponentially grow with a function of arity r.

  • The arity is dependent on the ordering.

  • How many possible orderings for BN’s variables?


Determination of the arity

A

C

B

F

D

G

Consider the ordering AFDCBG.

Determination of the Arity

BucketG

BucketB

1

G

4

BucketC

B

1

,3

C

BucketD

0

,2

D

BucketF

,1

0

F

BucketA

0

A


Determination of the arity1

A

C

B

1

1

F

G

D

4

4

B

G

3

1

C

2

0

D

1

0

F

0

0

A

d

Given the ordering, e.g., AFDCBG.

Determination of the Arity

The width of a graph is the maximum width of its nodes.

w(d) = 4

w*(d) = 4

w(d): width of initial graph

for ordering d.

w*(d): width of induced graph

for ordering d.

Width of node

Width of node

G

B

C

Induced

Graph

D

Initial

Graph

F

A


Definition of tree width

Definition of Tree-Width

Goal: Finding an ordering with smallest induced width.

Greedy heuristic and Approximation methods

Are available.

NP-Hard


Summary

Summary

  • The complexity of BuckElim algorithm is dominated by the time and space needed to process a bucket.

  • It is time and space is exponential in number of bucket variables.

  • Induced width bounds the arity of bucket functions.


Exercises

A

C

B

F

D

G

Exercises

  • Use BuckElim to evaluate P(a|b=1) with the following two ordering:

  • d1=ACBFDG

  • d2=AFDCBG

Give the details and make some conclusion.

How to improve the algorithm?


Bayesian networks bucket elimination algorithm3

Bayesian NetworksBucket Elimination Algorithm

Most Probable Explanation (MPE)

大同大學資工所

智慧型多媒體研究室


Bayesian networks bucket elimination algorithm

MPE

Goal:

evidence


Bayesian networks bucket elimination algorithm

MPE

Goal:


Notations

xi

Notations


Bayesian networks bucket elimination algorithm

MPE

Let


Bayesian networks bucket elimination algorithm

Xn

MPE

Some terms involve xn,

some terms not.

Xn is conditioned by its parents.

Xnconditions its children.


Bayesian networks bucket elimination algorithm

Xn

MPE

xnappears in these CPT’s

Not conditioned by xn

Conditioned by xn

Itself


Bayesian networks bucket elimination algorithm

MPE

Process the next bucket recursively.

Eliminate variable xnatBucketn.


Example

A

C

B

F

D

G

Example


Example1

A

C

B

F

D

G

Example

Consider ordering ACBFDG

BucketG

BucketD

BucketF

BucketB

BucketC

BucketA


Bucket elimination algorithm1

Bucket Elimination Algorithm


Exercise

Exercise

Consider ordering ACBFDG


Bayesian networks bucket elimination algorithm4

Bayesian NetworksBucket Elimination Algorithm

Maximum

A Posteriori (MAP)

大同大學資工所

智慧型多媒體研究室


Bayesian networks bucket elimination algorithm

MAP

Given a belief network, a subset of hypothesized variablesA=(A1, …, Ak), and evidence E=e, the goal is to determine


Example2

A

C

B

F

D

G

Example

Hypothesis (Decision)

Variables

g = 1


Bayesian networks bucket elimination algorithm

MAP

Ordering

Some of them may be observed


Bayesian networks bucket elimination algorithm

MAP


Bayesian networks bucket elimination algorithm

MAP


Bayesian networks bucket elimination algorithm

MAP

Bucket Elimination for MPE

Bucket Elimination for belief updating


Bucket elimination algorithm2

Bucket Elimination Algorithm


Example3

A

C

B

F

g = 1

D

G

Consider orderingCBAFDG

Example

BucketG

BucketD

BucketF

BucketA

BucketB

BucketC


Exercise1

A

C

B

F

g = 1

D

G

Consider orderingCBAFDG

Exercise

BucketG

BucketD

BucketF

BucketA

Give the detail

BucketB

BucketC


  • Login