# Bayesian Networks Bucket Elimination Algorithm - PowerPoint PPT Presentation

1 / 57

Bayesian Networks Bucket Elimination Algorithm. 主講人：虞台文 大同大學資工所 智慧型多媒體研究室. Content. Basic Concept Belief Updating Most Probable Explanation (MPE) Maximum A Posteriori (MAP). Bayesian Networks Bucket Elimination Algorithm. Basic Concept 大同大學資工所 智慧型多媒體研究室. Satisfiability.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Bayesian Networks Bucket Elimination Algorithm

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

## Bayesian NetworksBucket Elimination Algorithm

### Content

• Basic Concept

• Belief Updating

• Most Probable Explanation (MPE)

• Maximum A Posteriori (MAP)

## Bayesian NetworksBucket Elimination Algorithm

Basic Concept

### Satisfiability

Given a statement of clauses (in disjunction normal form), the satisfiability problem is to determine whether there exists a truth assignment to make the statement true.

Examples:

Satisfiable

A=True, B=True, C=False, D=False

Satisfiable?

### Resolution

can be true if and only if

can be true.

unsatisfiable

BucketA

BucketB

BucketC

BucketD

### Direct Resolution

Example:

Given a set of clauses

and an order d=ABCD

Set initial buckets as follows:

BucketA

BucketB

BucketC

BucketD

### Direct Resolution

Because no empty clause () is resulted, the statement is satisfiable.

How to get a truth assignment?

BucketA

BucketB

BucketC

BucketD

### Queries on Bayesian Networks

• Belief updating

• Finding the most probable explanation (mpe)

• Given evidence, finding a maximum probability assignment to the rest of variables.

• Maximizing a posteriori hypothesis (map)

• Given evidence, finding an assignment to a subset of hypothesis variables that maximize their probability.

• Maximizing the expected utility of the problem (meu)

• Given evidence and utility function, finding a subset of decision variables that maximize the expected utility.

### Bucket Elimination

• The algorithm will be used as a framework for various probabilistic inferences on Bayesian Networks.

### Preliminary – Elimination Functions

Given a function h defined over subset of variables S, where X S,

Eliminate parameterX fromh

Defined overU = S– {X}.

### Preliminary – Elimination Functions

Given a function h defined over subset of variables S, where X S,

### Preliminary – Elimination Functions

Given function h1,…, hn defined over subset of variables S1,…, Sn, respectively,

Defined over

### Preliminary – Elimination Functions

Given function h1,…, hn defined over subset of variables S1,…, Sn, respectively,

## Bayesian NetworksBucket Elimination Algorithm

Belief Updating

Normalization

Factor

A

C

B

F

D

G

Example:

Example:

G(f)

D(a, b)

F(b, c)

B(a, c)

C(a)

BucketG

BucketD

BucketF

BucketB

BucketC

BucketA

BucketG

BucketD

BucketF

BucketB

BucketC

BucketA

0.7

0.1

0.7

0.1

0.7

0.1

0.7

0.1

### Complexity

• The BuckElim Algorithm can be applied to any ordering.

• The arity of the function recorded in a bucket

• the numbers of variables appearing in the processed bucked, excluding the bucket’s variable.

• Time and Space complexity is exponentially grow with a function of arity r.

• The arity is dependent on the ordering.

• How many possible orderings for BN’s variables?

A

C

B

F

D

G

Consider the ordering AFDCBG.

### Determination of the Arity

BucketG

BucketB

1

G

4

BucketC

B

1

,3

C

BucketD

0

,2

D

BucketF

,1

0

F

BucketA

0

A

A

C

B

1

1

F

G

D

4

4

B

G

3

1

C

2

0

D

1

0

F

0

0

A

d

Given the ordering, e.g., AFDCBG.

### Determination of the Arity

The width of a graph is the maximum width of its nodes.

w(d) = 4

w*(d) = 4

w(d): width of initial graph

for ordering d.

w*(d): width of induced graph

for ordering d.

Width of node

Width of node

G

B

C

Induced

Graph

D

Initial

Graph

F

A

### Definition of Tree-Width

Goal: Finding an ordering with smallest induced width.

Greedy heuristic and Approximation methods

Are available.

NP-Hard

### Summary

• The complexity of BuckElim algorithm is dominated by the time and space needed to process a bucket.

• It is time and space is exponential in number of bucket variables.

• Induced width bounds the arity of bucket functions.

A

C

B

F

D

G

### Exercises

• Use BuckElim to evaluate P(a|b=1) with the following two ordering:

• d1=ACBFDG

• d2=AFDCBG

Give the details and make some conclusion.

How to improve the algorithm?

## Bayesian NetworksBucket Elimination Algorithm

Most Probable Explanation (MPE)

Goal:

evidence

Goal:

xi

Let

Xn

### MPE

Some terms involve xn,

some terms not.

Xn is conditioned by its parents.

Xnconditions its children.

Xn

### MPE

xnappears in these CPT’s

Not conditioned by xn

Conditioned by xn

Itself

### MPE

Process the next bucket recursively.

Eliminate variable xnatBucketn.

A

C

B

F

D

G

A

C

B

F

D

G

### Example

Consider ordering ACBFDG

BucketG

BucketD

BucketF

BucketB

BucketC

BucketA

### Exercise

Consider ordering ACBFDG

## Bayesian NetworksBucket Elimination Algorithm

Maximum

A Posteriori (MAP)

### MAP

Given a belief network, a subset of hypothesized variablesA=(A1, …, Ak), and evidence E=e, the goal is to determine

A

C

B

F

D

G

### Example

Hypothesis (Decision)

Variables

g = 1

### MAP

Ordering

Some of them may be observed

### MAP

Bucket Elimination for MPE

Bucket Elimination for belief updating

### Bucket Elimination Algorithm

A

C

B

F

g = 1

D

G

Consider orderingCBAFDG

### Example

BucketG

BucketD

BucketF

BucketA

BucketB

BucketC

A

C

B

F

g = 1

D

G

Consider orderingCBAFDG

BucketG

BucketD

BucketF

BucketA

Give the detail

BucketB

BucketC