Practically Perfect

1 / 14

# Practically Perfect - PowerPoint PPT Presentation

Practically Perfect. Chris Meek Max Chickering. X. Y. Perfectness. D-separation implies independence (Pearl, 1988) but… D-connection does not imply dependence. p ( Y =0| X =0) = p ( Y =0| X= 1) (parameter cancellation). X. Z. Y. W. Sampling Local Distributions.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Practically Perfect' - terry

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Practically Perfect

Chris Meek

Max Chickering

X

Y

Perfectness

D-separation implies independence (Pearl, 1988)

but…

D-connection does not imply dependence

p(Y=0|X=0) = p(Y=0|X=1)

(parameter cancellation)

X

Z

Y

W

Sampling Local Distributions

If we randomly sample local parameters, how

often will the joint be perfect?

Previous Result

Probability of sampling non-perfect

distribution is measure zero

(Meek, 1995 and Spirtes et al., 2001)

This Paper

Extend to finite-bit representations:

upper bound on probability of non-perfect

distribution

Why do we care?
• Previous theoretical result not applicable to real-world software
• Correctness of learning: typically need perfectness

Required for “gold standard” experimental

evaluation of structure learning

Continuous to Finite

Sampling

probability

0

1

Renormalized Delta Functions

Sampling

probability

11

00

01

10

X

Y

Independence = Polynomial Root

Base polynomial for “X independent of Y”:

p(X=0,Y=0) · p(X=1,Y=1) – p(X=0,Y=1) · p(X=1,Y=0)

Perfect Polynomial: Enumerate every d-connection in the model, and multiply corresponding base polynomials.

Non-perfect only if perfect polynomial is zero!

Schwartz-Zippel Theorem
• Every polynomial has a finite number of roots
• Sample each variable in the polynomial independently from a uniform over fixed set of values
• Bound on the probability that we sample a root!
Example

For a fixed X, at most three roots (3 roots per column)

For a fixed Y, at most two roots (2 roots per row)

1

Y

0

0

1

X

Problem 1: Variational Dependence

1

p(Y=0)

0

0

1

p(Y=1)

Y has three states: two independent parameters

Theorem does not apply.

Solution: Change Parameterization

1

0

0

1

Sample new parameters independently from Beta distribution

=

Sample “normal” parameters from a Dirichlet distribution

Problem 2: Non-uniform Sampling

1

Solution:

Simple Extension to S-Z:

Bound is a function of the maximum density height

0

0

1

Example
• b-bit representation
• Sample parameters from a uniform Dirichlet
• m variables
• No variable has more than rmax states

16 vars, 4 states, 64-bit numbers  p(non-perfect) ≤ 1/232