Loading in 2 Seconds...

Causal Models as Minimal Descriptions of Multivariate Systems

Loading in 2 Seconds...

75 Views

Download Presentation
##### Causal Models as Minimal Descriptions of Multivariate Systems

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Causal Models as Minimal Descriptions of Multivariate**Systems Jan Lemeire June 15th 2006 Causality & MDL**What can be learnt about the world from observations?**• We have to look for regularities • & model them Causality & MDL**MDL-approach to Learning**• Occam’s Razor “Among equivalent models choose the simplest one.” • Minimum Description Length (MDL) “Select model that describes data with minimal #bits.” model = shortest program that outputs data length of program = Kolmogorov Complexity Learning = finding regularities = compression Causality & MDL**Randomness vs. Regularity**• 0110001101011010101 random string=incompressible=maximal information • 010101010101010101 regularity of repetitionallows compression Separation by the Two-part code Causality & MDL**Model of Multivariate Systems**• Variables • Experimental data Probabilistic model of joint distribution with minimal description length? Causality & MDL**1 variable**• Average code length = Shannon entropy of P(x) • Multiple variables • With help of other, P(E| A…D) (CPD) • Factorization • Mutual information decreases entropy of variable Causality & MDL**I. Conditional Independencies**• Reduction of factorization complexity • Bayesian Network Ordering 1 Ordering 2 Causality & MDL**II. Faithfulness**Joint Distribution Directed Acyclic Graph Conditional independencies d-separation Theorem: if a faithful graph exists, it is the minimal factorization. Causality & MDL**III. Causal Interpretation**• Definition through interventions Causality & MDL**Reductionism**• Causality = reductionism • Canonical representation: unique, minimal, independent • Building block = P(Xi|parentsi) • Whole theory is based on modularity like asymmetry of causality • Intervention • = change of block Causality & MDL**Ultimate motivation for causality**Model = canonical representation able to explain all regularities • close to reality Reality Learnt Example taken from Spirtes, Glymour and Scheines 1993, Fig. 3-23 Causality & MDL**Causal model is MDL of joint distribution if**Incompressible Incompressible (random distribution) Causality & MDL**A Bayesian network with unrelated, random CPDs is faithful**• d-separation tells what we can expect from a causal model • Eg. D depends on C, unless a dependency in P(D|C,E) P(d1|c0,e0).P(e0)+ P(d1|c0,e1).P(e1) = P(d1|c1,e0).P(e0)+ P(d1|c1,e1).P(e1) Causality & MDL**When do causal models become incorrect?**• Other regularities! Causality & MDL**A. Lower-level regularities**• Compression of the distributions Causality & MDL**B. Better description form**• Pattern • in figure random patterns -> distribution Causal model?? • Other models are better • Why? Complete symmetry among the variables Causality & MDL**C. Interference with independencies**X and Y independent by cancellation of X→U → Y and X → V → Y • dependency of both paths • = regularity Causality & MDL**Violation of weak transitivity condition**One of the necessary conditions for faithfulness Causality & MDL**Deterministic relations**• Y=f(X1, X2) • Y becomes (unexpectedly) independent from Z conditioned on X1 and X2 • ~ violation of the intersection condition Solution: augmented model - add regularity to model - adapt inference algorithms • Learning algorithm: • variables possibly contain equivalent information about another • Choose simplest relation Causality & MDL**Conclusions**• Interpretation of causality by the regularities • Canonical, faithful representation • ‘Describe all regularities’ • Causality is just one type of regularity? • Occam’s Razor works • Choice of simplest model • models close to ‘reality’ • but what is reality? • Atomic description of regularities that we observe? Papers, references and demos: http://parallel.vub.ac.be Causality & MDL