Loading in 2 Seconds...
Loading in 2 Seconds...
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Causal Models as Minimal Descriptions of Multivariate Systems Jan Lemeire June 15th 2006 Causality & MDL
What can be learnt about the world from observations? • We have to look for regularities • & model them Causality & MDL
MDL-approach to Learning • Occam’s Razor “Among equivalent models choose the simplest one.” • Minimum Description Length (MDL) “Select model that describes data with minimal #bits.” model = shortest program that outputs data length of program = Kolmogorov Complexity Learning = finding regularities = compression Causality & MDL
Randomness vs. Regularity • 0110001101011010101 random string=incompressible=maximal information • 010101010101010101 regularity of repetitionallows compression Separation by the Two-part code Causality & MDL
Model of Multivariate Systems • Variables • Experimental data Probabilistic model of joint distribution with minimal description length? Causality & MDL
1 variable • Average code length = Shannon entropy of P(x) • Multiple variables • With help of other, P(E| A…D) (CPD) • Factorization • Mutual information decreases entropy of variable Causality & MDL
I. Conditional Independencies • Reduction of factorization complexity • Bayesian Network Ordering 1 Ordering 2 Causality & MDL
II. Faithfulness Joint Distribution Directed Acyclic Graph Conditional independencies d-separation Theorem: if a faithful graph exists, it is the minimal factorization. Causality & MDL
III. Causal Interpretation • Definition through interventions Causality & MDL
Reductionism • Causality = reductionism • Canonical representation: unique, minimal, independent • Building block = P(Xi|parentsi) • Whole theory is based on modularity like asymmetry of causality • Intervention • = change of block Causality & MDL
Ultimate motivation for causality Model = canonical representation able to explain all regularities • close to reality Reality Learnt Example taken from Spirtes, Glymour and Scheines 1993, Fig. 3-23 Causality & MDL
Causal model is MDL of joint distribution if Incompressible Incompressible (random distribution) Causality & MDL
A Bayesian network with unrelated, random CPDs is faithful • d-separation tells what we can expect from a causal model • Eg. D depends on C, unless a dependency in P(D|C,E) P(d1|c0,e0).P(e0)+ P(d1|c0,e1).P(e1) = P(d1|c1,e0).P(e0)+ P(d1|c1,e1).P(e1) Causality & MDL
When do causal models become incorrect? • Other regularities! Causality & MDL
A. Lower-level regularities • Compression of the distributions Causality & MDL
B. Better description form • Pattern • in figure random patterns -> distribution Causal model?? • Other models are better • Why? Complete symmetry among the variables Causality & MDL
C. Interference with independencies X and Y independent by cancellation of X→U → Y and X → V → Y • dependency of both paths • = regularity Causality & MDL
Violation of weak transitivity condition One of the necessary conditions for faithfulness Causality & MDL
Deterministic relations • Y=f(X1, X2) • Y becomes (unexpectedly) independent from Z conditioned on X1 and X2 • ~ violation of the intersection condition Solution: augmented model - add regularity to model - adapt inference algorithms • Learning algorithm: • variables possibly contain equivalent information about another • Choose simplest relation Causality & MDL
Conclusions • Interpretation of causality by the regularities • Canonical, faithful representation • ‘Describe all regularities’ • Causality is just one type of regularity? • Occam’s Razor works • Choice of simplest model • models close to ‘reality’ • but what is reality? • Atomic description of regularities that we observe? Papers, references and demos: http://parallel.vub.ac.be Causality & MDL