causal models as minimal descriptions of multivariate systems n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Causal Models as Minimal Descriptions of Multivariate Systems PowerPoint Presentation
Download Presentation
Causal Models as Minimal Descriptions of Multivariate Systems

Loading in 2 Seconds...

  share
play fullscreen
1 / 20
orson-gallegos

Causal Models as Minimal Descriptions of Multivariate Systems - PowerPoint PPT Presentation

75 Views
Download Presentation
Causal Models as Minimal Descriptions of Multivariate Systems
An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Causal Models as Minimal Descriptions of Multivariate Systems Jan Lemeire June 15th 2006 Causality & MDL

  2. What can be learnt about the world from observations? • We have to look for regularities • & model them Causality & MDL

  3. MDL-approach to Learning • Occam’s Razor “Among equivalent models choose the simplest one.” • Minimum Description Length (MDL) “Select model that describes data with minimal #bits.” model = shortest program that outputs data length of program = Kolmogorov Complexity Learning = finding regularities = compression Causality & MDL

  4. Randomness vs. Regularity • 0110001101011010101 random string=incompressible=maximal information • 010101010101010101 regularity of repetitionallows compression Separation by the Two-part code Causality & MDL

  5. Model of Multivariate Systems • Variables • Experimental data Probabilistic model of joint distribution with minimal description length? Causality & MDL

  6. 1 variable • Average code length = Shannon entropy of P(x) • Multiple variables • With help of other, P(E| A…D) (CPD) • Factorization • Mutual information decreases entropy of variable Causality & MDL

  7. I. Conditional Independencies • Reduction of factorization complexity • Bayesian Network Ordering 1 Ordering 2 Causality & MDL

  8. II. Faithfulness Joint Distribution Directed Acyclic Graph Conditional independencies  d-separation Theorem: if a faithful graph exists, it is the minimal factorization. Causality & MDL

  9. III. Causal Interpretation • Definition through interventions Causality & MDL

  10. Reductionism • Causality = reductionism • Canonical representation: unique, minimal, independent • Building block = P(Xi|parentsi) • Whole theory is based on modularity like asymmetry of causality • Intervention • = change of block Causality & MDL

  11. Ultimate motivation for causality Model = canonical representation able to explain all regularities • close to reality Reality Learnt Example taken from Spirtes, Glymour and Scheines 1993, Fig. 3-23 Causality & MDL

  12. Causal model is MDL of joint distribution if Incompressible Incompressible (random distribution) Causality & MDL

  13. A Bayesian network with unrelated, random CPDs is faithful • d-separation tells what we can expect from a causal model • Eg. D depends on C, unless a dependency in P(D|C,E) P(d1|c0,e0).P(e0)+ P(d1|c0,e1).P(e1) = P(d1|c1,e0).P(e0)+ P(d1|c1,e1).P(e1) Causality & MDL

  14. When do causal models become incorrect? • Other regularities! Causality & MDL

  15. A. Lower-level regularities • Compression of the distributions Causality & MDL

  16. B. Better description form • Pattern • in figure random patterns -> distribution Causal model?? • Other models are better • Why? Complete symmetry among the variables Causality & MDL

  17. C. Interference with independencies X and Y independent by cancellation of X→U → Y and X → V → Y • dependency of both paths • = regularity Causality & MDL

  18. Violation of weak transitivity condition One of the necessary conditions for faithfulness Causality & MDL

  19. Deterministic relations • Y=f(X1, X2) • Y becomes (unexpectedly) independent from Z conditioned on X1 and X2 • ~ violation of the intersection condition Solution: augmented model - add regularity to model - adapt inference algorithms • Learning algorithm: • variables possibly contain equivalent information about another • Choose simplest relation Causality & MDL

  20. Conclusions • Interpretation of causality by the regularities • Canonical, faithful representation • ‘Describe all regularities’ • Causality is just one type of regularity? • Occam’s Razor works • Choice of simplest model • models close to ‘reality’ • but what is reality? • Atomic description of regularities that we observe? Papers, references and demos: http://parallel.vub.ac.be Causality & MDL