Causal Models, Learning Algorithms and their Application to Performance Modeling

1. Causal Models, Learning Algorithms and their Application to Performance Modeling Jan Lemeire Parallel Systems lab November 15th 2006

2. Overview • I. Causal Models • II. Learning Algorithms • III. Performance Modeling • IV. Extensions Causal Performance Models

3. I. Multivariate Analysis • Variables • Experimental data Probabilistic model of joint distribution? Relational information? A priori unknown relations Causal Performance Models

4. A. Representation of distributions • Factorization • Reduction of factorization complexity • Bayesian Network Ordering 1 Ordering 2 Causal Performance Models

5. B. Representation of Independencies • Conditional independence • Qualitative property: P(rain|quality of speech)=P(rain)? • Markov condition in graph Variable becomes independent from all its non-descendants by conditioning on its direct parents. • graphical d-separation criterion Causal Performance Models

6. Faithfulness • Independence-map: • All independencies in the Bayesian network appear in the distribution • Faithfulness: Joint Distribution Directed Acyclic Graph Conditional independencies  d-separation • Theorem: if a faithful graph exists, it is the minimal factorization. Causal Performance Models

7. C. Representation of Causal Mechanisms Model of the underlying physical mechanisms • Definition through interventions • causal model + Conditional Probability Distributions + Causal Markov Condition = Bayesian network Causal Performance Models

8. Reductionism • Causal modeling = reductionism • Canonical representation: unique, minimal, independent • Building block = P(Xi|parents(Xi)) • Whole theory is based on this modularity • Intervention • = change of block Causal Performance Models

9. Ultimate motivation for causality If causal mechanisms are unrelated • model is faithful Model = canonical representation able to explain all qualitative properties (independencies) • close to reality Causal Performance Models

10. II. Learning Algorithms Two types: • Constraint-based based on the independencies • Scoring-based searches set of all models, give a score of how good they represent distribution Causal Performance Models

11. Step 1: Adjacency search • Property: adjacent nodes do not become independent • Algorithm: • start with full-connected graph • check for marginal independencies • check for conditional independencies Causal Performance Models

12. Step 2: Orientation • Property: • V-structure can be recognized • Algorithm: • look for v-structures • derived rules Causal Performance Models

13. Assumptions • General statistical assumptions: • No selection bias • Random sample • Sufficient data for correctness of statistical tests • Underlying network is faithful • Causal sufficiency • No unknown common causes Causal Performance Models

14. Criticism • Definition causality? • About predicting the effect of changes to the system • Faithfulness assumption • Eg.: accidental cancellation • Causal Markov Condition • “All relations are causal” • Learning algorithms are not robust • Statistical tests make mistakes Causal Performance Models

15. Part III: Performance Analysis • High-Performance computing parallel system 1 processor Performance Questions: • Performance prediction • Parameter-dependency? • Reasons of bad performance? • System-dependency? • Effect of Optimizations? Causal Performance Models

16. PhD?? Causal modeling (cf. COMO lab, VUB) • Representation form • Close to reality • Learning algorithms • TETRAD tool (open-source, java) Causal Performance Models

17. Performance Models • Aim performance analysis • Support software developer • High-performance applications • Expected properties • offer insight into causes performance degradation • prediction • estimate effect of optimizations • reusable submodels • separate application and system-dependency • reason under uncertainty • causal models Causal Performance Models

18. Integrated in statistical analysis • Statistical characteristics • Regression analysis • Probability table compression • Outlier detection Iterative process 1. Perform additional experiments 2. Extract additional characteristics 3. Indicate exceptions 4. Analyze the divergences of the data points with the current hypotheses Causal Performance Models

19. A. Model construction • Model of computation time of LU decom- position algorithm • elementsize (redundant variable) is sufficient for influence datatype -> cache misses • regression analysis on submodels X=f(parents) • analysis of parameters Causal Performance Models

20. B. Detection of unexpected dependencies • Point-to-point communication performance • background communication Causal Performance Models

21. C. Finding explanations for outliers Exceptional data in communication performance measurements Probability table compression => derived variable Interesting features Causal Performance Models

22. IV. Complexity of Performance Data • Mixture discrete and continuous variables • Mutual Information & Kernel Density Estimation • Non-linear relations • Mutual Information & Kernel Density Estimation • Deterministic relations • Augmented models & Complexity criterion • Context variables • Work in progress • Context-specific independencies • Work in progress Causal Performance Models

23. A. Information-theoretic Dependency • Discretized entropy for continuous variable • Entropy of random variable X • Mutual Information Causal Performance Models

24. B. Kernel Density Estimation • See applets Trade-off maximal entropy <> typicalness • Conclusions • Limited number data points needed • Discretization of continuous data justified • Form-free dependency measure Causal Performance Models

25. C. Deterministic relations • Y=f(X) • Y becomes independent from Z conditioned on X • ~ violation of the intersection condition (Pearl ’88) • Not faithfully describable Solution: augmented causal model - add regularity to model - adapt inference algorithms Causal Performance Models

26. The Complexity Criterion • X & Y contain equivalent information about Z • Select simplest relation Causal Performance Models

27. Augmented causal model • Restrict conditional independencies • Generalize d-separation • Reestablish faithfulness • Consistent models under Complexity Increase assumption { Causal Performance Models

28. Theory works! Deterministic B Probabilistic A Causal Performance Models

29. Conclusions • Benefit of the integration of statistical techniques • Causal modeling is a challenge • wants to know the inner from the outer • More information • http://parallel.vub.ac.be • http://parallel.vub.ac.be/~jan Causal Performance Models