gapminder

1. Discovering patterns in data: correlation and visualization www.gapminder.org Will Hamilton April 2017

2. Contents PART 2: Data visualisation • Rules for good graphing • Bad graphs PART 1: Correlation • What is correlation? • Types of correlation • Linear regression • Multiple linear regression • Linear regression diagnostics • Non-linear correlation • Logistic regression • Correlation and causation Conclusions • The age of data!

3. Dependence • Independence • Occurrence of one variable does not affect the probability of occurrence of the other • Dependence • Any statistical relationship between two random variables

5. Describing linear relationships: • Pearson Product-Moment Correlation Coefficient • Linear regression

6. mtcars

8. Linear regression • “Least squares method” • Minimise the sum of squared difference between each datapoint and the line

10. y = –5.34 χ + 37 • r2 = 0.74 • P = 1.29x10-10

11. Explaining linear regression in words y = –5.34 χ + 37 For every 1000lb increase in car weight (χ), miles per gallon (y) decreases by 5.34 mpg.

12. Explaining linear regression in words r2 = 0.74 Our model (y = –5.34 χ + 37) explains 74% of the variance in our data. 26% of the variance in car miles per gallon is not accounted for in our model.

13. Explaining linear regression in words P = 1.29x10-10 Statistical significance test i.e. The probability of obtaining data as extreme or more extreme than this if there were no association between mpg and weight

14. Don’t confuse r2 and m • y = χ • r2 = 1.0 • P < 2.2x10-16 • y = 2χ • r2 = 1.0 • P < 2.2x10-16

16. y = 112 χ – 131 • r2 = 0.78 • P = 1.22x10-11

17. Explaining linear regression in words • y = 112 χ – 131 • For every 1000lb increase in car weight (χ), engine displacement (y) increases by 112 cubic inches. • r2 = 0.78 • Our model explains 78% of the data variance • P = 1.22x10-11 • Probability of getting data as extreme or more extreme than those obtained if there were no association between car weight and engine displacement is very small

18. Multiple linear regression • What if multiple independent variables influence your dependent variable? • Y = c + m1x1 + m2x2 + m3x3 ... + mnxn

19. Size of bubble = total population living with HIV (PLHIV) Colour of bubble = WHO global region. Blue = Sub-Saharan Africa. Countries with HIV prevalence >3% are named. There is no obvious/ linear correlation between HIV prevalence and a nation’s wealth

20. Just Sub-Saharan African countries

21. There is no obvious/ linear correlation between HIV prevalence and a nation’s human development index

22. Just Sub-Saharan African countries

23. There is no obvious/ linear correlation between HIV prevalence and a nation’s adult female literacy…

24. … In fact, among Sub-Saharan African countries there is a positive correlation; i.e. more literate countries tend to have higher HIV prevalences!

25. Among Sub-Saharan African countries, high risk sex (defined by DHS data, e.g. multiple concurrent sexual partners) is positively correlated with HIV prevalence

26. Multiple linear regression • Y = c + m1x1 + m2x2 + m3x3 ... + mnxn • Y = HIV prevalence • X1 = Log10 GDP per capita • X2 = Human Development Index • X3 = Female literacy rate • X4 = High risk sex behaviours

27. Reality-checking your linear regressions Diagnostic plots: • Residuals vs Fitted • Normal Q-Q • Scale-Location • Residuals vs Leverege

30. Residuals vs Fitted values

32. Testing for homoscedasticity

36. Lesotho Swaziland Burundi

37. Non-linear rank correlations • Things that are correlated but not in straight line (linear) relationships • Ranking correlations • Spearman’s rho (ρ) • Kendall’s tau (τ)

38. Levenson RM, et al. 2015. Pigeons (Columba livia) as Trainable Observers of Pathology and Radiology Breast Cancer Images. PLoS ONE 10(11): e0141357.

39. Spearman’s rank correlation coefficient ρ = 0.97 P < 2.2x10-16

40. Logistic regression Analyses of binary categorical dependent variables 1 • Odds of dependent variable being 0 or 1 computed for every position along the independent variable • Odds Ratio (OR) imputed 0

41. Multiple logistic regression: worked example • 321 children with P. falciparum malaria at a hospital in rural Ghana. • 27 (8.41%) of the children died • Were any clinical variables associated with mortality? With thanks to Andrew McGovern

42. Correlation ≠ Causation !

43. From Tyler Vigen - http://www.tylervigen.com/spurious-correlations

44. “Establishing the why” Strength Consistency Specificity Temporality Biological gradient Plausibility Coherence Experiment Analogy The Environment and Disease: Association or Causation? A. Bradford Hill, Proceedings of the Royal Society of Medicine, 1965

45. Data visualisation :-D https://www.youtube.com/watch?v=9mnVWJpMhuE

46. The visual display of quantitative information • Show the data • Avoid distorting the data • High “data-ink ratio” • Avoid “chartjunk” • Data richness • Integrated Micro/ Macro-trend data displays • Neuropsychology of human perception Edward Tufte

48. The visual display of quantitative information Avoid “Chartjunk” Edward Tufte