Statistics for non-statisticians

1. Statistics for non-statisticians November 3, 2012 Nick Barrowman, PhD Senior Statistician, Clinical Research Unit Children's Hospital of Eastern Ontario Research Institute Ottawa, Canada

2. Objectives • The session will provide an introduction to the statistics for the non-researcher. By the end of the session, participants will be able to: • Define the principal/foundational concepts of statistics • Interpret and describe statistics commonly reported in the medical literature • Identify "red flags" in research reports that may be signs of trouble

3. Workshop outline • Introductions (11:45) • Variability • Statistical measures of location and variability • Fundamental statistical ideas • Lunch (12:30) • Association and causation • Hands-on: statistics in the medical literature (1:00) • Why confidence intervals are all that (2:00) • Resources • Questions (2:30)

4. Variability

5. Variability • Patients vary. • Physicians vary. • Nurses vary. • Hospitals vary. • Measurements vary. • Disease states vary. • Immune response varies. • Drug adherence varies.

6. Variability Think of variability as noise We would like to detect the signal

7. Too much noise makes it hard to detect the signal

8. What is statistics? Statistics is the science of variability (at least that’s my definition)

9. The principal concepts of statistics • Variability can be modeled using probability. • This provides a framework for drawing inferences and quantifying our uncertainty.

10. “Uncertainty is an uncomfortable position. But certainty is an absurd one.” - Voltaire

11. Statistical measures of location and variability

12. A few words on the normal distribution • The normal distribution (also called the Gaussian distribution) is the best known probability distribution in statistics. • It is symmetric and bell-shaped • It is mathematically convenient and many statistical techniques involve an assumption of “normality”

13. Parametric and nonparametric methods • Statistical methods that assume the data follow a particular probability distribution are called “parametric”. • Other statistical methods make weaker assumptions and are called “nonparametric”. • Both types of methods have their uses!

14. Types of statistical measures • Measures of location • e.g. mean, median, … • Measures of variability • e.g. range, interquartile range, standard deviation, variance, … • Measures of association • e.g. correlation, NNT, odds ratio, … • Other measures • e.g. tests of normality, lack of fit, …

15. Measures of location: mean • The mean is the ordinary average. It is mathematically convenient and has a close connection with the normal distribution. • But … • It is strongly influenced by extreme values • For skewed distributions, the mean may be misleading

16. Measures of location: median • The median is the middle value (or the average of the two middle values). • The median … • is not influenced at all by extreme values • is easily interpreted for both symmetric and skewed distributions Median Mean

17. Measures of variability: Range • The range of a sample usually means minimum , maximum But sometimes people quote maximum – minimum • A single enormous or tiny value will change the range – and it may be an error! • As the size of a sample increases, the range cannot get narrower.

18. Measures of variability: Interquartile range (IQR) • The interquartile range • Consider 100 observations along a continuum

19. Measures of variability: Interquartile range (IQR) • Rank them in order from 1 to 100 • Ranks are the key to many nonparametric methods

20. Measures of variability: Interquartile range (IQR) • Then separate them into 4 equal groups

21. Measures of variability: Interquartile range (IQR) • Then separate them into 4 equal groups IQR q1 q2 q3

22. Measures of variability: Interquartile range (IQR) • The middle 50% of the data lie within the IQR • Unlike the range, IQR doesn’t depend on the extremes IQR q1 q2 q3

23. Measures of variability: Standard deviation • The standard deviation (SD) is a more mathematically convenient measure of variability and is closely connected to the normal distribution. • The SD is called a parametric measure (whereas the IQR is nonparametric) • For a normal distribution, we expect 68% of observations to lie within 1 SD on either side of the mean, and 95% to lie within 2 SDs. • For skewed distributions SD is harder to interpret.

24. Red flag The standard deviation is greater than the mean for a variable that cannot take negative values. When this happens the variable may have a skewed distribution, and the mean may be a misleading measure of location. Be careful with statistical methods that assume normality.

25. Measures of variability: Variance • The variance is the square of the standard deviation. • So variance is measured in squared units rather than units of the variable in question. • Why bother with variance? • To obtain a more precise estimate, it is often useful to take the average of several repeat observations (e.g. blood pressure). The improved precision can easily be determined using variance. • Several statistical measures (such as measures of reliability) are defined in terms of variances rather than standard deviations.

26. Red flag Interpreting the standard error as a measure of variability. The standard error is closely related to the standard deviation, but it is a measure of the uncertainty in an estimate, often used to compute a confidence interval.

27. Fundamental statistical ideas

28. Fundamental statistical ideas The population and the sample Confidence intervals Hypothesis tests P-values

29. Population vs. sample Calculation Population mean blood pressure Sample mean blood pressure Inference Random sample Population

30. A typically cryptic description “The mean systolic blood pressure in group A was 110 mmHg, while in group B it was 104 mmHg … There was no difference in systolic blood pressure between the groups.”

31. A typically cryptic description “The mean systolic blood pressure in group A was 110 mmHg, while in group B it was 104 mmHg … There was no difference in systolic blood pressure between the groups.” Statements like this can be perplexing. For a start, how can there be no difference when there is clearly a difference?

32. Population vs. sample Calculation Sample difference between groups in mean blood pressure Population difference between groups in mean blood pressure Inference Random sample Population High BP Low BP

33. A typically cryptic description “The mean systolic blood pressure in group A was 110 mmHg, while in group B it was 104 mmHg … There was no difference in systolic blood pressure between the groups.”

34. A typically cryptic description Sample “The mean systolic blood pressure in group A was 110 mmHg, while in group B it was 104 mmHg … There was no difference in systolic blood pressure between the groups.” Population

35. A typically cryptic description “The mean systolic blood pressure in group A was 110 mmHg, while in group B it was 104 mmHg … There was no difference in systolic blood pressure between the groups.” statistically significant More on this later …

36. A typically cryptic description “The mean systolic blood pressure in group A was 110 mmHg, while in group B it was 104 mmHg … There was no statistically significant difference in systolic blood pressure between the groups.” Even if the means do not differ significantly between groups, systolic blood pressure varies within each group. mean

37. A typically cryptic description “The mean systolic blood pressure in group A was 110 mmHg, while in group B it was 104 mmHg … There was no statistically significant difference in mean systolic blood pressure between the groups.” (SD = 4.2 mmHg) (SD = 5.0 mmHg) SD is the standard deviation. Estimates of variability are essential.

38. Red flag Failing to report estimates of variability Variability is always present. Failing to report estimates of variability can be misleading and can also make it impossible for the reader to verify results.

39. Comparisons Many studies focus on comparisons between groups between a single group and a reference standard. e.g. Compare weight gain: On average, did group A gain more than group B? On average, did people in a single group gain weight? Here the reference standard is no change.

40. The null hypothesis The null hypothesis is a default assumption about the population, usually that there is no difference. For example: In the population, there is no difference between the mean blood pressure for groups A and B.

41. Weight gain example With two groups, the null hypothesis is: Mean weight gain is the same in the two groups i.e. Difference in mean weight gain = 0 With one group, the null hypothesis is: Mean weight gain = 0 In these two cases, zero is the “null value”

42. Other examples of the null hypothesis Example: Mortality in two groups Mortality rate in group A = mortality rate in group B i.e. Relative risk of mortality is 1. So 1 is the null value. Example: IQ in a single group Mean IQ is 100. So 100 is the null value.

43. Hypothesis testing An example: Bedside Limited Echocardiography by the Emergency Physician Is Accurate During Evaluation of the Critically Ill Patient Pershad et al. Pediatrics 2004;114;e667-e671. Goal: to compare echocardiography measurements made by emergency physicians and experienced pediatric echocardiography providers.

44. Hypothesis testing Patient Emerg Doc Cardiographer Shortening fraction SF (%)

45. Hypothesis testing Difference between measurements made by echocardiographers and emergency physicians

46. Hypothesis testing Difference between measurements made by echocardiographers and emergency physicians On average echocardiographers’ estimates were higher by 4.4%.

47. Hypothesis testing Difference between measurements made by echocardiographers and emergency physicians On average echocardiographers’ estimates were higher by 4.4%. Could this difference be a chance occurrence?

48. Hypothesis testing We need to test the hypothesis that in the population there is no difference. We often report a p-value: the probability of observing a difference that is at least as extreme as what was observed, assuming there is no differencein the population. Usually consider a p-value < 0.05 to be statistically significant.

49. Hypothesis testing Difference between measurements made by echocardiographers and emergency physicians On average echocardiographers’ estimates were higher by 4.4%. P=0.003 (t-test) statistically significant

50. Statistical vs. Clinical significance But the authors note, “Although statistically significant, the difference of 4.4% in the estimation of SF may not be clinically relevant.” A statistically significant finding is not always clinically significant. Subject area judgement is always needed.