How confident are we in the estimation of mean/proportion we have calculated?

1. How confident are we in the estimation of mean/proportion we have calculated?

5. Measures of precision: • Standard error of mean, SEM Standard error of proportion, SE(p) • Confidence interval for mean Confidence interval for proportion

6. Standard error of mean, SEM Standard deviation, SD • SEM is smaller (estimate is more precise): • the larger is N (number of patients) • the smaller is SD (dispersion of data) Number of patients

8. 95% confidence interval for mean, 95% CI • Together with SEM, 95% CI is also the measure of precision • Unlike SEM, 95% CI also estimates accuracy of the result ie. 95% is accurate that interval includes true (population) mean)

9. 95% confidence interval for mean • If we draw a 100 samples from our population we would find the true population value within 95% confidence interval in 95 samples. 20 samples

10. Critical values for 90%, 95% and 99% level of confidence 90% CI => mean ± 1.65 SEM 95% CI => mean ± 1.96 SEM 99% CI => mean ± 2.58 SEM Level of Confidence - Critical Value 0.75, or 75% 1.15 0.80, or 80% 1.28 0.85, or 85% 1.44 0.90, or 90% 1.65 0.95, or 95% 1.96 0.98, or 98% 2.33 0.99, or 99% 2.58

11. Example 1 • The average systolic BP before treatment in study A, of a group of 100 hypertensive patients, was 170 mmHg. After treatment with the new drug the mean BP dropped by 20 mmHg. • If the 95% CI is 15–25, this means: we can be 95% confident that the true effect of treatment is to lower the BP by 15–25 mmHg.

12. Example 2 • In study B 50 patients were treated with the same drug, also reducing their mean BP by 20 mmHg, but with a wider 95% CI of -5 to +45. This CI includes zero (no change). • This means: there is more than a 5% chancethat there was no true change in BP, and that the drug was actually ineffective..

13. Example 3 – Meta analysis Fig. Plot of 5 studies of a new antihypertensive drug. Which study showed the greatest change? Did all the studies show change in favour of theintervention? Were the changes statistically significant?

14. Watch out for... • The size of a CI is related to the sample size of the study. Larger studies usually have a narrower CI.

15. Proportion • Standard error of proportion, SE(p) SE(p) = √(p(1 – p)/n) • Confidence interval for proportion

16. The standard deviation describes the variability of a sample; The standard error of the mean(SEM)doesnotdescribethesample but describes the uncertainty of how the samplemean represents the populationmean.

17. SD CI • Standard deviation tells us about the variability (spread) in a sample. • The CI tells us the range in which the true value (the mean if the sample were infinitely large) is likely to be.

18. Krebs NF, Westcott JE, Culbertson DL et. al. Comparison of complementary feeding strategies to meet zinc requirements of older breastfed infants. Am J Clin Nutr. 2012; 96:30-35 “Mean (±SEM) total absorbed zinc amounts were 0.80 ± 0.08, 0.71 ± 0.09, and 0.52 ± 0.05 mg/d for the: meat, iron-and-zinc-fortified infant cereal, and whole-grain, iron-only-fortified infant cereal groups of infants.” Meat Fe&Zn Fe Meat Fe&Zn Fe SEM CI

19. TRUE or FALSE • What does a small standard error tell us about the sample estimate of the mean? • That it is highly variable • That the population standard deviation may be small • That the sample size is probably small • That it is imprecise

20. TRUE or FALSE What will tend to make the standard error larger? • A small variance • A large standard deviation • Imprecise data • Inaccurate data

21. Statistical inference:Hypothesis testing

22. Statistical Inference: Brief Overview • Statistics: Learning from Samples about Populations • Inference 1: Confidence Intervals • What does the 95% CI really mean? • Inference 2: Hypothesis Tests • What does a p-value really mean? • When to use which test?

23. Examples of hypothesis testing in medical research • In epidemiological studies: Is there a relationshipbetween a variable of interest and an outcome of interest? • Ie. smoking and lung cancer • Stress and thyroid cancer • In clinical trails: Is experimental therapy more effective than standard therapy or placebo?

24. Hypothesis testing = testing of statistical hypothesis

25. Statistical hypothesis Statements about population parameter values. • Null hypothesis (H0) says a parameter is unchanged from a default, pre-specified value; and • Alternative hypothesis (H1) says parameter has a value incompatible with H0

26. Example: Hypertension and Cholesterol Make appropriate statistical hypotheses: Assumption: Mean cholesterol in hypertensive men is equal to mean cholesterol in male general population (20-74 years old). In the 20-74 year old male population the mean serum cholesterol is 211 mg/ml with a standard deviation of 46 mg/ml

27. Example: Hypertension and Cholesterol Null hypothesis => nodifference between treatments • H0: μhypertensive = μgeneral population • H0: μhypertensive = 211 mg/ml • μ = population mean of serum cholesterol • Mean cholesterol for hypertensive men =mean for general male population Alternative hypothesis • HA: μhypertensive≠ μ general population • HA: μ hypertensive≠ 211 mg/ml

28. Null and alternative hypothesis Two-sidedtests One-sidedtests

29. How to choose one or the other?

30. Steps in Hypothesis Tests Assume H0 is true i.e. believe results are a matter of chance Quantify how far away are data from being consistent with H0 by evaluating quantity called atest statistic Assess probability of results at least this extreme - call this the p-value of the test Reject H0 (believe H1) if this p-value is small or keep H0 (do not believe H1) otherwise

31. Significant difference between the treatments Null hypothesis is rejected, alternative is accepted Interpretation of P-value (0.05) No difference between the treatments (observed difference having happened by chance) Null hypothesis is accepted P>=0.05 P<0.05 5%

32. P-value • The P value gives the probability of observed and more extreme difference having happened by chance. • P = 0.500 means that the probability of the difference having happened by chance is 0.5 in 1, or 1 in 2. • P = 0.05 means that the probability of the difference having happened by chance is 0.05 in 1, i.e. 1 in 20.

33. P-value • The lower the P value, the less likely it is that the difference happened by chance and so the higher the significance of the finding. • P = 0.01 is often considered to be “highly significant”. It means that the difference will only have happened by chance 1 in 100 times. This is unlikely, but still possible.

34. Chance • https://www.learningplace.com.au/en/teachingandlearning/summerschools/lo_chanceSimulator/lo_chanceSimulator.html

35. Example 1 • Out of 50 new babies on average 25 will be girls, sometimes more, sometimes less. • Say there is a new fertility treatment and we want to know whether it affects the chance of having a boy or a girl. • Null hypothesis –the treatment does not alter the chance of having a girl.

36. Example 1 • Null hypothesis –the treatment does not alter the chance of having a girl. • Out of the first 50 babies resulting from the treatment, 15 are girls. • We need to know the probability that this just happened by chance, i.e. did this happen by chance or has the treatment had an effect on the sex of the babies? • P=0.007

37. Example 1 • The P value in this example is 0.007. • This means the result would only have happened by chance in 0.007 in 1 (or 1 in 140) times if the treatment did not actually affect the sex of the baby. • This is highly unlikely, so we can reject our hypothesis and conclude that the treatment probably does alter the chance of having a girl.

38. Example 2 • Patients with minor illnesses were randomized to see either Dr Smith or Dr Jones. Dr Smith ended up seeing 176 patients in the study whereas Dr Jones saw 200 patients.

39. Example 2 • Patients with minor illnesses were randomized to see either Dr Smith or Dr Jones. Dr Smith ended up seeing 176 patients in the study whereas Dr Jones saw 200 patients (Table 2).

40. How to choose the appropriate statistical test? • Type of data (type of variable)? • Number of groups? • Related or independent groups? • Normal or asymmetric distribution?

41. Numerical

42. Example: Hypertension and Cholesterol Make appropriate statistical hypotheses: Mean cholesterol in hypertensive men is 220 mg/ml with a standard deviation of 39 mg/ml. In the 20-74 year old male population the mean serum cholesterol is estimated to 211 mg.

45. Research hypothesis Statistical hypothesis Hypothesis vs Statictical Hypothesis • Alcohol intake increases driver’s reaction time. • Mean reaction time in examinees drinking alcohol is greater than in nondrinking controls.