Synthesis and Review 2/20/12

1. Synthesis and Review • 2/20/12 • Hypothesis Tests: the big picture • Randomization distributions • Connecting intervals and tests • Review of major topics • Open Q+A • Section 4.4, 4.5, ES 2 • Professor Kari Lock Morgan • Duke University

2. To Do • Make one double-sided page of notes for in-class exam • WORK PRACTICE PROBLEMS! • (old exams and solutions to review questions under Documents on course website) • Read sections corresponding to anything you are still confused about • Practice using technology to summarize, visualize, and perform inference on data

3. Office Hours this Week • Today, 3 – 4 pm (Prof Morgan) • Today, 4 – 6 pm (Christine) • Tuesday, 3 – 6 pm (Prof Morgan) • Tuesday, 6 – 8 pm (Yue) • Tuesday, 8 – 9 pm (Michael) • (My office hours this week have been moved to Monday and Tuesday to answer questions before the exam)

4. Hypothesis Testing Define the parameter(s) of interest State your hypotheses Set significance level,  (usually 0.05 if unspecified) (Collect your data) Plot your data Calculate the observed sample statistic Create a randomization distribution Calculate the p-value Assess the strength of evidence against H0 Make a formal decision based on the significance level Interpret the conclusion in context

5. Exercise and Gender • Among college students, does one gender exercise more than the other?

6. Exercise and Gender

7. Exercise and Gender www.lock5stat.com/statkey Think: Results this extreme would happen about 22% of the time just by random chance if H0 were true, so this study does not provide adequate evidence against H0 Conclusion:  This study does not provide evidence that there is any association between gender and exercise times among college students  Little evidence against H0 p-value = 0.218  Do not reject H0

8. Randomization Distribution • A randomization distribution is the distribution of statistics that would be observed, just by random chance, if the null hypothesis were true • Simulate randomizations assuming the null hypothesis is true • Calculate the statistic for each simulated randomization

9. Randomized Experiments • In a randomized experiment the “randomness” is the random allocation of cases to treatment groups • If the null hypothesis is true, it doesn’t make any difference which treatment group you get placed in • Simulate randomizations assuming H0 is true by reallocate units to treatment groups, and keeping the response values the same

10. Observational Studies • In observational studies, there is no random allocation to treatment groups • In observational studies, what does “by random chance” even mean? What is random??? • How could we generate a randomization distribution for observational studies?

11. Bootstrapping • When data is collected by random sampling, without random allocation between groups, we can bootstrap to see what would happen by random chance • Bootstrapping (resampling with replacement) simulates the distribution of the sample statistic that we would observe when taking many random samples of the population

12. Bootstrapping • For a randomization distribution however, we need to know the distribution of the sample statistic, when the null hypothesis is true • How could we bootstrap assuming the null hypothesis is true? • Add/subtract values to each unit first to make the null hypothesis true (“shift the distribution”)

13. Reallocating versus Resampling • In both cases, we need to make the null hypothesis true for a randomization distribution

14. Exercise by Gender • Was the exercise by gender data collected via a randomized experiment? • Yes • No • There is no way to tell

15. Exercise by Gender • The randomness is not who is which gender (as with randomized experiments), but who is selected to be a part of the study • Male sample mean: 12.4 hours • Female sample mean: 9.4 hours • Add 3 hours to all the females, and then resample using bootstrapping • www.lock5stat.com/statkey

16. Method of Randomization • Reallocating and resampling usually give similar answers in terms of a p-value • For this class, it is fine to just use reallocating for tests, even if it is not actually a randomized experiment • The point is to understand the reason for generating a randomization distribution

17. Body Temperatures • Let’s return to the body temperature data • Using bootstrapping, we found a 95% confidence interval for the mean body temperature to be (98.05, 98.47 ) • Let’s do a hypothesis test to see how much evidence this data provides against  = 98.6 • H0 :  = 98.6 • Ha :  ≠ 98.6

18. Body Temperatures • How would we create a randomization distribution? • The sample mean is 98.26. Add 0.34 to each unit so we can sample with replacement mimicking sampling from a population with mean 98.6 • Take many bootstrap samples to create a randomization distribution

19. Randomization Distribution p-value = 0.002

20. Two Distributions

21. Intervals and Tests • If a (1 – α)% confidence interval does not contain the value of the null hypothesis, then a two-sided hypothesis test will reject the null hypothesis using significance level α • Intervals provide a range of plausible values for the population parameter, tests are designed to assess evidence against a null hypothesis

22. Body Temperatures • Using bootstrapping, we found a 95% confidence interval for the mean body temperature to be (98.05, 98.47 ) • H0 :  = 98.6 • Ha :  ≠ 98.6 • At α = 0.05, we would reject H0

23. REVIEW

24. Sample • The Big Picture Population Sampling Statistical Inference Exploratory Data Analysis

25. Data Collection Was the explanatory variable randomly assigned? Was the sample randomly selected? Yes No Yes No Possible to generalize to the population Should not generalize to the population Can not make conclusions about causality Possible to make conclusions about causality

27. Descriptive Statistics Think of a topic or question you would like to use data to help you answer. • What would the cases be? • What would the variables be? (Limit to one or two variables)

28. Descriptive Statistics How would you visualize and summarize the variable or relationship between variables? • bar chart/pie chart, proportions, frequency table/relative frequency table • dotplot/histogram/boxplot, mean/median, sd/range/IQR, five number summary • side-by-side or segmented bar plots/mosaic plots, difference in proportions, two-way table • side-by-side boxplot, stats by group • scatterplot, correlation

29. Statistic vs Parameter • A sample statisticis a number computed from sample data. • A population parameter is a number that describes some aspect of a population

30. Sampling Distribution • A sampling distributionis the distribution of statistics computed for different samples of the same size taken from the same population • The spread of the sampling distribution helps us to assess the uncertainty in the sample statistic • In real life, we rarely get to see the sampling distribution – we usually only have one sample

31. Bootstrap • A bootstrap sample is a random sample taken with replacement from the original sample, of the same size as the original sample • A bootstrap statisticis the statistic computed on the bootstrap sample • A bootstrap distributionis the distribution of many bootstrap statistics

32. BootstrapSample Bootstrap Statistic BootstrapSample Bootstrap Statistic Original Sample Bootstrap Distribution . . . . . . Sample Statistic BootstrapSample Bootstrap Statistic

33. Confidence Interval • A confidence intervalfor a parameter is an interval computed from sample data by a method that will capture the parameter for a specified proportion of all samples • A 95% confidence interval will contain the true parameter for 95% of all samples

34. Standard Error • The standard error (SE) is the standard deviation of the sample statistic • The SE can be estimated by the standard deviation of the bootstrap distribution • For symmetric, bell-shaped distributions, a 95% confidence interval is

35. Percentile Method • If the bootstrap distribution is approximately symmetric, a P% confidence interval can be gotten by taking the middle P% of a bootstrap distribution

36. Bootstrap Distribution

37. Hypothesis Testing • How unusual would it be to get results as extreme (or more extreme) than those observed, if the null hypothesis is true? • If it would be very unusual, then the null hypothesis is probably not true! • If it would not be very unusual, then there is not evidence against the null hypothesis

38. p-value • The p-value is the probability of getting a statistic as extreme (or more extreme) as that observed, just by random chance, if the null hypothesis is true • The p-value measures evidence against the null hypothesis

39. Hypothesis Testing

40. Randomization Distribution • A randomization distribution is the distribution of sample statistics we would observe, just by random chance, if the null hypothesis were true • The p-value is calculated by finding the proportion of statistics in the randomization distribution that fall beyond the observed statistic

41. Statistical Conclusions Strength of evidence against H0: Formal decision of hypothesis test, based on  = 0.05 :

42. Formal Decisions For a given significance level, , p-value <   Reject Ho p-value >  Do not Reject Ho

43. Errors Decision  TYPE I ERROR Truth  TYPE II ERROR

44. QUESTIONS???