1 / 38

Hypothesis Testing III 2/15/12

Hypothesis Testing III 2/15/12. Statistical significance Errors Power Significance and sample size. Section 4.3. Professor Kari Lock Morgan Duke University. To Do. P roject 1 Proposal (due today, 5pm) Homework 4 (due Monday)

loyal
Download Presentation

Hypothesis Testing III 2/15/12

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Hypothesis Testing III • 2/15/12 • Statistical significance • Errors • Power • Significance and sample size • Section 4.3 • Professor Kari Lock Morgan • Duke University

  2. To Do • Project 1 Proposal(due today, 5pm) • Homework 4 (due Monday) • If you turn in your HW4 by 5pm on Friday (either slide it under the door of Old Chem 216 or email it to your TA), it will be graded by class on Monday • NO LATE HOMEWORK ACCEPTED! • Study/prepare for Exam 1!

  3. Exam 1 • Exam 1: • In-class portion: Wednesday, 2/22 • Lab portion: Thursday, 2/23 • In-class portion: (75%) • Open only to a calculator and one double sided page of notes prepared by you • Emphasis on conceptual understanding • Lab portion: (25%) • Open to everything except communication of any form with other humans • Emphasis on actually analyzing data

  4. Practice • Last year’s in-class and lab midterms, with solutions, are available on the course website (under documents) • Full solutions to ALL the essential synthesis and review problems from Units 1 and 2 are available on the course website • Doing problems is the key to success!!!

  5. Keys to In-Class Exam Success • Work lots of practice problems! • Take last year’s exams under realistic conditions (time yourself, do it all before looking at the solutions, etc.) • Prepare a good cheat sheet and use it when working problems • Read the corresponding sections in the book if there are concepts you are still confused about

  6. Keys to Lab Exam Success • Primarily, make sure you know how to summarize, visualize, create an interval, and conduct a test for any one variable or relationship between two variables. • Beyond that, make sure you are comfortable with the content from the labs • Open-book does NOT mean you don’t have to study. You will not have time to look up every command you need during the exam.

  7. Office Hours before Exam • You have LOTS of opportunities for help! • Today, 3 – 5pm (Prof Morgan) • Friday, 1 – 3 pm (Prof Morgan • Sunday, 5 – 7 pm (Jessica) • Sunday, 7 – 9 pm (Michael) • Monday, 3 – 4 pm (Prof Morgan) • Monday, 4 – 6 pm (Christine) • Tuesday, 3 – 6 pm (Prof Morgan) • Tuesday, 6 – 8 pm (Yue) • (My office hours next week have been moved to Monday and Tuesday to answer questions before the exam)

  8. Statistical Conclusions Strength of evidence against H0: Formal decision of hypothesis test, based on  = 0.05 :

  9. Red Wine and Weight Loss • Resveratrol, an ingredient in red wine and grapes, has been shown to promote weight loss in rodents, and has recently been investigated in primates (specifically, the Grey Mouse Lemur). • A sample of lemurs had various measurements taken before and after receiving resveratrol supplementation for 4 weeks BioMed Central (2010, June 22). “Lemurs lose weight with ‘life-extending’ supplement resveratrol. Science Daily.

  10. Red Wine and Weight Loss In the test to see if the mean resting metabolic rate is higher after treatment, the p-value is 0.013. Using  = 0.05, is this difference statistically significant? (should we reject H0: no difference?) (a) Yes (b) No

  11. Red Wine and Weight Loss In the test to see if the mean body mass is lower after treatment, the p-value is 0.007. Using  = 0.05, is this difference statistically significant? (should we reject H0: no difference?) (a) Yes (b) No

  12. Red Wine and Weight Loss In the test to see if locomotor activity changes after treatment, the p-value is 0.980. Using  = 0.05, is this difference statistically significant? (should we reject H0: no difference?) (a) Yes (b) No

  13. Red Wine and Weight Loss In the test to see if mean food intake changes after treatment, the p-value is 0.035. Using  = 0.05, is this difference statistically significant? (should we reject H0: no difference?) (a) Yes (b) No

  14. Formal Decisions Suppose many researchers around the world are all investigating the same question of interest. If the null hypothesis is true, using  = 0.05, what proportion of hypothesis tests will incorrectly reject the null? None 95% 5% It depends

  15. Errors • There are four possibilities: Decision  TYPE I ERROR Truth  TYPE II ERROR

  16. Red Wine and Weight Loss • In the test to see if resveratrol is associated with food intake, the p-value is 0.035. • If resveratrolis not associated with food intake, a Type I Error would have been made • In the test to see if resveratrol is associated with locomotor activity, the p-value is 0.980. • If resveratrolis associated with locomotor activity, a Type II Errorwould have been made

  17. Errors • Usually, we have no way of knowing whether an error has been made, without doing another study • Analogously, we have no way to knowing whether our confidence interval actually contains the true parameter

  18. Errors If the null hypothesis is true, what is the probability of making a Type I error? 0  1 –  It depends

  19. Significance Level • Why would you use a smaller significance level, like  = 0.01? • Why would you use a larger significance level, like  = 0.10?

  20. Analogy to Law Ho Ha A person is innocent until proven guilty. Evidence must be beyond the shadow of a doubt.  Types of mistakes in a verdict? Convict an innocent Type I error Release a guilty Type II error

  21. Errors If the alternative hypothesis is true, what is the probability of making a Type II error? 0  1 –  It depends

  22. Power • The power of a hypothesis test is the probability of correctly detecting a significant effect, is there is one (correctly rejecting the null hypothesis when it is false)

  23. Errors • There are four possibilities: Decision  TYPE I ERROR Truth  TYPE II ERROR

  24. Power What factors influence the power of a test? Sample size True value or effect size Variability of values (standard deviation)

  25. Power If you want to increase the power of your test, what can you do? Increase the sample size Make the true value farther from the null value Decrease the standard deviation of your variables Any of the above

  26. Significance and Sample Size www.lock5stat.com/statkey

  27. Significance and Sample Size • The smaller the sample size, the • (a) smaller • (b) larger • the chance of a Type II error (failing to reject the null hypothesis, even when it is false).

  28. Significance and Sample Size • Just because you fail to get significant results, does NOT mean the null hypothesis is true • This is particularly true for small sample sizes. Unless the truth is very far from the null value, it is hard to find significant results if the sample size is small. • With small sample sizes, Type II Errors are very likely!

  29. Statistical vs Practical Significance • With small sample sizes, even large differences or effects may not be significant • With large sample sizes, even a very small difference or effect can be significant • A statistically significant result is not always practically significant, especially with large sample sizes

  30. Statistical vs Practical Significance • Example: Suppose a weight loss program recruits 10,000 people for a randomized experiment. • A difference in average weight loss of only 0.5 lbs could be found to be statistically significant • Suppose the experiment lasted for a year. Is a loss of ½ a pound practically significant?

  31. Videogames and GPA If you get put with a roommate who brings a videogame to college, does that lower your GPA? What are the null and alternative hypotheses? • H0: pv – pnv= 0, Ha: pv – pnv< 0 • H0: µv – µnv= 0, Ha: µv – µnv< 0 • H0: pv – pnv< 0, Ha: pv – pnv= 0 • H0: µv – µnv< 0, Ha: µv – µnv= 0

  32. Videogames and GPA If you get put with a roommate who brings a videogame to college, does that lower your GPA? A study at Berea college conducted this test and obtained a p-value of 0.036. What does this mean? • The probability that H0 is true is 0.036 • The probability that H0 is false is 0.036 • The probability of seeing a difference in mean GPA as extreme as that in the sample is 0.036 • If H0 is true, the probability of seeing a difference in mean GPA as extreme as that in the sample is 0.036 Source: Stinebrickner, R. and Stinebrickner, T.R. (2008). “The Causal Effect of Studying on Academic Performance,” The B.E. Journal of Economic Analysis & Policy: Vol. 8: Iss. 1 (Frontiers), Article 14.

  33. Videogames and GPA • In the study about roommates bringing videogames to college and GPA, the p-value is 0.036. Using  = 0.05, what would you conclude? • People assigned roommates who bring videogames have significantly lower GPAs • People assigned roommates who bring videogames do not have lower GPAs • Nothing

  34. Videogames and GPA • Based on this p-value, can you conclude that getting put with a roommate who brings a videogame to campus causes you to have a lower GPA, on average? • Yes • No

  35. Significance and Causation • The p-value alone tells you whether there is a significant association between two variables, but NOT whether this is a causal association • The data collection method tells you whether causal conclusions are possible, but not whether an association is significant • If the study is a randomized experiment AND the p–value indicates statistically significant results, only then you can conclude that the explanatory variable has a causal effect on the response variable

  36. Videogames and GPA • Roommates are assigned randomly at Berea college. Based on this knowledge and the p-value (0.036), can you conclude that getting put with a roommate who brings a videogame to campus causes you to have a lower GPA, on average? • Yes • No

  37. Videogames and GPA • The study also tested whether students who bring a videogame to college themselves have lower GPAs, on average. The p-value of this test is 0.068. Using  = 0.05, what would you conclude? • People who bring videogames have significantly lower GPAs • People who bring videogames do not have lower GPAs • In this study, the difference in average GPA between students who bring videogames and those who don’t is not statistically significant • Nothing • Either (c) or (d)

  38. Summary • In making formal decisions, reject H0 if the p-value is less than α, otherwise do not reject H0 • There are two types of errors that can be made in hypothesis testing: rejecting a true null or failing to reject a false null • The larger your sample size, the higher your chance of finding a significant result, if one exists

More Related