ANOVA

1. STAT 101 Dr. Kari Lock Morgan 11/1/12 ANOVA • SECTION 8.1 • Testing for a difference in means across multiple categories

2. What Next? • If you have enjoyed learning how to analyze data, and want to learn more: • take STAT 210 (Regression Analysis) • Applied, focused on data analysis • Recommended for any major involving data analysis • Only prerequisite is STAT 101 • If you like math and want to learn more of the mathematical theory behind what we’ve learned: • take STAT 230 (Probability) • and then STAT 250 (Mathematical Statistics) • Prerequisite: multivariable calculus

3. Two Options for p-values • We have learned two ways of calculating p-values: • The only difference is how to create a distribution of the statistic, assuming the null is true: • Simulation (Randomization Test): • Directly simulate what would happen, just by random chance, if the null were true • Formulas and Theoretical Distributions: • Use a formula to create a test statistic for which we know the theoretical distribution when the null is true, if sample sizes are large enough

4. Two Options for Intervals • We have learned two ways of calculating intervals: • Simulation (Bootstrap): • Assess the variability in the statistic by creating many bootstrap statistics • Formulas and Theoretical Distributions: • Use a formula to calculate the standard error of the statistic, and use the normal or t-distribution to find z* or t*, if sample sizes are large enough

5. Inference Which way did you prefer to learn inference? • Simulation methods • Formulas and theoretical distributions

6. Inference Which way gave you a better conceptual understanding of confidence intervals and p-values? • Simulation methods • Formulas and theoretical distributions

7. Inference Which way do you prefer to do inference? • Simulation methods • Formulas and theoretical distributions

8. Pros and Cons • Simulation Methods • PROS: • Methods tied directly to concepts, emphasizing conceptual understanding • Same procedure for every statistic • No formulas or theoretical distributions to learn and distinguish between • Works for any sample size • Minimal math needed • CONS: • Need entire dataset (if quantitative variables) • Need a computer • Newer approach, so different from the way most people do statistics

9. Pros and Cons • Formulas and Theoretical Distributions • PROS: • Only need summary statistics • Only need a calculator • The approach most people take • CONS: • Plugging numbers into formulas does little for conceptual understanding • Many different formulas and distributions to learn and distinguish between • Harder to see the big picture when the details are different for each statistic • Doesn’t work for small sample sizes • Requires more math and background knowledge

10. Two Options • If the sample size is small, you have to use simulation methods • If the sample size is large, you can use whichever method you prefer • It is redundant to use both methods, unless you want to check your answers

11. Accuracy • The accuracy of simulation methods depends on the number of simulations (more simulations = more accurate) • The accuracy of formulas and theoretical distributions depends on the sample size (larger sample size = more accurate) • If the sample size is large and you have generated many simulations, the two methods should give essentially the same answer

12. Multiple Categories • So far, we’ve learned how to do inference for a difference in means IF the categorical variable has only two categories • Today, we’ll learn how to do hypothesis tests for a difference in means across multiple categories

13. Hypothesis Testing • State Hypotheses • Calculate a statistic, based on your sample data • Create a distribution of this statistic, as it would be observed if the null hypothesis were true • Measure how extreme your test statistic from (2) is, as compared to the distribution generated in (3) test statistic

14. Cuckoo Birds • Cuckoo birds lay their eggs in the nests of other birds • When the cuckoo baby hatches, it kicks out all the original eggs/babies • If the cuckoo is lucky, the mother will raise the cuckoo as if it were her own • Do cuckoo birds found in nests of different species differ in size? http://opinionator.blogs.nytimes.com/2010/06/01/cuckoo-cuckoo/

15. Length of Cuckoo Eggs

16. Notation • k = number of groups • nj = number of units in group j • n = overall number of units • = n1 + n2 + … + nk

17. Cuckoo Eggs • k = 5 • n1 = 15, n2 = 60, n3 = 16, n4 = 14, n5 = 15 • n = 120

18. Hypotheses • To test for a difference in means across k groups:

19. Test Statistic Why can’t use the familiar formula to get the test statistic? • More than one sample statistic • More than one null value • We need something a bit more complicated…

20. Difference in Means Whether or not two means are significantly different depends on • How far apart the means are • How much variability there is within each group

21. Difference in Means

22. Analysis of Variance • Analysis of Variance (ANOVA) compares the variability between groupsto the variability within groups Total Variability Variability Between Groups Variability Within Groups

23. Analysis of Variance • If the groups are actually different, then • the variability between groups should be higher than the variability within groups • the variability within groups should be higher than the variability between groups If the groups are different, there will be high variability between the groups.

24. Discoveries for Today • How to measure variability between groups? • How to measure variability within groups? • How to compare the two measures? • How to determine significance?

25. Discoveries for Today • How to measure variability between groups? • How to measure variability within groups? • How to compare the two measures? • How to determine significance?

26. Sums of Squares • We will measure variability as sums of squared deviations (aka sums of squares) • familiar?

27. Sums of Squares Total Variability Variability Between Groups Variability Within Groups data value i overall mean mean in group j overall mean ithdata value in group j mean in group j Sum over all data values Sum over all groups Sum over all data values

28. Deviations Group 1 Group 1 Mean Group 2 Overall Mean

29. Sums of Squares Total Variability Variability Between Groups Variability Within Groups SST (Total sum of squares) SSG (sum of squares due to groups) SSE (“Error” sum of squares)

30. Cuckoo Birds

31. ANOVA Table The “mean square” is the sum of squares divided by the degrees of freedom average variability variability

32. ANOVA Table • Fill in the beginnings of the ANOVA table based on the Cuckoo birds data. SSG = 35.9 SSE = 101.20

33. ANOVA Table • Fill in the beginnings of the ANOVA table based on the Cuckoo birds data.

34. Discoveries for Today • How to measure variability between groups? • How to measure variability within groups? • How to compare the two measures? • How to determine significance?

35. F-Statistic • The F-statisticis a ratio of the average variability between groups to the average variability within groups

36. ANOVA Table

37. Cuckoo Eggs

38. F-statistic • If there really is a difference between the groups, we would expect the F-statistic to be • Higher than we would observe by random chance • Lower than we would observe by random chance The numerator of the F-statistic measures between group variability, and the denominator measures within group. If there is a difference, we expect the between group variability to be higher. • If the null hypothesis is true, what kind of F-statistics would we observe just by random chance?

39. Discoveries for Today • How to measure variability between groups? • How to measure variability within groups? • How to compare the two measures? • How to determine significance?

40. How to determine significance? • We have a test statistic. What else do we need to perform the hypothesis test? • A distribution of the test statistic assuming H0 is true • How do we get this? Two options: • Simulation • Distributional Theory

41. Simulation • www.lock5stat.com/statkey Because a difference would make the F-statistic higher, calculate proportion in the upper tail An F-statistic this large would be very unlikely to happen just by random chance if the means were all equal, so we have strong evidence that the mean lengths of cuckoo birds in nests of different species are not all equal.

42. F-distribution F-distribution

43. F-Distribution • If the following conditions hold, • Sample sizes in each group are large (each nj≥ 30) OR the data are relatively normally distributed • Variability is similar in all groups • The null hypothesis is true • then the F-statistic follows an F-distribution • The F-distribution has two degrees of freedom, one for the numerator of the ratio (k – 1) and one for the denominator (n – k)

44. Equal Variance • The F-distribution assumes equal within group variability for each group • As a rough rule of thumb, this assumption is violated if the standard deviation of one group is more than double the standard deviation of another group

45. F-distribution • Can we use the F-distribution to calculate the p-value for the Cuckoo bird eggs? • Yes • No • Need more • information The equal variability condition is satisfied, but the sample sizes are small so we can only use the F-distribution if the data is normal.

46. Length of Cuckoo Eggs

47. F-distribution p-values • StatKey – simulation or theoretical • RStudio: • tail.p(“f”,stat,df1,df2,tail=“upper”) • TI-83: • 2nd DISTR  7:Fcdf(  • lower bound, upper bound, df1, df2 • For F-statistics, the p-value (the area as extreme or more extreme) is always the upper tail

48. ANOVA Table

49. Cuckoo Eggs

50. ANOVA Table • Equal variability • Normal(ish) data We have very strong evidence that average length of cuckoo eggs differs for nests of different species