BMS 617

1. BMS 617 Lecture 13: One-way ANOVA Marshall University Genomics Core Facility

2. GRHL2 Gene Expression (again) • Revisit our GRHL2 expression experiment • Compared the expression of GRHL2 in three different types of cell line • Basal A, Basal B, Luminal • Previously compared just one group against another, using a t-test • Really want to compare all three groups Marshall University School of Medicine

3. GRHL2 Expression Marshall University School of Medicine

4. What’s wrong with t-tests? • Many investigator’s instinct here is to use t-tests: • Could use three t-tests: • Basal A vs Basal B • Basal A vs Luminal • Basal B vs Luminal • But this becomes a multiple-hypothesis testing problem • If we assume the null hypothesis: the expression is the same in all three cell types, the chances of seeing data that produce one or more p-values less than 0.05 are 14.3%. • Problem increases dramatically as the number of groups increases • 40.1% for 5 groups, 76.2% for 8 groups Marshall University School of Medicine

5. One-Way ANOVA • The correct way to analyze these data is with one-way ANOVA • ANOVA is an abbreviation for “Analysis of Variance” • This just describes how the technique works • We are still comparing the means of the groups • Not comparing the variance Marshall University School of Medicine

6. How ANOVA works • Can think of ANOVA as a comparison of two models • Similarly to the way we described linear regression • The null hypothesis model • All groups have the same mean • Yi = μ + εi • The alternative model • Each group has a different mean • Yi,j = μj + εi,j • μjis the mean for group j • We quantify how well the data fit the model by computing the sum of squares of the residuals (deviations from the model) for each model • For the null hypothesis, sum of squares of deviations from overall mean • For the alternative hypothesis, sum of squares of deviations from the group means Marshall University School of Medicine

7. ANOVA for GRHL2 data as a model comparison • R2=0.699 • Same interpretation as before • The proportion of variation which is “accounted for” by the model • In the context of an ANOVA, this is usually called η2 • Note that any grouping will reduce the sum of squares • So we need to do something more sophisticated than just seeing if R2 increases to know if this is a “good” model • Must account for the sample size and the number of parameters in the model Marshall University School of Medicine

8. Usual presentation of ANOVA Marshall University School of Medicine

9. Interpreting the ANOVA table • In this presentation, the first row is the “improvement from the model” • This is the “total sum of squares” minus the “model sum of squares” • Total sum of squares is the sum of squares of differences between points and the overall mean • Model sum of squares is the sum of squares of differences between points and their group mean • The difference turns out to be the sum of squares of the differences between the group means and the overall mean, weighted by the number in each group Marshall University School of Medicine

10. Data plot(again) Marshall University School of Medicine

11. ANOVA as partition of sum of squares • This is the traditional view of ANOVA • Partitioning the total sum of squares (variance) into • The within-groups sum of squares • The between-groups sum of squares • If the between-groups sum of squares is “large” compared to the within-groups sum of squares, we conclude the groups do not have the same mean • In order to make this precise we have to compute the degrees of freedom Marshall University School of Medicine

12. Degrees of Freedom • Easiest way to think of degrees of freedom is via models • Number of data points, subtract number of parameters in the model • In our example, we have 51 data points • Null hypothesis model (total sum of squares) has one parameter (the mean) • So total sum of squares has 50 degrees of freedom • Alternative model (within groups sum of squares) has 3 parameters (mean for each group) • So within-groups sum of squares has 48 degrees of freedom • Between groups is the difference in sum of squares • Degrees of freedom is the difference in degrees of freedom in the two models • In our case, 50-48=2 degrees of freedom. Marshall University School of Medicine

13. Mean squares and F ratio • As with model comparison, the mean squares is the sum of squares divided by the degrees of freedom • The F ratio is the between-groups mean squares divided by the within-groups mean squares • Distribution of F is known for each pair of d.f. • So p-value can be computed Marshall University School of Medicine

14. Post-Hoc tests • The ANOVA analysis showed a small p-value (p=3.17 x 10-13), so we have strong evidence to reject the null hypothesis • I.e. we are justified in believing the means for each group are not all the same • However, this doesn’t tell us which groups are different to which other groups • This is usually what you want to know • “Post-hoc” tests can be used to determine this Marshall University School of Medicine

15. Fisher’s Least Significant Difference • Fisher’s Least Significant Difference was the first Post-Hoc test developed for ANOVA • Has really been superseded by more sophisticated tests • Fisher’s Least Significant Difference can only be performed after an ANOVA gives a significant result • Why it is called a “post-hoc” test • Use the same terminology for more modern tests • Though most of these actually make sense even without performing an ANOVA Marshall University School of Medicine

16. Tukey’s Honest Significant Differences • Tukey’s HSD compares every group to every other group • Works by computing the largest t-ratio of all possible t-tests, under the assumption of the null hypothesis • Tukey’s HSD associates confidence intervals with each pairwise comparison • these are family-wise confidence intervals Marshall University School of Medicine

17. Results of Tukey HSD for GRHL2 data Marshall University School of Medicine

18. Interpreting the CIs • The confidence intervals are family-wise 95% confidence intervals • We are 95% confident that all the intervals shown contain the true difference in means in the population • Only makes sense to show all of these (not a subset of them) • If the 95% confidence interval contains zero, the difference is not statistically significant at a significance level of 0.05 • If the 95% confidence interval does not contain zero, the difference is statistically significant at a significance level of 0.05 • Since these are family-wise measures, it only makes sense to think of the whole family at once • Think of this as dividing the comparisons into two groups • Those that are statistically significantly difference, and those that are not • Cannot talk about statistical significance of one comparison in isolation Marshall University School of Medicine

19. Dunnett’s Test • Dunnett’s test is another post-hoc test for one-way ANOVA • Instead of comparing every group to every other group, it compares every group to a single control group • Must decide before the experiment which is to be the control group (and why) • Since there are fewer comparisons than Tukey’s test, Dunnett’s test has more statistical power Marshall University School of Medicine

20. Other post-hoc tests • There are other approaches to testing between groups in a one-way ANOVA • Again, must consider multiple hypotheses when doing these kinds of tests • Specialized post-hoc tests account for this • Most naïve approach: • Compare with t-tests and use Bonferroni correction • Flexible – can compare arbitrary groups • But not statistically powerful Marshall University School of Medicine

21. Scheffe’s Test • Scheffe’s test is the most flexible post-hoc test for ANOVA • Can be used with an arbitrary set of comparisons between groups • Called “contrasts” • More powerful than Bonferroni • Less powerful than Tukey or Dunnett’s in cases where those are applicable • Decision as to which post-hoc test should be used should be made at experimental design time • Not based on the data Marshall University School of Medicine

22. Summary • One-way ANOVA is used to compare means across more than two groups • The generated p-value is the probability of seeing differences between the groups at least as big as those observed, assuming the groups are all sampled from populations with the same mean • Can think of this as a comparison of models • Post-hoc tests are usually used to determine which groups are difference • Most common are Tukey’s test (all comparisons) and Dunnett’s test (each group compared to control) Marshall University School of Medicine