Create Presentation
Download Presentation

Download Presentation
## Multiple comparisons

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Multiple comparisons**- multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions**Multiple tests**Problem: Because we examine the same data in multiple comparisons, the result of the first comparison affects our expectation of the next comparison.**Multiple tests**ANOVA shows at least one different, but which one(s)? • T-tests of all pairwise combinations significant significant Not significant**T-test: <5% chance that this difference was a fluke…**affects likelihood of finding a difference in this pair! Multiple tests**Multiple tests**Solution: Make alpha your overall “experiment-wise” error rate T-test: <5% chance that this difference was a fluke… affects likelihood (alpha) of finding a difference in this pair!**Alpha / 3 = 0.0167**Alpha / 3 = 0.0167 Alpha / 3 = 0.0167 Multiple tests Solution: Make alpha your overall “experiment-wise” error rate e.g. simple Bonferroni: Divide alpha by number of tests**Orthogonal contrasts**Orthogonal = perpendicular = independent Contrast = comparison Example. We compare the growth of three types of plants: Legumes, graminoids, and asters. These 2 contrasts are orthogonal: 1. Legumes vs. non-legumes (graminoids, asters) 2. Graminoids vs. asters**Trick for determining if contrasts are orthogonal:**1. In the first contrast, label all treatments in one group with “+” and all treatments in the other group with “-” (doesn’t matter which way round). Legumes Graminoids Asters + - -**Trick for determining if contrasts are orthogonal:**1. In the first contrast, label all treatments in one group with “+” and all treatments in the other group with “-” (doesn’t matter which way round). 2. In each group composed of t treatments, put the number 1/t as the coefficient. If treatment not in contrast, give it the value “0”. Legumes Graminoids Asters +1 - 1/2 -1/2**Trick for determining if contrasts are orthogonal:**1. In the first contrast, label all treatments in one group with “+” and all treatments in the other group with “-” (doesn’t matter which way round). 2. In each group composed of t treatments, put the number 1/t as the coefficient. If treatment not in contrast, give it the value “0”. 3. Repeat for all other contrasts. Legumes Graminoids Asters +1 - 1/2 -1/2 0 +1 -1**Legumes Graminoids Asters**+1 - 1/2 -1/2 0 +1 -1 0 - 1/2 +1/2 Sum of products = 0 Trick for determining if contrasts are orthogonal: 4. Multiply each column, then sum these products.**Legumes Graminoids Asters**+1 - 1/2 -1/2 0 +1 -1 0 - 1/2 +1/2 Sum of products = 0 Trick for determining if contrasts are orthogonal: 4. Multiply each column, then sum these products. 5. If this sum = 0 then the contrasts were orthogonal!**What about these contrasts?**1. Monocots (graminoids) vs. dicots (legumes and asters). 2. Legumes vs. non-legumes**Important!**You need to assess orthogonality in each pairwise combination of contrasts. So if 4 contrasts: Contrast 1 and 2, 1 and 3, 1 and 4, 2 and 3, 2 and 4, 3 and 4.**How do you program contrasts in JMP (etc.)?**Treatment SS } Contrast 1 } Contrast 2**F1,20 = (67)/1 = 6.7**10 From full model! How do you program contrasts in JMP (etc.)? Legumes vs. non-legumes Normal treatments “There was a significant treatment effect (F…). About 53% of the variation between treatments was due to differences between legumes and non-legumes (F1,20 = 6.7).” Legume 1 1 Legume 1 1 Graminoid 2 2 Graminoid 2 2 Aster 3 2 Aster 3 2 SStreat 122 67 Df treat 2 1 MStreat 60 MSerror 10 Df error 20**Even different statistical tests may not be independent !**Example. We examined effects of fertilizer on growth of dandelions in a pasture using an ANOVA. We then repeated the test for growth of grass in the same plots. Problem?**Multiple tests**b Convention: Treatments with a common letter are not significantly different a,b a significant Not significant Not significant