Research Methods I

1. Research Methods I ANOVA � Post hoc testing

2. Review on ANOVA Steps to calculating F Calculate SStotal Calculate SSbetween Calculate SSwithin Calculate dftotal , dfwithin , dfbetween Calculate MSbetween and MSwithin F = MSbetween / MSwithin

3. Review on ANOVA MSwithin = SSwithin / dfwithin dfbetween = (A) � 1 - number of groups - 1 dfwithin = A (N-1) or A (S-1) Number of subjects per group - 1 times number of groups dftotal = A x S � 1 or dfbetween + dfwithin

4. Comparisons F-test gives a global effect of the independent variable on the dependent variable (omnibus test) Researchers may want to know more precise conclusions They will use focused comparisons Two kinds of comparisons: Planned (a priori) comparisons Selected before running the experiment Normally correspond to research hypothesis Post hoc (a posteriori) comparisons Decided upon after the experiment Used if three or more means were compared

5. Post hoc testing - general When we reject the null hypothesis in ANOVA we conclude that there is at least one pair of samples are different from one another to negate H0. Why not do a series of t-test to find the right pair? T-test is based on the assumption that only a single comparison of means was to be made. If more than one comparison is made the chance of type I error increases. Type I error = rejecting a true null hypothesis Experiment-wise alpha level: the overall probability of a Type I error that accumulates over a series of separate hypothesis tests. Typically, the experiment-wise alpha level is substantially greater than the value of alpha used for any one of the individual tests. If one compares three means � 3 pair wise comparisons are needed. Each has a 5% chance of Type I error. Type I error = rejecting a true null hypothesis Experiment-wise alpha level: the overall probability of a Type I error that accumulates over a series of separate hypothesis tests. Typically, the experiment-wise alpha level is substantially greater than the value of alpha used for any one of the individual tests. If one compares three means � 3 pair wise comparisons are needed. Each has a 5% chance of Type I error.

6. Post-hoc testing - general Several different post-hoc tests can be conducted: Scheffe Evaluate all possible contrasts Student-Newman-Keul�s Multiple Range Test Duncan�s Multiple Range test Tukey�s Honestly Significant Difference Test Pairwise comparisons

7. Scheffe Test Is considered to be a flexible and robust test Can be used if groups have different sizes Less sensitive to departures from normality assumption Less sensitive to departures from assumptions concerning equal variances in the population Does not have as much power (Tukey and Duncan for example have more) It is easier to reject the null hypothesis with more powerful tests. It is a very safe test (little chance of type I error)

8. Scheffe Test Uses F-ratio to test for a significant difference between any two treatment groups (�1 = �2) FA versus B = SSbetween = MSbetween = Numerator of F-ratio is an MS between treatment that is calculated using only the two treatments you want to compare. The denominator is the same MS within treatments that was used for the overall ANOVA. Numerator of F-ratio is an MS between treatment that is calculated using only the two treatments you want to compare. The denominator is the same MS within treatments that was used for the overall ANOVA.

9. Tukey�s Test Calculate the honestly significant difference: A single value that determines the minimum difference between treatment means that is necessary for significance. HSD = q-value can be found on page 696 MSwithin is the within treatments variance from the ANOVA n = the number of scores per treatment group (must be the same for all treatment groups)