Analyzing the Results of an Experiment…

1. Analyzing the Results of an Experiment… • -not straightforward.. • Why not?

2. Variability and Random/chance outcomes

3. Inferential Statistics • Statistical analysis appropriate for inferring causal relationships and effects. • Many different formulas…which one do you use?

4. Inferential Stat selection • -Determine that you are analyzing the results of an experimental manipulation, not a correlation • Identify the IV and DV. • The IV Will always be nominal on some level, even when it may seem to be continuous..low, medium and high doses of a drug

5. Inf. Stat Selection • What is the scale of the DV? • Scale of DV -Statistic to use

6. t-test or ANOVA? How many levels of the IV are there?

7. There are different forms of T-tests and ANOVA’s:Did the Study Use a Within Group or Between group Experimental Design?

8. In some ways all inferential Stats are similar. • They calculate the probability that a result was due to the IV as opposed to random variability… • Let’s focus on the Basic ANOVA since it is likely to be the statistic you may use most commonly.

9. ANOVA • ANOVA produces an F-value. • F values are the ratio of overall between group Variability to the Mean within group variability Between Var. (+ chance) /Mean within grp. Variability (+ chance) What does this mean?

10. Lets suppose: • Experiment- IV marijuana • Control • Placebo control • Low dose • High dose

11. Dependent Variable is: • Performance on a short term memory task measured number correct out of 10 test items. • 9 subjects in each group

12. Possible out come 1

13. Possible Outcome 1Control Placebo Low dose High dose • 4 2 2 2 • 5 3 3 3 • 6 4 4 5 • 5 6 4 3 • 5 5 5 4 • 6 5 4 4 • 4 4 5 4 • 3 4 6 6 • 7 3 3 5

14. Distribution of scores for control sample

15. Placebo scores

16. Low dose scores

17. High dose scores

18. The population distribution of scores

19. F value relatively low High low placebo control w/in grp. var Between grp. Var

20. Now consider this: Possible Outcome 2Control Placebo Low dose High dose • 4 2 2 2 • 5 3 3 3 • 6 4 4 5 • 5 6 4 3 • 5 5 5 4 • 6 5 4 4 • 4 4 5 4 • 3 4 6 6 • 7 3 3 5

21. Distribution of scores for control sample

22. Placebo scores

23. Low dose scores

24. High dose scores

25. F value relatively High High low placebo control w/in grp. var Between grp. Var

26. The high F value reflects • Logic! • Distribution of score are much more obviously separated, and in this case are completely non-overlapping • Low F values indicate highly overlapping score distributions

27. So how do we decide if an F value is large enough to consider the result as causal? • We consult a table of established probabilities of different F values, within the context of Degree of freedom terms:

30. ANOVA Significance table

33. Where is/are the difference (s)?

34. Inferential Statistics

35. The story of “Scratch”

36. Why not jus use repeated t-tests? Probability pyramiding • 15 t-tests required for this data set • Post-hocs include compensations for repeated testing of a large data set

37. After all this where so we stand?We can still be wrong.

40. Factors that affect “power.”Sample size

41. One vs two-tailed testing

43. Effect size