Experiments. Pre and Post condition. Classic experimental design. Random assignment to control and treatment conditions Why random assignment and control groups?. Classic experimental design. Random assignment helps with internal validity Some threats to internal validity:
Pre and Post condition
t2 = F
R X O
R O X O
R O O
If these two groups are different, pretest sensitization is an issue.
Pre X Treatment interaction
If these two groups are different, there is a testing effect in general.
Reliability = Truevar/ Obsvar = 1 – Errorvar/ Obsvar
If observed variance goes up, power will decrease
However if observed variance goes up, we don’t know automatically what happens to reliability
Obsvar = Truevar + Errorvar
If it is error variance that is causing the increase in observed variance, reliability will decrease*
Reliability goes down, Power goes down
If it is true variance that is causing the increase in observed variance, reliability will increase
Reliability goes up, Power goes down
The point is that psychometric properties of the variables play an important, and not altogether obvious role in how we will interpret results, and not having a reliable measure is a recipe for disaster.