Experimentation

1. Experimentation

2. What experiments are not: • Two commonsense uses of “experiment”: • any study • to “try something out” • …both of these understandings are flawed.

3. What Experiment are: • Experiments test causal relationships. • A causal relationship has three elements: • Stable temporal ordering • An association between the phenomena studied • No alternative explanations for that association. • Among all the methods available to social researchers, onlyexperimentscan verify all three of these elements that define a causal relationship.

4. “Real” experiments have three necessary conditions: • A hypothesis; • The modification of something in a specific situation; • The comparison of outcomes with and without the modification.

5. Experimentation requires that “subjects” (“Ss”) be randomly assigned to “treatment” and “control” groups. • The meaning of “random assignment” • The purpose of random assignment • Attainment of comparable groups • Even distribution of bias • Methods of random assignment • Random process, like coin flips • Random number generator • Computer-based assignment

6. Experiment designs vary, but must have 7 essential features to be true experiments: • Random assignment • A “treatment” or “independent” variable • A “dependent” variable • Pretesting • Post-testing • An “experimental” or “treatment” group • A “control” group

7. Some examples of experimental and pseudo-experimental design • “Pre-experimental designs” (shortcuts) • Case study (no comparison, no pretest) • Static group comparison (no random assignment, no pretest) • Classical Experimental Design • With dichotomous independent variable • With continuous independent variable • With more than one independent variable • Quasi-experimental Designs • Two-group, post-test only with random assignment • Time series designs with one group or one subject • Latin square • Solomon n-group • Field experiments

8. Issues regarding internal and external validity • Internal Validity • Selection Bias • History Effects • Maturation • Testing Effects • Instrument Reliability • Contamination • Experimenter Expectancy • External Validity • Realism (is setting or task “real”?) • experimental realism • mundane realism • Reactivity (are Ss only reacting to experiment?) • The Hawthorne effect • response to demand characteristics • the placebo effect