Group Comparison Research Causal-Comparative (Ex post facto) Research
Purpose of causal comparative research • Attempts to determine cause for • Existing conditions • Preexisting differences in groups • Alleged cause and effect have already occurred • Orientations • Retrospective (basic): starts with an effect and seeks possible causes • Prospective (variation): starts with a cause and investigates its effects on some variable
Causal-comparative (ex post facto) research • The independent variable (IV) is not manipulated; it has already occurred • Independent variables sometimes called “attribute variables” • Less costly and time-consuming to conduct • Establishing cause-effect relationships is more difficult than in experiments
Procedures in causal-comparative research • Identify an existing condition or event (e.g., differences in socialization among 1st grade students) • Look “backwards” to see what may have caused this difference/condition to occur (i.e., some attended preschool, some did not) • Rule out other causal factors
Sometimes confused with correlational research: • Both lack manipulation of variables • Both require caution in interpreting results • Both can support subsequent experimental research
Causal comparative Attempts to identify cause-effect relationships At least one independent variable Two or more groups Involves a comparison Correlational No attempt to understand cause and effect Two or more variables Only one group Causal comparative vs. correlational research
Sometimes confused with experimental research: • Both try to establish cause-effect relationships • Both can test hypotheses concerning the relationship between an independent (X) and a dependent variable (Y) • Both involve group comparisons
Causal comparative Individuals already in groups before study begins Independent variable has already occurred Independent variable is not manipulated Cannot be Should not be Could be, but is not Experiment Individuals randomly assigned to groups (e.g., treatment or control) Independent variable manipulated by the researcher Comparison to experiments
Examples of non-manipulated independent variables • Age • Sex • Ethnicity • “Learning style” • Socioeconomic status (SES) • Parent educational level • Family environment • Type of school attended
Design of causal-comparative research • Select 2 groups that differ on some IV • One group possesses a characteristic that the other does not • Each group possesses the characteristic, but in differing amounts • Randomly sample Ss from each group • Collect info on Ss to determine equality of the groups • Compare groups on the DV
Difficulty in interpreting findings • Establishing cause and effect requires caution! • Alternative explanations: • Different causal variable • Order of causation • Reverse causality • Order of occurrence
Evidence necessary to demonstrate that X causes Y: • Establish statistical relationship between X and Y (i.e., correlational research); • determine that X precedes Y in time (collect data over time, i.e., longitudinal research); • demonstrate that other, unknown factors did not determine the dependent variable (i.e., experimental research).
Becker & Gersten (1982): “Effects of Project Follow-Through…” • Quasi-experimental study • Ex post facto study • Problem: Are the two groupsin this study comparable to one another?
In order to make sure that the two groups are comparable, and to ensure that the only post-test differences between the groups are due to the independent variable (the Follow-Through intervention), data were obtained on students’: family income gender language spoken in home mother’s education ethnicity number of siblings.
FOLLOW-THROUGH Year 1 (1975) Gr 5 Site 1 Site 2 Site 3 Site 4 Site 5 Year 2 (1976) Gr 6 Site 1 Site 2 Site 3 Site 4 Site 5 NO FOLLOW-THROUGH Year 1 (1975) Gr 5 Site 1 Site 2 Site 3 Site 4 Site 5 Year 2 (1976) Gr 6 Site 1 Site 2 Site 3 Site 4 Site 5 Research Design
Dependent variables • Wide-Range Achievement Test (WRAT) • reading • mathematics • Metropolitan Achievement Test (MAT) • reading • mathematics
RESULTS • A total of 180 comparisons of FT to No-FT students. Of these, only 56 (31%) favored FT students! • Largest differences between FT and No-FT students were in basic skills areas. • FT students’ achievement declined by grades 5 and 6 (2-3 years after end of FT).
Critique of this research • What are the strengths of the study? • Groups are comparable to one another. • Contrasted statistical with practical significance. • Large sample size. • Multiple “replications” of treatment effect. • What are the weaknesses of the study? • Lack of random assignment. • Focus on standardized test performance.