Maximizing Validity in Experimental Designs

1. EXPERIMENTAL DESIGNS

2. Categories • Lab experiments • Experiments done in artificial or contrived environment • Field experiments • Experiments done in natural environment in which activities regularly takes place

3. Lab Experiment • Control • When a cause-and-effect relationship between an independent variable and dependent variable of interest is to be clearly established, then all other variables that might contaminate or confound the relationship have to be tightly controlled • Manipulation • We create different levels of independent variables to assess the impact on dependent variable

4. Controlling the Contaminating Exogenous or ‘Nuisance’ Variables • Matching Groups • To match the various groups by picking the confounding characteristics and deliberately spreading them across groups • Randomization • Distributing the confounding variables among the groups equally

5. Internal Validity • Refers to the confidence we place in the cause-and-effect relationship

6. External Validity • To what extent the results found in the lab setting be transferable or generalizable to the actual organizational or field settings?

7. The Field Experiment • Field experiments have more external validity, but less internal validity • In the lab experiments, the reverse in true

8. Factors Affecting Internal Validity • History Effects • Certain events or factors that would have an impact on the independent variable-dependent variable relationship might unexpectedly occur while the experiment is in progress, and this history of events would confound the cause-and-effect relationship between two variables

9. Factors Affecting Internal Validity (Cont’d) • Maturation Effects • Cause-and-effect inferences can also be contaminated by the effects of the passage of time • Testing Effects • The fact that respondents were exposed to the pretest might influence their responses on the posttest

10. Factors Affecting Internal Validity (Cont’d) • Instrumentation Effects • This might arise because a change in the measuring instrument between pretest and posttest. • Selection Bias Effects • Improper or unmatched selection of subjects for the experimental and control groups

11. Factors Affecting Internal Validity (Cont’d) • Statistical Regression • When the members chosen for the experimental group have extreme scores on the dependent variable • Mortality • Mortality or attrition on the members in the experimental or control group or both

12. Additional Threats to Internal Validity Diffusion of treatment Compensatory equalization Compensatory rivalry Resentful disadvantaged Local history

13. Factors Affecting External Validity • The effects of the treatment will not be the same in the field • Selection of the subjects

14. Types of Experimental Designs • Quasi-Experimental Designs • No comparison between groups, nor any recording of the status of the dependent variable as it was prior to the experimental treatment and how it changes after the treatment • Pretest and posttest experimental group design • Posttest only with experimental and control group

15. Types of Experimental Designs (Cont’d) • True Experimental Designs • Pretest and posttest experimental and control group designs • Solomon four-group design • Two experimental groups and two control groups

16. Simulation • An experiment conducted in a specially created setting that very closely represents the natural environment in which activities are usually carried on

17. Ethical Issues • Putting pressure on individuals to participate in experiments through coercion, or applying social pressure • Giving mental tasks and asking demeaning questions that diminish their self-respect • Deceiving subjects by deliberately misleading them as to the true purpose of the research

18. Ethical Issues (Cont’d) • Exposing participants to physical or mental stress • Not allowing subjects to withdraw from the research when they want to • Using the research results to disadvantage of participants, of for purposes nor their liking • Exposing respondents to hazardous and unsafe environment

19. Ethical Issues (Cont’d) • Not debriefing participants fully and accurately after the experiment is over • Not preserving the privacy and confidentiality of the information given by the participants • Withholding benefits from control groups