Experiments in social science

1. Experiments in social science Seeking causal explanations

2. Causality • Telecommunications managers, like social scientists, would like to be able to make causal statements--even if they only point to partial causality • “X causes Y” • Incomplete or imperfect causality • Multiple causality • Partial causality (increased likelihood) • Necessary and sufficient conditions

3. Three conditions for establishing a causal relationship between two concepts • Covariation • Time order • Elimination of alternative explanations

4. Experiments • The experiment is a method where the researcher manipulates one variable (independent variable) and observes its effect on another (dependent variable) under controlled conditions

5. Experiments • Example: A researcher may expose a group of students to a movie with one ending and a second group to the same movie with a different ending (both under laboratory conditions), then measure their emotional response to the movie

6. Features of the experiment • Independent variable • Dependent variable • Subjects • Control

7. Independent variable • The independent variable is the ‘cause’ or ‘causal variable’ in the hypothesis to be tested • The researcher manipulates the independent variable and subsequently measures subjects on the dependent variable • A factor in an experiment is an independent variable whose levels are set by the experimenter • http://www.stat.yale.edu/Courses/1997-98/101/expdes.htm

8. Independent variable • The levels of the factor that are introduced into the experiment are called the ‘treatments’ • If a group is measured on the dependent variable but is not exposed to a non-zero treatment, it is called a ‘control group’ • Some consider this a zero-level treatment, others say it is not a treatment

9. Independent variable • The factor in an experiment must be represented by at least two treatments—experimental and control treatments or two experimental treatments • Stronger experiments include multiple treatment levels

10. Examples: • exposure to a video game v. non-exposure • exposure to different executions of a creative idea • having one half of a class use a website as part of the course and the other half not use it • Exposing one group of subjects to one hour of rap music, another to two hours, another to three, and another to four hours

11. Dependent variable (effect) • The outcome of interest in the study • All groups are measured on the DV • Represents the ‘effect’ • Examples: • liking for a show or a television personality • recall of information from a website • time spent at a website • purchase of cell phones • political activity

12. Subjects • People who are assigned to experimental conditions and measured on the dependent variable • They should be members of the target market/audience • They are often a ‘convenience sample’—especially students in lower-level psychology classes • Nonrandom sampling

13. Control • Any procedures used to see that the only thing that varies for the subject groups is the independent variable • The goal is to isolate impact of the independent variable on the dependent variable

14. Forms of control • Control of the environment • Minimize distraction from noise, lights, action other than exposure to the independent variable • Keep the environment the same across groups • Random assignment of subjects to treatments (randomization) • Trying to make subject groups equivalent in terms of personalities, experiences, demographics

15. Forms of control (continued) • Identical presentation of treatment and measure of dependent variable among groups • Placebo • Timing • Statistical control • Statistically remove the influence of demographics, prior experience, etc. • Requires measuring all variables you will use as controls

16. Forms of control (cont’d) • Experimental Design • Blocking

17. Basic experimental design R X O R O

18. Goal • The ultimate goal of the research is to determine whether the independent variable ‘causes’ the dependent variable under specified conditions • This sounds simpler than it is

19. Strengths of the experimental method • Strong claims to ‘internal’ validity • Strongest ability to infer causality • Relatively low cost • Straightforward interpretation • Scientific aura

20. Weaknesses of the experimental method • Troubles with ‘external’ validity • Artificial setting • Demand characteristics • Hawthorne effect • Forced exposure • Multiple influences controlled • Non-representative samples • Measures divorced from concepts • Kicking a Bobo doll