Experiments

1. Experiments Inference for Experiments

2. Experiments: What Can Go Wrong? • The logic of a randomized comparative experiment depends on our ability to treat all the subjects the same in every way except for the actual treatments being compared. • Good experiments, therefore, require careful attention to details to ensure that all subjects really are treated identically.

3. A response to a dummy treatment is called a placebo effect. The strength of the placebo effect is a strong argument for randomized comparative experiments. Whenever possible, experiments with human subjects should be double-blind. Definition: In a double-blind experiment, neither the subjects nor those who interact with them and measure the response variable know which treatment a subject received.

4. Inference for Experiments • In an experiment, researchers usually hope to see a difference in the responses so large that it is unlikely to happen just because of chance variation. • We can use the laws of probability, which describe chance behavior, to learn whether the treatment effects are larger than we would expect to see if only chance were operating. • If they are, we call them statistically significant. Definition: An observed effect so large that it would rarely occur by chance is called statistically significant. A statistically significant association in data from a well-designed experiment does imply causation.

5. Activity: Distracted Drivers Is talking on a cell phone while driving more distracting than talking to a passenger? Read the Activity on page 245. Perform 10 repetitions of your simulation and report the number of drivers in the cell phone group who failed to stop Teacher: Right-click (control-click) on the graph to edit the counts. Experiments In what percent of the class’ trials did 12 or more people in the cell phone group fail to stop? Based on these results, how surprising would it be to get a result this large or larger simply due to chance involved in random assignment? Is this result statistically significant?

6. Randomized Block Design • If an experimenter is aware of specific differences among groups of subjects or objects within an experimental group, he or she may prefer a randomized block design to a completely randomized design.

7. Blocking • Completely randomized designs are the simplest statistical designs for experiments. But just as with sampling, there are times when the simplest method doesn’t yield the most precise results. Definition A block is a group of experimental units that are known before the experiment to be similar in some way that is expected to affect the response to the treatments. In a randomized block design, the random assignment of experimental units to treatments is carried out separately within each block.

8. Form blocks • Randomization will average out the effects of the remaining lurking variables and allow an unbiased comparison of the treatments. • Control what you can, block on what you can’t control, and randomize to create comparable groups.

9. Randomized Block Design In a block design, before the experimental units are randomly assigned to a treatment group: • experimental subjects are divided into homogeneous blocks • The blocks are based on the most important unavoidable sources of variability (lurking variables) • The variability within blocks is less than the variability between blocks. • Reduces variability and potential confounding • Produces a better estimate of treatment effects.

10. Example: Randomized Block Design Suppose a researcher is carrying out a study of the effectiveness of four different skin creams for the treatment of a certain skin disease. He has ninety subjects and plans to divide them into 3 treatment groups of thirty subjects each. If the experimenter has reason to believe that age might be a significant factor in the effect of a given medication, he might choose to first divide the experimental subjects into age groups, such as under 30 years old, 30-60 years old and over 60 years old. Then, within each age level, individuals would be assigned to treatment groups using a completely randomized design. Another way we could do randomized bock design would be to have the subjects assessed and put in blocks of three according to how severe their skin condition is; the four most severe cases are the first block, the moderate cases in the second block, and mildest cases in the third block. The members of each block are then randomly assigned, one to each of the four treatment groups.

11. A researcher is carrying out a study of the effectiveness of four different skin creams for the treatment of a certain skin disease. • He has eighty subjects and plans to divide them into 4 treatment groups of twenty subjects each. • Using a randomized block design, the subjects are assessed and put in blocks of four according to how severe their skin condition is; the four most severe cases are the first block, the next four most severe cases are the second block, and so on to the fourth block. • The four members of each block are then randomly assigned, one to each of the four treatment groups.

12. Example • Suppose you have 500 individuals (250 males, 250 females) participating in a study for a new vaccine. Since it is known that men and women are physiologically different and react differently to medication, we might consider blocking by gender. Then, within each block, subjects are randomly assigned to treatments.

13. Vaccine Example • This design ensures that each treatment condition has an equal proportion of men and women. As a result, differences between treatment conditions cannot be attributed to gender. • This randomized block design removes gender as a potential source of variability and as a potential confounding variable.

14. Matched-Pairs Design • A common type of randomized block design for comparing two treatments is a matched pairs design. The idea is to create blocks by matching pairs of similar experimental units. Definition A matched-pairs design is a randomized blocked experiment in which each block consists of a matching pair of similar experimental units. Chance is used to determine which unit in each pair gets each treatment. Sometimes, a “pair” in a matched-pairs design consists of a single unit that receives both treatments. Since the order of the treatments can influence the response, chance is used to determine with treatment is applied first for each unit.

15. Example • To do a matched pair design using the previous example, the 1000 subjects are grouped into 500 matched pairs. • Each pair is matched on gender and age. • For example, Pair 1 might be two women, both age 21. Pair 2 might be two women, both age 22, and so on.

16. For the acne example, the matched pairs design is an improvement over the completely randomized design and the randomized block design. • Like the other designs, the matched pairs design uses randomization to control for confounding. • However, unlike the others, this design explicitly controls for two potential lurking variables - age and gender.