Section 5.2 Notes: Designing Experiments Experimental Units: Subjects: Treatment: Factors:

1. Section 5.2 Notes: Designing Experiments Experimental Units: Subjects: Treatment: Factors:

2. Section 5.2 Notes: Designing Experiments Experimental Units: the individuals/ objects on which the experiment is done Subjects: Treatment: Factors:

3. Section 5.2 Notes: Designing Experiments Experimental Units: the individuals/ objects on which the experiment is done Subjects: Human experimental units Treatment: Factors:

4. Section 5.2 Notes: Designing Experiments Experimental Units: the individuals/ objects on which the experiment is done Subjects: Human experimental units Treatment: specific experimental condition applied to the experimental units Factors:

5. Section 5.2 Notes: Designing Experiments Experimental Units: the individuals/ objects on which the experiment is done Subjects: Human experimental units Treatment: specific experimental condition applied to the experimental units Factors: explanatory variable

6. What are some advantages of an experiment? • . • . • .

7. What are some advantages of an experiment? • Gives good evidence for causation • . • .

8. What are some advantages of an experiment? • Gives good evidence for causation • Controls for lurking variables • .

9. What are some advantages of an experiment? • Gives good evidence for causation • Controls for lurking variables • Allows us to study the effects of several factors

10. Determine if each study is an observational study or an experiment, and then identify the explanatory and response variables in each situation. Remember: A response variable measures the outcome of a study. An explanatory variable attempts to explain the observed outcomes. One effect of alcohol is a drop in body temperature. To study this effect, researchers give several amounts of alcohol to mice, and then measured the change in each mouse’s body temperature.

11. Determine if each study is an observational study or an experiment, and then identify the explanatory and response variables in each situation. Remember: A response variable measures the outcome of a study. An explanatory variable attempts to explain the observed outcomes. A study if done to try and find the correlation between verbal and math SAT scores. The scientist wants to use the verbal score to predict the math score.

12. Determine if each study is an observational study or an experiment, and then identify the explanatory and response variables in each situation. Remember: A response variable measures the outcome of a study. An explanatory variable attempts to explain the observed outcomes. Some breast cancer patients were given each a new treatment. The patients were closely followed to see how long they lived following surgery.

13. Determine if each study is an observational study or an experiment, and then identify the explanatory and response variables in each situation. Remember: A response variable measures the outcome of a study. An explanatory variable attempts to explain the observed outcomes. To find out how well a child’s height predicts their age a study was done where they measured the heights of a group of children at age 6, wait until they are 16 and then measure their heights again.

14. The 3 Principles of Experimental Design 1. 2. 3. Good experiments have all 3!!

15. The 3 Principles of Experimental Design 1. Control 2. Replicate 3. Randomize Good experiments have all 3!!

16. Experimental Design Control the effects of lurking variables on the response, most simply by comparing two or more treatments. We rely on the controlled environment of the laboratory to protect us from lurking variables. ___________are a way to control for lurking variables. ________are a dummy treatment. The ______________occurs when subjects who have no active drug ingredient (e.g. a sugar tablet) think they experience a certain beneficial effect.

17. Experimental Design Control the effects of lurking variables on the response, most simply by comparing two or more treatments. We rely on the controlled environment of the laboratory to protect us from lurking variables. Placebos are a way to control for lurking variables. Placebos are a dummy treatment. The Placebo effect occurs when subjects who have no active drug ingredient (e.g. a sugar tablet) think they experience a certain beneficial effect.

18. Experimental Design Control the effects of lurking variables on the response, most simply by comparing two or more treatments. We rely on the controlled environment of the laboratory to protect us from lurking variables. Another way to take lurking variables into account is with a __________. A ___________ is treated identically in all respects to the group receiving the treatment except that the members of the control group do not receive the treatment.

19. Experimental Design Control the effects of lurking variables on the response, most simply by comparing two or more treatments. We rely on the controlled environment of the laboratory to protect us from lurking variables. Another way to take lurking variables into account is with a control group. A control group is treated identically in all respects to the group receiving the treatment except that the members of the control group do not receive the treatment. Note: Don’t confuse control and control group. Control refers to the overall effect to minimize variability in the way the experimental units are obtained and treated.

20. Experimental Design • Replicate each treatment on many units to reduce chance variation on the results. • Even with control, there will be natural variability among experimental units. If each treatment is assigned to only one unit, you won’t know whether any systematic differences in responses were due to the treatments or to the natural variability in the units. So obviously, to reduce chance variation, we need to use a large amount of subjects.

21. Experimental Design Randomize-- use impersonal change to assign experimental units to treatments. The use of chance to divide experimental units into groups is called _____________________. ________________ is the most important element of any experiment. It must be incorporated either in the selection process of experimental units and/or the distribution of experimental units into treatment and control groups. You can use your calculator, the random digit table or names out of a hat or flipping a coin to randomize an experiment.

22. Experimental Design Randomize-- use impersonal change to assign experimental units to treatments. The use of chance to divide experimental units into groups is called randomization. Randomization is the most important element of any experiment. It must be incorporated either in the selection process of experimental units and/or the distribution of experimental units into treatment and control groups. You can use your calculator, the random digit table or names out of a hat or flipping a coin to randomize an experiment.

23. When comparing groups in experiments, we hope to see a difference in the responses so large that it is unlikely to happen just because of chance. If an observed effect is so large that it would rarely occur by chance, it is called statistically significant.

24. Types of Designs Part 1—Randomized Comparative Design The design of an experiment is crucial. Experiments are designed with the purpose of isolating the effect of the treatment on the response variable and removing any confounding effects. Randomized Comparative Design/Complete Randomized Design a. This is the most generic type of experimental design. b. In this type of design… Diagram:

25. Randomized Comparative Design Does talking on a hands-free cell phone distract drivers? Undergraduate students “drove” in a high-fidelity driving simulator with a hands-free cell phone. The car ahead brakes: how quickly does the subject respond? 20 students (the control group) simply drove. Another 20 (experimental group) talked on the cell phone while driving. Design a randomized comparative experiment.

26. Randomized Comparative Design A teacher is interested in the effect of three different teaching methods on student’s learning. She teaches 100 students. Design a randomized comparative experiment.

27. Types of Designs Part 2—Block Design One way that we have seen already of removing the effect of any confounding variables is to randomly assign subjects to the treatment or control group. This way any possible bias in the population should be evenly spread among the treatment and control groups. Sometimes instead of relying on randomization to make the groups as even as possible we actually force the groups to be similar. Consider the following situation… Suppose a fitness instructor believes a certain fitness regimen will increase upper-body strength. He tests his theory by getting his recruits to complete push-ups. You would expect a certain amount of variability because some subjects will be stronger than others. We try to control for these differences by placing subjects into groups of similar individuals. One way to accomplish this is to separate the men and the women. By separating subjects by gender, we can reduce the effect of variation in strength on the number of push-ups. This is the idea behind a block design.

28. Block Design • A block is …. A group of units that are similar in some way that is expected to effect responses. • In a block design… the random assignment of units to treatments is carried out separately within each block. • Facts about Block Designs: • Can be any size • Block by variables we can’t control

29. Block Design Diagram:

30. Criteria for Blocking • Remember, the purpose of blocking is to sort experimental units into groups within each of which the elements are homogeneous with respect to the response variable, such that the differences between groups are as small as possible. • Characteristics associated with the unit • For persons: gender, age, income, education, job experience, etc. • For geographical areas: population size, average income • Characteristics associated with the experimental setting: • Observer, time of processing, machine, batch of material, measuring instrument, etc.

31. Block Design Example: The progress of a type of cancer differs in women and men. A clinic wants to experiment to compare the effect of three different therapies for this cancer. Produce a block design for this experiment.

32. Block Design Example: A chemist is studying the reaction rate of five chemical agents. At most five agents can be analyzed effectively per day. It is known that day-to-day differences may substantially affect the reaction rate. Make a block design for this experiment that takes place over a course of three days.

33. Part 3—Matched Pairs Design Consider the following situation… There are 50 students in a math class. They took a pretest at the beginning of the semester and a posttest at the end of the semester. Interested in how the students improved over the course of the semester, the teacher chose 25 pretest scores at random and 25 posttest scores at random. She then compared the different scores. Was this the best method to use for this experiment?

34. Matched Pair Design Comparing the pretest and posttest of each individual student would have been more effective because of several lurking variables that exist with the teacher’s first experiment. Conducting the experiment using this better method is an example of matched pairs design. In a matched pairs design… either the same unit or two matched units are assigned to receive 2 treatments.

35. Matched Pair Design • FACTS ABOUT MATCHED PAIRS DESIGNS: • Each unit receives both treatments. • Each unit serves as his/her own block. • Two treatments only. • Randomize the order of when each treatment is received. • Takes into account person-to-person variation.

36. Matched Pairs Design Diagram:

37. Matched Pairs Design A medical researcher is interested in testing a new medication for poison ivy. He decides to conduct a clinical trial on 250 volunteers who are allergic to poison ivy. He purposefully rubs poison ivy on their calf, then after the rash appears, he gives half of the volunteers calamine lotion, and the other half he gives new medication. How can this experiment be conducted in matched pairs design?

38. Matched Pairs Design A medical researcher is interested in testing a new medication for poison ivy. He decides to conduct a clinical trial on 250 volunteers who are allergic to poison ivy. He purposefully rubs poison ivy on their calf, then after the rash appears, he gives half of the volunteers calamine lotion, and the other half he gives new medication. How can this experiment be conducted in matched pairs design? To make a matched pairs design, all subjects will receive both lotions; both legs of the 250 volunteers will be infected with the poison ivy. Randomly choose 125 to get the calamine lotion on the right leg and the new medication on the left leg. The remaining 125 will have the new medication on their right leg and the calamine lotion on their left leg. Compare each of the subjects’ legs to determine the best treatment.

39. Matched Pairs Design Advertising that hopes to attract Mexican Americans must keep in mind the cultural orientation of these consumers. There are several psychological tests available to measure the extent to which Mexican Americans are oriented toward Mexican/Spanish culture or Anglo/English culture. Two such tests are the BI and the ARSMA. To study the relationship between the scores on these two tests, researchers will give both tests to a group of 22 Mexican Americans. Briefly describe a matched pairs design for the study. A matched pairs design involves administering each test to the participants of the same gender. For example, BI to only the females and ARSMA to only the males. A matched pairs design involves administering BI to 11 participants and ARSMA to the other 11 participants. A matched pairs design involves administering the two tests to each individual in the experiment. None of the above.

40. Part 4—Design Strategies and Tools Recall the placebo effect: To control for the placebo effect, researchers often administer a neutral treatment (i.e. a placebo) to the control group. The classic example is using a sugar pill in drug research. The drug is effective only if participants who receive the drug have better outcomes than participants who receive the sugar pill. Of course, if participants in the control group know that they are receiving a placebo, the placebo effect will be reduced or eliminated; and the placebo will not serve its intended control purpose…So, we just don’t tell them! This is called blinding.

41. Design Strategies and Tools Blinding is… when participants are not told which treatment they are receiving. Double Blinding is… when participants and the administrators are not told who receives which treatment.

42. Design Strategies and Tools To study the effects of Vitamin C, a researcher takes a SRS of 100 students at BCHS. He randomly assigns half the students into a treatment group and half into a control group. He administers Vitamin C pills to the treatment group and sugar pills to the control group. He does this over a course of three days. One week later, the researcher observes the number of sicknesses in both the treatment and control groups. What kinds of strategies or tools are being implemented here?

43. Design Strategies and Tools In cold studies, the doctors in charge label the treatments with a code number (such as #90210). Patients receive a treatment labeled with a code number. The nurses who give the treatments record the responses know the treatment by its code number. Neither the nurse nor the patient know if the treatment being imposed is an experimental drug or a placebo. What strategy or tool is being implemented here? Note: Blinding and Double-Blinding are not types of experimental designs. We just use them in our designs. The three main designs are:

44. Which of the following are characteristics of a sound experimental design? Compare several treatments to control the effects of lurking variables on the response. Use randomization to assign subjects to treatments. To reduce chance variation in the results, repeat the experiment on many subjects. I only II only III only I and III only I, II, and III

45. A statistics student wishes to test the strength of various brands of paper towel. He chooses 5 brands and selects 6 towels from each brand. He numbers them 1-30. He randomly selects a towel and places it in an embroidery hood. Exactly 10 ml of water and a large weight are placed in the center of the towel. The time it takes for the towel to break is recorded. In this case, the explanatory variable is the Amount of time it takes for the towel to break 10 ml of water and the large weight Brand of paper towel Large weight Number of paper towels used in the experiment

46. A Physicians’ Study Group wished to design a matched pairs test to determine the effects of a medication on reducing the elevated cholesterol levels in 200 adults in the 35-65 age group. Which of the following represents such a design? The Physicians’ Group compares cholesterol levels of each patient before he/she begins a daily regimen of the medicine to the patient’s cholesterol level after six weeks of using the medication. The Physicians’ Group compares the cholesterol levels of the patients using the medicine for six weeks with the cholesterol levels of 200 adults in the same age group who exhibited elevated cholesterol levels and did not take the medicine. The Physicians’ Group compared the mean cholesterol level of the 200 patients six weeks after using the medicine with a published national mean cholesterol level for all adults in this age group. The Physicians’ Group compares cholesterol levels of the 200 patients using the medicine for six weeks with the cholesterol levels of 200 adults in the same age group who were taking a placebo. All of the above are representative of a matched pairs design.

47. A local university has a “drop-in center” located near the Student Union where students can “drop-in” to speak to a counselor on any issue or concern whether personal or academic. A researcher stations herself at a receptionist’s desk during a few random evenings to collect data on whether men or women are more likely to use this service and whether a particular class (freshmen, sophomore, junior, senior) is more likely to “drop-in”. This study may be described as a Controlled experiment Matched pairs design Census Observational study A double blind completely randomized experiment

48. Which of the following are true statements? In a block design, the random assignment of units to treatments is carried out separately within each block. The purpose of blocking is to reduce variation in results. I only II only I and II only II and III only I, II, and III