LT 4.2 Designing Experime nts

1. LT 4.2 Designing Experiments Thanks to James Jaszczak, American Nicaraguan School

2. 3 Important Principles of Experimental Design • Control the effects of lurking variables on the response • Randomize by using impersonal chance to assign experimental units to treatments • Replicate each treatment on many units to reduce chance variation in the results

3. Randomized Comparative Experiments • Randomization produces groups that should be similar in all respects before the treatment is applied • Comparative design ensures that influences other than the treatment operate equally on both groups • Therefore, differences in the two groups must be due to the treatments or that random chance favored one group over the other

4. Control Group • We can reduce the effects of Confounding variables by using a control group • It is the first basic principle of statistical design of experiments • A control group is a group that doesn’t get the treatment • Comparison of several treatments in the same environment is the simplest form of control

5. Control Group • The results of the control group are compared to those of the experimental group • Since the same confounding factors are present in both groups the only difference to show up should be the effect of the treatment • Without control groups the confounding effects of things like the placebo effect can take over and dominate the results, making useless treatments seem very effective

6. Randomization • Comparison of treatments is valid only if the groups are approximately equal • We attempt to match groups by elaborate balancing acts • In medicine we try to match age, sex, weight, blood cholesterol--anything we think might affect the results

7. The Remedy • The remedy for bias that inevitably creeps in is Randomization, the second big idea in statistical design • It doesn’t depend on any characteristic of experimental units or the judgment of the experimenter • The simplest designs create 2 groups, each randomly selected • Randomization is an essential ingredient for good experimental design

8. Replication • In selecting any two groups there will always be some differences • Therefore it is impossible to say with just one trial that the treatment is absolutely the cause of the difference • The more experimental units we assign to the treatment the more those chance variations balance out • “Use enough EU’s to reduce chance variation” is another big idea in experimental design--Replication

9. Placebo Effect • Documented in the early part of the 20th century, the Placebo Effect has confounded medical experiments • The placebo effect is what happens when the subject feels better after having a treatment • Patients are often given a placebo, usually a sugar pill, to measure the actual effect of a treatment

10. Statistical Significance • We use mathematics to see if the response to a treatment is so large that it could not happen just by chance • If they are we say they are statistically significant • Statistical significance is when an effect could occur only rarely just by chance • Notice that it is not impossible, only very unlikely

11. Advantages to Experiments • Experiments can often give good evidence for causation • Observational studies can only show correlation, not necessarily causation • Experiments allow us to study specific factors. We change only one thing to study its effect • Experiments allow us to study the combined effects of several factors simultaneously

12. Studies • A Study is when we actually do something and monitor the response • Experimental Units are the individuals on which the experiment is done • Subjects are humans that are the experimental units • Treatments are the specific experimental conditions applied to subjects or experimental units

13. Explanatory and Response Variables • A treatment, or explanatory variable, is applied to EU’s to measure the response • Determining which is the explanatory and response variables becomes very important • Explanatory variables are sometimes called Factors • Experiments often study the effects of more than one factor • Each treatment is formed by giving specific Levels to each of the factors

14. Completely Randomized • When all experimental units are allocated at random among all treatments we say the experimental design is Completely Randomized • Completely randomized experiments can compare any number of treatments

15. Cautions about Experimentation • We have to be able to treat all the experimental units identically in every way except the treatment • Sometimes we achieve this by making a Double Blind experiment • In a double blind experiment neither the subject nor the researcher knows what treatment the subject is getting • This removes the effect of hidden clues being given by the researcher

16. Lack of Realism • If students know a test doesn’t count it is far more likely they won’t do well than if they know it does. • Telling them it won’t count then is not a good idea to test which questions are the ones being tested • In short the treatment doesn’t replicate the real conditions we want to study • Lack of realism means we can’t extrapolate our results to the whole population, which makes them pretty useless

17. Matched Pairs Design • Matching subjects in various ways can improve our results over complete randomization • Matched pairs compare just two treatments • Two treatments in a pair form a Block • Blocks are chosen randomly as are the position or timing of the treatments within the block

18. Block Designs • Matched pairs are a type of block design • A Block is a group of experimental units that are chosen to be similar in some way • Treatments within the block are chosen completely randomly • Block designs can have blocks of any size • Blocking allows more precise conclusions • Blocks are formed on the most important unavoidable sources of variability among the experimental units • Randomization will then average out the differences between the blocks to give an unbiased comparison of the treatments.