Experimental Design

1. Experimental Design Source: Christine Ambrosino @ http://www.hawaii.edu/fishlab/NearsideFrame.htm

2. Definition • In general usage: is the design of any information-gathering exercises where variation is present, whether under the full control of the experimenter or not. – wikipedia • In research: a study design used to test cause-and-effect relationships between variables. The classic experimental design specifies an experimental group and a control group. The independent variable is administered to the experimental group and not to the control group, and both groups are measured on the same dependent variable. Subsequent experimental designs have used more groups and more measurements over longer periods. True experiments must have control, randomization, and manipulation. – on-line medical dictionary

3. Originality in Research • Science depends on original thinking in two main areas: • The original ‘guess’ – called an hypothesis • The test or experiment to determine likelihood of hypothesis being correct A good scientist relies on ‘inspiration’ in the same way as a good artist. Some feel this point is largely ignored in present day science education. Encourage your students to be creative, original or inspired by the everyday when designing their research!

4. Being Discerning in Design • Examples: • Teaching a dog French • Fortune Teller’s long term success

5. Elements of a Good Experiment • Features common to all good experiments which exist regardless of utilizing advanced equipment or basic technique: • Discrimination • Replication & Generality • Controls • ‘Blind’ Design • Measurement

6. Discrimination • Experiments should be capable of discriminating clearly between different hypotheses. It often turns out that two or more hypotheses give indistinguishable results when tested by poorly-designed experiments.

7. Replication & Generality • Living material is notoriously variable. • Usually experiments must be repeated enough times for the results to be analyzed statistically. • Similarly, because of biological variability, we must be cautious of generalizing our results either from individual creatures to others of the same species, or to other species. For instance, if our hypothesis is about mammals, it is inadequate simply to carry out our experiments on laboratory rats. Similarly, it is dangerous to extrapolate from healthy students to elite athletes.

8. Controls • The experiment must be well controlled. We must eliminate by proper checks the possibility that other factors in the overall test situation produce the effect we are observing, rather than the factor we are interested in. • Example

9. “Blind” Design • Investigators can subconsciously 'fudge' their data if they know what result they want to find. • The answer is to do the experiment 'blind', so the investigators (and the subjects, if humans are being studied) do not know which treatment's effect they are observing. • This can make the logistics of doing the experiment more complex. • Example

10. Measurement • Good experiments usually involve measuring something i.e. - length. • Important you know both the accuracy and the precisionof your measuring system. • These two terms are not synonymous: • 'accuracy' means the ability of the method to give an unbiased answer on average, • 'precision' is an index of the method's reproducibility. • Ideally your method should be both • accurate (i.e., give the true mean) • and precise (i.e., have a low standard deviation). Sometimes one is more important than the other. For example, if you were looking for small changes with time in a quantity (such as an athlete's hemoglobin concentration), you would need a precise measure of it rather more than an accurate one. Accuracy and precision together help you to judge the reliability of your data. They also help you to judge to how many significant figures you should quote your results. For example, if you use a balance reading to the nearest gram, you should give the results to the nearest gram and not to the nearest tenth of a gram. • Some experiments are very difficult to do because it is not obvious what can be measured. • This is a real problem in animal behavior: for example, there is no obvious unit or measure for 'emotional state'. It is usually necessary to isolate measurable components of behavior. Thus the speed at which a tiger paces up and down a cage can give some indication of the internal state of the animal but can never give a full picture of it.

11. Measurement • Plant Example • Detergent Example

12. Experimental Design & Statistics • Good experimental design involves having a clear idea about how we will analyze the results when we get them. • That's why statisticians often tell us to think about the statistical tests we will use before we start an experiment.

13. Examples • Example 1. Experiments that yield no useful results because we did not collect enough data • 9:3:3:1

14. Example 1 (Cont) • Chi squared test • Poisson distribution • Student’s t-Test • How many replicates to use?

15. Examples • Example 2. Experiments that seem to give useful results but our procedures let us down! • Latin Square vs Randomization

16. Example 2 (Cont) • Never put all the replicates of one treatment together! • To block or not to block – real world dilemmas • Field Examples

17. More on Eliminating Bias • Yale Experimental Design Example

18. Steps to Follow • 1. Define the objectives. • Record (i.e. write down) precisely what you want to test in an experiment. • 2. Devise a strategy. • Record precisely how you can achieve the objective. • This includes thinking about the size and structure of the experiment – • how many treatments? • how many replicates? • how will the results be analysed? • 3. Set down all the operational details. • How will the experiment be performed in practice? • In what order will things be done? • Should the treatments be randomised or follow a set structure? • Can the experiment be done in a day? • Will there be time for lunch? etc.

19. The Really Easy Statistics Site by: Jim Deacon, Biology Teaching Organisation, University of Edinburgh SOURCE USED FOR ABOVE INFO

20. Introduction to Experimental Design Lesson Plan with Worksheets By: Daniell Difrancesca; Critical Thinking in Science Concepts of Experimental Design (pdf) By: SAS; A SAS White Paper ADDITIONAL RESOURCES

21. Questions?