1 / 65

Variation, sampling, replicates

Variation, sampling, replicates. Importance of visualising data Comparison of 2 groups - What affects statistical significance? Sample size determination – power analysis Case of >1 treatment group compared to one control group Components of variance. Example. Scatter Plot. Scatter Plot.

afya
Download Presentation

Variation, sampling, replicates

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Variation, sampling, replicates • Importance of visualising data • Comparison of 2 groups - What affects statistical significance? • Sample size determination – power analysis • Case of >1 treatment group compared to one control group • Components of variance

  2. Example

  3. Scatter Plot Scatter Plot 600 600 400 400 200 200 0 0 -200 -200 -400 -400 0.3mg/kg/day 0.3mg/kg/day vehicle vehicle Treatment Treatment … now see raw data:

  4. Individual data profiles

  5. Scatter Plot Type - lean Diet - breeding Diet - maintenance 30 25 20 15 10 5 30 25 20 15 10 5 0 10 20 30 40 50 60 70 80 0 10 20 30 40 50 60 70 80 Time OGTT (profile graphs)

  6. Visualising then summarise • Step 1) Visualise individual data using a plot that captures the main features and questions of the experiment and characteristics of the response variable. • Step 2) If a plot of the means and SD adequately summarises the data and is required for presentation, use Excel to generate a plot of the means. Otherwise present summary statistics in a pivot table.

  7. Visualising: 3 main reasons • Data integrity – Check assumptions, validation, looking for outliers. • Visual analysis – To see the data in a way that is in line with the experimental design. • Summary – to convey the key information to others. • Easier to get information from a picture.

  8. Visualising Data • A good visualisation should…. • Show all the data • Avoid biasing the ‘information’ • Encourage the eye to make comparisons • Be integrated with the statistical/verbal descriptions of the data • Always begin by presenting the raw data rather than a summary. • Show contextual information • Use lines to join related pieces of information • Avoid bar graphs • In summary plots, show individual points as well as means. • Convince the reader that the conclusions are reasonable

  9. Variation, sampling, replicates • Importance of visualising data • Comparison of 2 groups - What affects statistical significance? • Sample size determination – power analysis • Case of >1 treatment group compared to one control group • Components of variance

  10. What is Statistical Significance? • Is there a difference in group means? • … relative to spread or variability of data within each group • … based on amount of available data • Often a numerical quantification of what you can see by eye

  11. - + + + + + + + Response - + + + + + + + A B Scenario 1 Large difference in meansrelative to spread within each group – strong statistical evidence of a difference in means.

  12. Response - + + + + + + + - + + + + + + + A B Scenario 2 Smaller difference in means relative to spread within each group – weaker statistical evidence of a difference in means.

  13. +++++++ Response - - +++++++ A B Scenario 3 Large difference in meansrelative to spread within each group – strong statistical evidence of a difference in means.

  14. + + + Response - - + + + A B Scenario 4 Small amount of data on which to base any conclusions – no statistical evidence for a difference in means.

  15. Learning from exercise • Plot the data first • A significant difference is more likely to be detected as: • the difference between the means for the two groups increases • the variability within a group decreases • the number of results per group increases • Ideally, the data are randomly distributed around the mean

  16. Variation, sampling, replicates • Importance of visualising data • Comparison of 2 groups - What affects statistical significance? • Sample size determination – power analysis • Case of >1 treatment group compared to one control group • Components of variance

  17. What is Power Analysis? The power analysis procedure depends on the following 6 features: • What difference would be of biological importance? • What is the typical spread (variability) within group?

  18. What is Power Analysis (ctd)? 3.Significance level (usually 5%) • 1 or 2 sided test? • What is the desired power of the experiment? (usually ~ 80%) • Sample size

  19. What is POWER? • The statistical POWER (sensitivity), of a test is the probability that the test will detect a difference or effect if one is present • The closer the POWER is to 1, the more sensitive the test • POWER is often expressed as a percentage

  20. Sample Size Issues • What is the minimum number of animals that will give us a good chance of detecting a definitive outcome? • Too small group sizes – effect may be big but not statistically significant • Too large group sizes – may see statistical significance, when size of difference is biologically irrelevant

  21. Use of baseline data

  22. Opportunity to improve precision:

  23. Variation, sampling, replicates • Importance of visualising data • Comparison of 2 groups - What affects statistical significance? • Sample size determination – power analysis • Case of >1 treatment group compared to one control group • Components of variance

  24. Group Size Specification Control Group Compound 1 Compound 2 … Compound N In the scenarios which follow, specify how many experimental units (e.g. animals) you would apply to control and treated groups, assuming equal variability in each group:

  25. Group Size Specification Treated Group Size Control Group Size Scenario 1: Total available number of units = 10 1 control group, 1 treated group Treated Group Size Control Group Size Scenario 2: Total available number of units = ~30 1 control group, 4 treated groups Treated Group Size Control Group Size Scenario 3: Total available number of units = ~ 50 1 control group, 9 treated groups If you know from previous experience / data, that the control group is likely to be more variable, how could this influence your allocation?

  26. Treated Group Size Treated Group Size Treated Group Size Control Group Size Control Group Size Control Group Size 10 14 5 5 4 5 Exercise 2 Scenario 1: Total available number of units = 10 1 control group, 1 treated group Scenario 2: Total available number of units = ~30 1 control group, 4 treated groups Scenario 3: Total available number of units = ~ 50 1 control group, 9 treated groups If you know from previous experience / data, that the control group is likely to be more variable, how could this influence your allocation?

  27. Variation, sampling, replicates • Importance of visualising data • Comparison of 2 groups - What affects statistical significance? • Sample size determination – power analysis • Case of >1 treatment group compared to one control group • Components of variance

  28. Section: 1 2 3 1 2 3 1 2 3 Analysis: 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 Components of Variance:Example of a Nested Design Animal: 1 2 3 Q: How many animals / samples per animal / analyses per sample?

  29. Pilot Study • Carry out structured validation study to determine relative sources of variability • Use statistical software to carry out “Components of Variance” analysis • Set sample sizes reflecting relative sources of variability.

  30. Example scenarios: Case 2: Opportunity to increase power with same number ofanimals by taking more sections per animal

  31. Experimental designs to minimize error and improve power • Design to get better precision: • Blocking • Tools to identifying factors to investigate: • Fishbone diagram • Design to improve understanding & maximise information from limited resources: • Factorial design

  32. “The Design of Animal Experiments”, Laboratory Animal Handbooks No. 14 Michael Festing, Philip Overend et al, 2002

  33. Randomised Block Design • Examples of “blocks” include: litter, day, person, cage position • Treatments compared within “block” – differences between “blocks” eliminated in statistical analysis • Increases sensitivity of experiment

  34. Example of Blocking Red squares = day 1, blue circles = day 2 Scatter Plot 20 18 16 14 12 10 8 6 T V Treatment

  35. Example of Blocking Scatter Plot 20 18 16 14 12 10 8 6 T V Treatment

  36. Example Case 1 Day 1: Control Group (N=10) Day 2: Treated Group (N=10) Case 2 Day 1: Control Group (N=10) Treated Group (N=10) Case 3 Day 1: Control Group (N=5) Treated Group (N=5) Day 2: Control Group (N=5) Treated Group (N=5)

  37. Benefits / Uses • Increase precision • Convenience (e.g. time) • Often applies to time / space variables • Increases confidence • Evidence that it is greatly under-used in biomedical literature

  38. Experimental designs to minimize error and improve power • Design to get better precision: • Blocking • Tools to identifying factors to investigate: • Fishbone diagram • Design to improve understanding & maximise information from limited resources: • Factorial design

  39. Source of Variation

  40. Rat Ova Assay Sensitisation Termination Eos counts ip/sc? dose Lungs processed Time of Challenge (14/21 day?) Ovalbumin/Saline? Ovalbumin Dose (10/20) Challenge Restriction: max 50 animals per day.

  41. Group exercise • Fishbone diagram / brainstorming • Draw out fixed and random sources of variation

  42. Experimental designs to minimize error and improve power • Design to get better precision: • Blocking • Tools to identifying factors to investigate: • Fishbone diagram • Design to improve understanding & maximise information from limited resources: • Factorial design

  43. One-Variable-At-A-Time (OVAT) First vary age keeping dose at a fixed level: Glucose (Treated-Control) Age

  44. Reduction by Use of Factorial Experimental Design (FED) Now vary dose keeping age at a fixed level: Glucose (Treated-Control) Dose

  45. One-Variable-At-A-Time (OVAT) Dose Age

  46. One-Variable-At-A-Time (OVAT) Dose Age

  47. What Could I Have Done Instead? Direction for Further Experiments Dose Age

  48. 2 Factors: One-Variable-At-A-Time O Age N=10 N=10 Y H L Dose

  49. N=10 O Age N=10 N=10 Y H L Dose 2 Factors: One-Variable-At-A-Time

  50. 2 Factors: One-Variable-At-A-Time N=10 O Age N=10 N=10 Y H L Dose

More Related