1 / 38

Inference in Biology

Inference in Biology. BIOL4062/5062 Hal Whitehead. What are we trying to do? Null Hypothesis Significance Testing Problems with Null Hypothesis Significance Testing Alternatives: Displays, confidence intervals, effect size statistics Model comparison using information-theoretic approaches

ifama
Download Presentation

Inference in Biology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Inference in Biology BIOL4062/5062 Hal Whitehead

  2. What are we trying to do? • Null Hypothesis Significance Testing • Problems with Null Hypothesis Significance Testing • Alternatives: • Displays, confidence intervals, effect size statistics • Model comparison using information-theoretic approaches • Bayesian analysis • Methods of Inference in Biology

  3. What are we trying to do? • Descriptive or exploratory analyses • Fitting predictive models • Challenging research hypotheses

  4. What are we trying to do? • Descriptive or exploratory analyses • What factors influence species diversity? • Fitting predictive models • Can we make global maps of species diversity? • Challenging research hypotheses • Is diversity inversely related to latitude?

  5. The traditional approach:Null Hypothesis Significance Testing • Formulate null hypothesis • Formulate alternative hypothesis • Decide on test statistic • Collect data • What is probability (P) of test statistic, or more extreme value, under null hypothesis? • If P<α (usually 0.05) conclude: • Reject null in favour of alternative • If P>α conclude: • Do not reject null hypothesis

  6. Null Hypothesis Significance TestingAn example • Formulate null hypothesis • “Species diversity does not change with latitude” • Formulate alternative hypothesis • “Species diversity decreases with latitude” • Decide on test statistic • Correlation between diversity measure and latitude, r • Collect data • 405 measures of diversity at different latitudes • What is probability (P) of test statistic, or more extreme value, under null hypothesis? • r = -0.1762; P = 0.002 (one-sided) • If P<α (usually 0.05) conclude: • Reject “Species diversity does not change with latitude”

  7. Criticisms of: Null Hypothesis Significance Testing (1) • α is arbitrary • Most null hypotheses are false, so why test them? • Statistical significance is not equivalent to biological significance • with large samples, statistical significance but not biological significance • with small samples, biological significance but not statistical significance • If statistical power is low, the null hypothesis will usually not be rejected when false • Encourages arbitrary inferences when many tests carried out

  8. Criticisms of: Null Hypothesis Significance Testing (2) • Power analysis does not save NHST • arbitrary, confounded with P-value • “vacuous intellectual game” (Shaver 1993) • Incomplete reporting and publishing • only report statistically significant results • only publish statistically significant results • Focussing on one null and one alternative hypothesis limits scientific advance • Emphasis on falsification obscures uncertainty about “best” explanation for phenomenon

  9. Misuse of: Null Hypothesis Significance Testing • Failure to reject null hypothesis does not imply null is true • Probability of obtaining data given null hypothesis is not probability null hypothesis is true • Poor support for null hypothesis does not imply alternative hypothesis is true

  10. Johnson (1999) “The insignificance of statistical significance testing”J. Wild. Manage.

  11. Johnson (1999) “The insignificance of statistical significance testing”J. Wild. Manage.

  12. Null Hypothesis Significance Testing: • “no longer a sound or fruitful basis for statistical investigation” (Clarke 1963) • “essential mindlessness in the conduct of research” (Bakan 1966) • “In practice, of course, tests of significance are not taken seriously” (Guttman 1985) • “simple P-values are not now used by the best statisticians” (Barnard 1998) • “The most common and flagrant misuse of statistics... is the testing of hypotheses, especially the vast majority of them known beforehand to be false” (Johnson 1999)

  13. “The problems with Null Hypothesis Significance Testing are so severe that some have argued for it to be completely banned from scholarly journals”Denis (2003) Theory & Science

  14. Alternatives to:Null Hypothesis Significance Testing • Displays, confidence intervals, effect size statistics • Model comparison using information-theoretic approaches • Bayesian statistics

  15. 95% c.i. Diversity and latitude • r = -0.1762; P = 0.002 • r = -0.1762; 95% c.i.: -0.2690; -0.0801

  16. Diversity and latitude:Maybe by focussing on the diversity-latitude hypothesis, we have missed the real story

  17. Effect Size Statistics • indicate the association that exists between two or more variables • Pearson’s r correlation coefficient (or r2) • for two continuous variables • Cohen’s d • for one continuous, one two-level category (t-test) • Hedges’ g • better than d when sample sizes are very different • Cohen’s f2 • for one continuous, one multi-level category (F-test) • Cramer’s φ • for two categorical variables (Chi2 test) • Odds ratio • for two binary variables

  18. Cohen’s d • d = 0.2 indicative of a small effect size • d = 0.5 a medium effect size • d = 0.8 a large effect size d = Difference between means of two groups Pooled standard deviation

  19. Problems with effect size statistics • No serious problems • But they don’t tell the whole story

  20. Model fitting:How can we best predict diversity?

  21. constant SST SST, SST2 SST, SST2, SST3 lat lat, lat2 lat, lat2, lat3 SST, SST2, lat SST, SST2, lat, lat2 SST, SST2, lat, lat2, lat3 ocean SST, SST2, ocean area SST, SST2, area Some models of diversity SST = Sea Surface Temperature lat = Latitude Ocean = Atlantic /Pacific area = Ocean area (categorical) But which is best?

  22. Lowest RSS but many parameters Which model is best? Model: Residual sum of squares Parameters constant 0.854 2 SST 0.774 3 SST, SST2 0.724 4 SST, SST2, SST3 0.726 5 lat 0.835 3 lat, lat2 0.804 4 lat, lat2, lat3 0.785 5 SST, SST2, lat 0.725 5 SST, SST2, lat, lat2 0.722 6 SST, SST2, lat, lat2, lat3 0.724 7 ocean 0.844 3 SST, SST2, ocean 0.725 5 area 0.831 4 SST, SST2, area 0.723 6 How to weigh fit of model against number of parameters?

  23. Which model is best? • Information-theoretic AIC • Akaike Information Criterion • A measure of the similarity between the statistical model and the true distribution • Trades off the complexity of a model against how well it fits the data

  24. Lowest AIC: Best Model Which model is best? Model: RSSParameters AIC constant 0.854 2-61.08 SST 0.774 3-99.81 SST, SST2 0.724 4-125.54 SST, SST2, SST3 0.726 5-123.64 lat 0.835 3-69.09 lat, lat2 0.804 4-83.19 lat, lat2, lat3 0.785 5-92.19 SST, SST2, lat 0.725 5-124.10 SST, SST2, lat, lat2 0.722 6-125.05 SST, SST2, lat, lat2, lat3 0.724 7-123.05 ocean 0.844 3-64.88 SST, SST2, ocean 0.725 5-124.27 area 0.831 4-69.77 SST, SST2, area 0.723 6-124.59

  25. How much support for different models? Model: AICΔAIC constant-61.0864.46 SST-99.8125.73 SST, SST2-125.540.00 SST, SST2, SST3-123.641.90 lat-69.0956.45 lat, lat2-83.1942.35 lat, lat2, lat3-92.1933.35 SST, SST2, lat-124.101.45 SST, SST2, lat, lat2-125.050.49 SST, SST2, lat, lat2, lat3-123.052.49 ocean-64.8860.66 SST, SST2, ocean-124.271.27 area-69.7755.77 SST, SST2, area-124.590.96

  26. How much support for different models? Model: AICΔAIC constant-61.0864.46 No support SST-99.8125.73 No support SST, SST2-125.540.00 Best model SST, SST2, SST3-123.641.90Some support lat-69.0956.45 No support lat, lat2-83.1942.35 No support lat, lat2, lat3-92.1933.35 No support SST, SST2, lat-124.101.45Some support SST, SST2, lat, lat2-125.050.49Some support SST, SST2, lat, lat2, lat3-123.052.49Little support ocean-64.8860.66 No support SST, SST2, ocean-124.271.27Some support area-69.7755.77 No support SST, SST2, area-124.590.96Some support

  27. Relative importance of variablesfrom AIC SST 1.000 SST2 1.000 SST3 0.211 lat 0.398 lat2 0.280 lat3 0.075 ocean 0.128 area 0.141

  28. Best model of diversity:Diversity = 0.293 + 0.261SST - 0.00614SST2 Use this model to predict:

  29. Global pattern of diversityapply equation to global SST map

  30. Global pattern of diversityapply equation to SST predictionsfrom global circulation models

  31. Indicates “best” model and support for other models Can compare very different models Balances complexity of model against fit Produces predictive models Fairly simple mathematically and computationally Model averaging Philosophical basis “nuanced” Which models to consider is subjective Advantages and criticisms of information-theoretic model-fitting

  32. Bayesian Analysis • Given prior distribution of models or model parameters • Collect data • Work out probability of data for each model and combination of model parameters • Work out posterior distribution of models or model parameters • using Bayes’ theorem

  33. Bayes’ Theorem Posterior probability of model given data = Probability of data given model X Probability of model Probability of data

  34. Bayesian Analysis • So, Bayesian analysis gives: • the probability of models or parameters given prior knowledge and data • very nice! • but may need considerable computation

  35. Example of Bayesian Analysis • Trying to work out survival rate of newly studied species of rodent • Ten other species in genus have mean survival per year of 0.72 (SD 0.13) • Of 20 animals marked, 17 survive for 1 year • Standard (binomial) estimate of survival = 0.850 (95% c.i. 0.621 - 0.968) • Bayesian estimate of survival = 0.797 (95% c.i. 0.637 - 0.921)

  36. Philosophically very nice Gives probability of model given data and prior information Updates estimates as more information becomes available Does not give biologically implausible estimates e.g. survival >1 Fits adaptive management paradigm Choice of priors somewhat arbitrary Bayesian analysis with “uninformative priors” gives similar results to simpler methods Complex Computation can be VERY time consuming and opaque Advantages and Difficulties with Bayesian Analysis

  37. Methods of Inference in Biology • Descriptive or exploratory analyses • Displays, confidence intervals, effect size statistics • Model comparisons using AIC, etc • Bayesian analysis (if prior information) • Null hypothesis significance tests? • Fitting predictive models • Model comparisons using AIC, etc • Bayesian analysis (if prior information) • Challenging research hypotheses • Model comparisons using AIC, etc • Null hypothesis significance tests

  38. This class • Displays, confidence intervals, effect size statistics *** • Model comparisons using AIC, etc ** • Bayesian analysis • Null hypothesis significance tests *

More Related