Statistical and practical significance
This presentation is the property of its rightful owner.
Sponsored Links
1 / 24

Statistical and Practical Significance PowerPoint PPT Presentation


  • 96 Views
  • Uploaded on
  • Presentation posted in: General

Statistical and Practical Significance. Advanced Statistics Petr Soukup. Outline. Reminder of statistical significance Limits of statistical significance Misuses of statistical significance Alternatives to statistical significance Practical significance Effect sizes.

Download Presentation

Statistical and Practical Significance

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Statistical and practical significance

Statistical and Practical Significance

Advanced Statistics

Petr Soukup


Outline

Outline

  • Reminder of statistical significance

  • Limits of statistical significance

  • Misuses of statistical significance

  • Alternatives to statistical significance

  • Practical significance

  • Effect sizes


Statistical and practical significance

REMINDER OF STATISTICAL SIGNIFICANCE (NHST)


Hypotheses and tests

Hypotheses and tests

  • Tested hypothesis in experiments (Fisher, 1925)

  • Null and alternative hypothesis (NHST) (Neyman&Pearson, 1937)

  • Common tests - t-tests, analysis of variance, analysis of covariance, correlation analysis etc.


Definition of statistical significance

Definition of statistical significance

Definition: Conditionalprobability, that our sample can

be drawn from population in which null hypothesis

is valid (α). Statistical significance is P(D/H0) and not P(H0/D)


Statistical and practical significance

LIMITSOF NHST


Assumptions for classical nhst

Assumptions for classical NHST

  • Big big probability samples from infinite or very big finite populations

    Three assumptions:

  • Big (infinite) population (at least 100times bigger than the sample)

  • Probability sampling (all units same probability of selection)

  • Big sample (> 30-50 units)


Limits of nhst

LIMITS OF NHST

  • 1.data from censuses

  • 2. data from non-probability samples

  • 3. data from small samples

  • 4. data based on sample that are big proportion of the basic population

  • 5. big data samples from merged (internationally or by time) files


Beyond the limits of nhst in csr

Beyond the limits of NHST in CSR*

N=32 articles, Czech sociological review 2000-2006 (selected 29 issues), own research

*CSR-Czech sociological review


Statistical and practical significance

MISUSES OF NHST


Objections against nhst misuses of nhst

Objections against NHST (Misuses of NHST)

a) Insufficient statement about population,

b) null hypotheses are unreal (nill null),

c) mechanical usage of classical 5% statistical significance (asterisks, stepwise methods, best models etc.),

d) statistical significant doesn’t mean important,

e) publishing only statistical significant results (file drawer problem).


Misuses of nhst in csr

Misuses of NHST in CSR*

N=32 articles, Czech sociological review 2000-2006 (selected 29 issues), own research

*CSR-Czech sociological review


Conference examples p 0 01

Conference examples (P<0,01)


Conference examples

Conference examples (***)


Conference examples and stepwise

Conference examples (*** and stepwise)


Statistical and practical significance

ALTERNATIVESTO NHST


Some alternatives to statistical significance

Some alternatives to statistical significance

a) Confidence Intervals (Problems for r, formulas, regression etc.)

b) Test power (quite good in sociology),

c) Estimate of minimum sample size & What if strategy,

d) Comparison of models via information criterias (AIC, BIC)

e) Bayesian approach


Statistical and practical significance

PRACTICAL SIGNIFICANCE


Practical significance terminology

Practical significance - terminology

  • Practical significance

    b) Substantive significance

    c) Logical significance

    d) Scientific significance

    sometimes also:

    e) result importance or

    f) result meaningfulness


How to measure practical sig

How to measure Practical sig.?

History - Absolute and relative approach

Example: Income differencies

Absolute and relative difference


How to measure practical sig1

How to measure Practical sig.?

Effect sizes – measures of practical significance

Some well known:

Cohen d

Hayes ω

But also R2, r, C, Fisher η2 are effect sizes

Problem: Sometimes published but not interpreted


Statistical and practical significance

OTHER SIGNIFICANCES


Special significances

Special significances

  • Economic significance

  • Clinical significance

  • Etc.


Conclusion

CONCLUSION?

Statistical significance is:

LIMITED

MISUSED

BUT NOT BAD

Substantive significance is:

NOT OFTEN USED

BUT NECESSARY


  • Login