1 / 24

# Statistical and Practical Significance - PowerPoint PPT Presentation

Statistical and Practical Significance. Advanced Statistics Petr Soukup. Outline. Reminder of statistical significance Limits of statistical significance Misuses of statistical significance Alternatives to statistical significance Practical significance Effect sizes.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' Statistical and Practical Significance ' - jensen

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Statistical and Practical Significance

Petr Soukup

• Reminder of statistical significance

• Limits of statistical significance

• Misuses of statistical significance

• Alternatives to statistical significance

• Practical significance

• Effect sizes

• Tested hypothesis in experiments (Fisher, 1925)

• Null and alternative hypothesis (NHST) (Neyman&Pearson, 1937)

• Common tests - t-tests, analysis of variance, analysis of covariance, correlation analysis etc.

Definition: Conditionalprobability, that our sample can

be drawn from population in which null hypothesis

is valid (α). Statistical significance is P(D/H0) and not P(H0/D)

LIMITSOF NHST

• Big big probability samples from infinite or very big finite populations

Three assumptions:

• Big (infinite) population (at least 100times bigger than the sample)

• Probability sampling (all units same probability of selection)

• Big sample (> 30-50 units)

• 1.data from censuses

• 2. data from non-probability samples

• 3. data from small samples

• 4. data based on sample that are big proportion of the basic population

• 5. big data samples from merged (internationally or by time) files

N=32 articles, Czech sociological review 2000-2006 (selected 29 issues), own research

*CSR-Czech sociological review

MISUSES OF NHST

b) null hypotheses are unreal (nill null),

c) mechanical usage of classical 5% statistical significance (asterisks, stepwise methods, best models etc.),

d) statistical significant doesn’t mean important,

e) publishing only statistical significant results (file drawer problem).

N=32 articles, Czech sociological review 2000-2006 (selected 29 issues), own research

*CSR-Czech sociological review

ALTERNATIVESTO NHST

a) Confidence Intervals (Problems for r, formulas, regression etc.)

b) Test power (quite good in sociology),

c) Estimate of minimum sample size & What if strategy,

d) Comparison of models via information criterias (AIC, BIC)

e) Bayesian approach

• Practical significance

b) Substantive significance

c) Logical significance

d) Scientific significance

sometimes also:

e) result importance or

f) result meaningfulness

History - Absolute and relative approach

Example: Income differencies

Absolute and relative difference

Effect sizes – measures of practical significance

Some well known:

Cohen d

Hayes ω

But also R2, r, C, Fisher η2 are effect sizes

Problem: Sometimes published but not interpreted

• Economic significance

• Clinical significance

• Etc.

Statistical significance is:

LIMITED

MISUSED