Loading in 5 sec....

Statistical and Practical Significance PowerPoint Presentation

Statistical and Practical Significance

- 140 Views
- Uploaded on
- Presentation posted in: General

Statistical and Practical Significance

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Statistical and Practical Significance

Advanced Statistics

Petr Soukup

- Reminder of statistical significance
- Limits of statistical significance
- Misuses of statistical significance
- Alternatives to statistical significance
- Practical significance
- Effect sizes

REMINDER OF STATISTICAL SIGNIFICANCE (NHST)

- Tested hypothesis in experiments (Fisher, 1925)
- Null and alternative hypothesis (NHST) (Neyman&Pearson, 1937)
- Common tests - t-tests, analysis of variance, analysis of covariance, correlation analysis etc.

Definition: Conditionalprobability, that our sample can

be drawn from population in which null hypothesis

is valid (α). Statistical significance is P(D/H0) and not P(H0/D)

LIMITSOF NHST

- Big big probability samples from infinite or very big finite populations
Three assumptions:

- Big (infinite) population (at least 100times bigger than the sample)
- Probability sampling (all units same probability of selection)
- Big sample (> 30-50 units)

- 1.data from censuses
- 2. data from non-probability samples
- 3. data from small samples
- 4. data based on sample that are big proportion of the basic population
- 5. big data samples from merged (internationally or by time) files

N=32 articles, Czech sociological review 2000-2006 (selected 29 issues), own research

*CSR-Czech sociological review

MISUSES OF NHST

a) Insufficient statement about population,

b) null hypotheses are unreal (nill null),

c) mechanical usage of classical 5% statistical significance (asterisks, stepwise methods, best models etc.),

d) statistical significant doesn’t mean important,

e) publishing only statistical significant results (file drawer problem).

N=32 articles, Czech sociological review 2000-2006 (selected 29 issues), own research

*CSR-Czech sociological review

ALTERNATIVESTO NHST

a) Confidence Intervals (Problems for r, formulas, regression etc.)

b) Test power (quite good in sociology),

c) Estimate of minimum sample size & What if strategy,

d) Comparison of models via information criterias (AIC, BIC)

e) Bayesian approach

PRACTICAL SIGNIFICANCE

- Practical significance
b) Substantive significance

c) Logical significance

d) Scientific significance

sometimes also:

e) result importance or

f) result meaningfulness

History - Absolute and relative approach

Example: Income differencies

Absolute and relative difference

Effect sizes – measures of practical significance

Some well known:

Cohen d

Hayes ω

But also R2, r, C, Fisher η2 are effect sizes

Problem: Sometimes published but not interpreted

OTHER SIGNIFICANCES

- Economic significance
- Clinical significance
- Etc.

Statistical significance is:

LIMITED

MISUSED

BUT NOT BAD

Substantive significance is:

NOT OFTEN USED

BUT NECESSARY