issues about statistical inference n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Issues About Statistical Inference PowerPoint Presentation
Download Presentation
Issues About Statistical Inference

Loading in 2 Seconds...

play fullscreen
1 / 19

Issues About Statistical Inference - PowerPoint PPT Presentation


  • 78 Views
  • Uploaded on

Issues About Statistical Inference. Dr R.M. Pandey Additional Professor Department of Biostatistics All-India Institute of Medical Sciences New Delhi.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Issues About Statistical Inference' - maren


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
issues about statistical inference

Issues About Statistical Inference

Dr R.M. Pandey

Additional Professor

Department of Biostatistics

All-India Institute of Medical Sciences

New Delhi

slide2

We study a sample to get an observed value of a statistic representing an interesting effect, such as the relationship between physical activity and health or performance.

  • But we want the true (= population) value of the statistic
  • The observed value and the variability in the sample allow us to make an inference about the true value.
  • Use of the P value and statistical significance is one approach to making such Statistical Inferences.
slide3

Hypothesis Testing:

  • Based on notion that we can disprove, but not prove, things.
  • Therefore, we need something to disprove.
  • Let's assume the true effect is zero: the null hypothesis.
  • If the value of the observed effect is unlikely under this assumption, we reject (disprove) the null hypothesis..
  • P < 0.05 is regarded as unlikely enough to reject the null hypothesis (i.e., to conclude the effect is not zero).
    • We say the effect is statistically significant at the 0.05 or 5% level.
    • Some also say "there is a real effect".
  • P > 0.05 means not enough evidence to reject the null.
    • Some accept the null and say "there is no effect".
    • We say the effect is statistically non-significant(NS)
slide4

Problems with this philosophy

    • We can disprove things only in pure mathematics, not in real life.
    • Failure to reject the null doesn't mean we have to accept the null.
    • In any case, true effects in real life are never zero.
    • So, THE NULL HYPOTHESIS IS ALWAYS FALSE!
    • Therefore, to assume that effects are zero until disproved is illogical, and sometimes impractical
    • 0.05 is arbitrary.
  • The answer? We need better ways to represent the uncertainties of real life:
    • Better interpretation of the classical P value
    • More emphasis on (im)precision of estimation, through use of confidence limits for the true value
    • Better types of P value, representing probabilities of clinical or practical benefit and harm
slide5

R.A.Fisher:

  • Use of p-values in isolation for decision making is inappropriate
  • We can’t “prove” the null hypothesis
  • “ the null hypothesis is never proved or established, but is possibly disapproved. Every experiment may be said to exist only in order to give the facts a chance of disproving the null hypothesis”.
slide6

Why Sample Size is an issue in hypothesis testing?.

  • What stage of Research should it be considered?
  • Sample Size Calculation Depends On
    • Type of Study Design
    • Choice of outcome variable
    • Magnitude of anticipated primary outcome
    • Statistical Considerations
  • (Choice of confidence level and Power)
  • Implications of large or small sample size
    • When ‘n’ is more than adequate
    • When “n” is inadequate
slide7

Problems with small sample sizes

  • Survey of 71 completed studies with negative findings (Freiman)
  • Power:
    • 67 had 10% of missing a 25% effect
    • 50% had >10% chance of missing 50% effect
  • Treatment Effect:
    • 57 had CIs consistent with 25% effect
    • 34 consistent with 50% effect
slide8

Sample size:

    • increases as alpha, beta, or delta decrease
    • increases as SD increase for continuous data
    • choice between 1- and 2-sided tests
    • using 1-sided test allows use of alpha instead of alpha/2, and hence smaller n
slide9

probability distributionof true value given the observed value

Area = 0.95

probability

observed value

lower likely limit

upper likely limit

0

negative

positive

value of effect statistic

  • CI limits of the true value define a range within which the true value is likely to fall.
    • "Likely" is usually a probability of 0.95 (defining 95% limits).
  • Problem: 0.95 is arbitrary and gives an impression of imprecision.
  • Problem: still have to assess the upper and lower limits and the observed value in relation to clinically important values.
slide10

P/2 is the probability that the true value is negative.

    • Example: P = 0.24

probability distributionof true value given the observed value

probability

(P value)/2= 0.12

observedvalue

0

negative

positive

value of effect statistic

  • Easier to understand, and avoids statistical significance, but…
  • Problem: having to halve the P value is awkward, although we could use one-tailed P values directly.
  • Problem: focus is still on zero or null value of the effect.
slide11

Statistical significance focuses on the null value of the effect.

  • More important is clinical significance defined by the smallest clinically beneficial and harmful values of the effect.
    • These values are usually equal and opposite in sign.
    • Example:

smallest clinicallyharmful value

smallest clinicallybeneficial value

observed value

0

negative

positive

value of effect statistic

  • We now combine these values with the observed value to make a statement about clinical significance.
slide12

The smallest clinically beneficial and harmful values help define probabilities that the true effect could be clinically beneficial, trivial, or harmful (Pbeneficial, Ptrivial, Pharmful).

probability

Pbeneficial= 0.80

Ptrivial= 0.15

observed value

Pharmful= 0.05

0

negative

positive

value of effect statistic

  • These Ps make an effect easier to assess and (hopefully) to publish.
    • Warning: these Ps areNOT the proportions of+ ive, non- and - iveresponders in the population.
  • The calculations are easy.
  • More challenging: choosing the smallest clinically important value, interpreting the probabilities, and publishing the work.
slide13

Ex.: P = 0.20 for an observed positive value

    • If the true value is zero, there is a probability of 0.20 of observing a more extreme +ive or –ive value.

probability

probability distributionof observed valueif true value = 0

P value =0.1 + 0.1

observedvalue

0

negative

positive

value of effect statistic

  • Problem: Hard to understand.
slide14

Examples of Clinical Significance

  • Examples for a minimum worthwhile change of 2.0 units.
  • Example 1–clinically beneficial, statistically non-significant(inappropriately rejected by editors):
    • The observed effect of the treatment was 6.0 units (90% likely limits –1.8 to 14 units; P = 0.20).
    • The chances that the true effect is practically beneficial/trivial/harmful are 80/15/ 5%.
  • Example 2–clinically beneficial, statistically significant (Accepted)
    • The observed effect of the treatment was 3.3 units (90% likely limits 1.3 to 5.3 units; P = 0.007).
    • The chances that the true effect is practically beneficial/trivial/harmful are 87/13/0%.
slide15

Example 3–clinically unclear, statistically non-significant(the worst kind of outcome, due to small sample or large error of measurement; usually rejected, but could/should be published to contribute to a future meta-analysis):

    • The observed effect of the treatment was 2.7 units (90% likely limits –5.9 to 11 units; P = 0.60).
    • The chances that the true effect is practically beneficial/trivial/harmful are 55/26/18%.
  • Example 4–clinically unclear, statistically significant(good publishable study; true effect is on the borderline of beneficial):
    • The observed effect of the treatment was 1.9 units (90% likely limits 0.4 to 3.4 units; P = 0.04).
    • The chances that the true effect is practically beneficial/trivial/harmful are 46/54/0%.
slide16

Example 5–clinically trivial, statistically significant(publishable rare outcome that can arise from a large sample size usually misinterpreted as a worthwhile effect):

    • The observed effect of the treatment was 1.1 units (90% likely limits 0.4 to 1.8 units; P = 0.007).
    • The chances that the true effect is practically beneficial/trivial/harmful are 1/99/0%.
  • Example 6–clinically trivial, statistically not significant(publishable, but sometimes not submitted or accepted):
    • The observed effect of the treatment was 0.3 units (90% likely limits –1.7 to 2.3 units; P = 0.80).
    • The chances that the true effect is practically beneficial/trivial/harmful are 8/89/3%.
slide17

Sparse Data:

  • In a contingency table:
  • Frequencies less than 5
  • due to small sample size
  • rare outcome
  • In Chi-square: get warning message
  • Reason: Asymptotic methods, which invoke Central Limit Theorem, are not reliable when sparse data, skewed, heavily tied.
slide18

Way out:

  • Categorical outcome variable:
    • Collapse the categories, but could be problematic!!!.
    • Fisher’s exact test
  • Quantitative Outcome variable:
    • Appropriate transformation
    • Non-parametric methods of hypothesis testing