Evidence Based Medicine: Review of the basics. Deb Bynum, MD August 2010. Moving beyond sensitivity and specificity. Hierarchy of Strength of Evidence. N of 1 randomized controlled trial Systematic reviews of randomized controlled trials Single randomized trial
Deb Bynum, MDAugust 2010
N of 1 randomized controlled trial
Systematic reviews of randomized controlled trials
Single randomized trial
Systematic review of observational studies addressing patient important outcomes
Single observational study addressing patient important outcomes
Unsystematic clinical observations
1. Are the results of the study valid?
2. What are the results?
3. How can I apply these results to patient care?
(1-Y/X) x 100
Why is this the most commonly reported measure of treatment effect?
How can this be misleading????
Pink Potion is new on the market, and is a wonder drug according to Big Bucks Pharmaceutical Company. When taken every day for 20 years, it decreases the risk of developing cell phone related brain cancer by 50% ! (RRR is 50%)
You find the risk of such cancers to be 0.25 % in one high risk population. With this 20 year treatment, 0.13% of the population developed the cancer…
Calculate ARR whenever possible – this often will put the overall treatment effect into perspective
Most reports (especially when advertising/promoting a treatment) will report effects as RRR…
Number of patients who must receive a treatment/intervention to prevent one bad outcome or produce one positive outcome
(Pink Potion: NNT = 1/.12% = 1/.0012= 833== need to treat 833 people for 20 years to prevent one case of cell phone related brain cancer…)
Absolute risk of adverse outcome with treatment – risk of adverse outcome without treatment
NNH = 1/ absolute difference in adverse outcomes (just like NNT)
Weight NNT with NNH….
RR= a/(a+b) / c/(c+d)
OR = (a/b) / (c/d)
If outcome is infrequent in both treatment and control groups, then OR and RR will
Be nearly the same
Probability of diagnosis: 0-100%
Probability below test threshold: no testing
Probability in between : test
Probability above treatment threshold: no testing, treat
Sensitivity: If the patient has the disease, how likely is it that he will
have a+ test?
“rules out” disease (not really….)
If the patient does not have the disease, how likely is he to have
a negative test?
“rules in” disease (again, not really….)
In real life, the question is “the patient has a positive test, how likely is it that he has the disease? Or the patient has a negative test, how likely is it that he does not have the disease?”
Positive Predictive Value:
How many (%) people with a positive test will have the disease?
PPV = a/ a+b
Negative Predictive Value:
How many people with a negative test will NOT have the disease?
NPV = d/c+d
Problem: depend upon prevalence (low prevalence population, positive test is more
Likely to be a false positive; high prevalence, a negative more likely to be a false negative)
The likelihood that disease is present given X test result (positive, negative, intermediate, 250)
LR: How many patients with X test result HAVE disease compared to number of patients with X test result who do NOT have disease
X test result can be positive, negative, intermediate, a number
LR always looks at ONE certain test result and always compares likelihood of having disease to likelihood of not having disease
Likelihood that disease is present given a “positive” test
How many patients with a + test HAVE disease compared to # patients with a + test who do NOT HAVE disease
True Positive Rate/False Positive Rate
LR + test: a /(a+c) / b/(b+d)
Higher # (over 10) : better predicting
LR + “infinity” = 100% specificity (if the test is positive, the patient has disease, no false positives)
Likelihood that disease is present given a negative test
How many patients with a Negative test HAVE disease compared to # patients with a Negative Test who DO NOT HAVE disease
False negative rate/true negative rate
c/(a+c) / d/(b+d)
Smaller number = better test (fewer patients with a negative test HAVE disease compared to DO NOT HAVE disease)
LR – of <.10 usually signficant
LR – of 0 = 100% sensitivity (if the test is negative, the patient does NOT have disease – rules “out” disease…
LR + test: a/(a+c) / b/(b+d)
LR – test: c/(a+c) / d/(b+d)
LR is ODDS note a %
Determine pretest probability (ok to estimate)
Determine pre test ODDS (odds= probability/1-probability)
Determine Post test ODDS: pretest odds xLR
Convert post test ODDS back to probability