- 115 Views
- Uploaded on
- Presentation posted in: General

Measures of diagnostic accuracy

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

- Positive and negative predictive values
- Sensitivity and specificity
- Likelihood ratios
- Area under the ROC curve
- Diagnostic odds ratio

- Results from the index test are compared with the results obtained with the reference standard on the same subjects
- Accuracy refers to the degree of agreement between the results of the index test and those from the reference standard

Series of patients

Index test

Reference standard

Cross-classification

- Diagnostic value of B type natriuretic (BNP) measurement
- Does BNP measurement distinguish between those with and without left ventricular dysfunction in the elderly?
- Smith et al. BMJ 2000; 320: 906.

- Target population: unscreened elderly
- Index test: BNP
- Target condition: LVSD
- Final diagnosis (reference standard): echocardiography – global and regional assessment of ventricular function including measurement of LV ejection fraction

Elderly patients

BNP measurement

Echocardiography for LVSD

Cross-classification

- sensitivity
- 11 / 12 = 92% < Pr(T+|D+) >

- specificity
- 93 / 143 = 65% < Pr(T-|D-) >

- positive predictive value
- 11 / 61 = 18% < Pr(D+|T+) >

- negative predictive value
- 93 / 94 = 99% < Pr(D-|T-) >

- sensitivity
- 131 / 143 = 92%

- specificity
- 93 / 143 = 65%

- positive predictive value
- 131 / 181 = 72%

- negative predictive value
- 93 / 105 = 89%

- Test performance is sometimes observed to be different in different settings, patient groups, etc.
- Occasionally attributed to differences in disease prevalence, but:
- diseased and non-diseased spectrums differ as well.

- e.g. using a test in primary care and secondary care referrals
- the diseased group are different (cases more difficult)
- the non-diseased group are different (conditions more similar)
- sensitivity may decrease, specificity certainly decreases

- Why likelihood ratios?
- Applicable in situations with more than 2 test outcomes
- Direct link from pre-test probabilities to post-test probabilities

- Information value of a test result expressed as likelihood ratio

- How more often a positive test result occurs in persons with compared to those without the target condition

- Likelihood ratio of a negative test result
- How less likely a negative test result is in persons with the target condition compared to those without the target condition

- A LR=1 indicates no diagnostic value
- LR+ >10 are usually regarded as a ‘strong’ positive test result
- LR- <0.1 are usually regarded as a strong negative test result
- But it depends on what change in probability is needed to make a diagnosis

92%

LR+ = 10

55%

10%

50%

- Still useful when there are more than 2 test outcomes

- Dichotomisation of BNP(high vs. low) means loss of information
- Higher values of BNP are more indicative of LVSD

- Stratum specific likelihood ratios in case of more than 2 test results

Post-test odds for disease

=

Pre-test odds for disease x Likelihood ratio

- Pre-test odds
- chance of disease expressed in odds
- example: if 2 out of 5 persons have the disease: probability = 2/5 in odds = 2/3

- odds = probability / (1 – probability)
- probability = odds / (1 + odds)

- Pre-test probability = 0.5
- Pre-test odds = 0.5 / (1-0.5) = 1
- LR(BNP >26.7) = 3.83
- Post-test odds = 1x3.83 = 3.83
- Post-test probability = 3.83 / (1+3.83) = 0.79

- Pre-test probability = 0.5
- Pre-test odds = 0.5 / (1-0.5) = 1
- LR(CK< 40) = 0.13
- Post-test odds = 1 x 0.13 = 0.13
- Post-test probability = 0.13 / (1+0.13)
= 0.12

79%

52%

12%

50%

5%

17%

5%

1%

- Sample uncertainty should be described for all statistics, using confidence intervals

+ gives upper limit - gives lower limit

Standard error of estimate

estimate of effect

Normal deviate (1.96 for 95% CI)

- Sensitivity, specificity, positive and negative predictive values, and overall accuracy are all proportions

- Asymptotic CI are approximations
- Inappropriate when
- proportion is near 0% or near 100%
- sample sizes are small
(confidence intervals are not symmetric in these cases)

- Preferable to use Binomial exact methods
- can be computed in many statistics packages
- or refer to tables

- Odds ratios are ratios of odds

- Likelihood ratios are ratios of probabilities

- Sensitivity = 92% (62%, 100%)
- Specificity = 65% (57%, 73%)
- PPV = 82% (70%, 91%)
- NPV = 99% (94%, 100%)
- LR(>= 26.7) = 3.8 (2.4, 6.1)
- LR(18.7 < 26.7) = 1.1 (0.3, 4.1)
- LR(<18.7) = 0.13 (0.02, 0.84)

- ROC stands for Receiver Operating Characteristic
- ROC-curve shows the pairs of sensitivity and specificity that correspond to various cut-off points for the continuous test result

Decreasing threshold increases sensitivity but decreases specificity

Increasing threshold increases specificity but decreases sensitivity

Cut-off: 18.7

Cut-off: 19.8

Cut-off: 26.7

- Shows the effect of different cut-off values on sensitivity and specificity
- Better tests have curves that lie closer to the upper left corner
- Area under the ROC is a single measure of test performance (higher is better)
- Shape
- RAW continuous data gives steps
- GROUPED data gives straight sloping lines
- FITTED ROC curves are smoothed.

At what level, is a test result categorised as +ve, and how should the threshold be selected?

Threshold affects the performance of the test, as described by ROC curves, and likelihood ratios

Depends on

disease prevalence (affects +ve and -ve predictive values)

relative costs of false positive and false negative misdiagnoses

relative benefits of true positive and true negative diagnoses

LVSD

+ve

-ve

MI or BNP

+ve

36

63

-ve

4

23

40

86

- Q16 page 8
Compute post-test probabilities for a high risk patient, pre-test prob=50%

Q19 page 10