AVERAGE OF DELTA – A NEW CONCEPT IN QUALITY CONTROL

1. AVERAGE OF DELTA – A NEW CONCEPT IN QUALITY CONTROL GRD Jones Department of Chemical Pathology, St Vincents Hospital, Sydney, Australia APCCB 2004

2. Background • The Average of Normals (AON) is an accepted QC process for clinical laboratories. • AON is the average of a set number of patient results, usually within set limits (eg normal range). • The AON rule “fires” when the function exceeds a pre-set limit (eg 2.5 x analytical CV). • Delta checks are the comparison of a result with a previous result from the same patient. • Delta checks are used to detect blunders or other errors • I combine these concepts to produce theAverage of Delta(AOD) a new QC tool for clinical laboratories. APCCB 2004

3. Terminology • A Delta Valueis a recent patient result minus the preceding result for that patient. • An AOD function is the average of a series of delta values. • AODN is an AOD function averaging N delta values. • NAOD is the number of samples included in an AOD function. • N90 is the number of samples with valid previous result, required to detect a change in assay bias with 90% probability using an AOD function. • CVwi is the within-individual Biological Variation. • CVa is the analytical variation expressed as a CV. • SDAOD is the SD of the AOD function APCCB 2004

4. Methods • AOD functions were modelled in a spreadsheet application using Microsoft Excel. • Variations in CVa and CVwi were modelled using the random number generator with a Normal distribution. • Models were based on 100 data sets, each of 110 delta values. • Factors adjusted in the model were: • the ratio of Cva/Cvwi • NAOD • Bias changes in assay performance. APCCB 2004

5. Modelling Equations • Data sets were generated for various values of CVa and CVwi with the variation (CVdata set) in results described as follows CVdata set = SQRT(CVa2 + CVwi2) • Second data sets were independently generated using the same values for CVa and CVwi • Delta Values were were obtained by subtracting the data points from the second data set from those in the first to produce a series of delta values. • Changes in bias were modelled by addition of fixed amounts to the delta values at a fixed point in the data set. • AOD functions were set to trigger if a data point fell outside limits defines by +/- 2.5 SDAOD. APCCB 2004

6. AOD Functions Figure 1 • AOD functions for various values of NAOD.CVa = 0.1, CVwi = 0.2 • As the value of N increases, the scatter of the AOD function decreases. • The decrease in SD with increasing N is equal to dividing by the square root of N. (data not shown) AOD value Sample Number Purple: n=2 SD = 0.22 Red: n=10 SD = 0.10 Blue: n=50 SD = 0.045 APCCB 2004

7. Effects of Bias on AOD Functions • A fixed bias was added after delta value 10 in each data set. • The AOD function followed the change in bias with the following features: • With smaller values of NAOD, the response occurred more rapidly, but was smaller relative to the scatter of the AOD function • With higher values of NAOD, the response was slower, but was larger relative to the scatter of the AOD function. • Examples are shown in figure 2. APCCB 2004

8. Figure 2. • AOD functions for NAOD of 2, 10 and 50. • CVa = 0.1, CVwi = 0.2 • A fixed bias of 0.3 was introduced at sample 10. • The red lines show the 2.5 x SDAOD. For AOD2 the value is 0.56; for AOD10, 0.24; for AOD 50, 0.11. • Five example AOD functions are shown in each graph ( ). • Blue arrows show N90. • Orange arrows show average first rule firing. AOD2 14 36 AOD10 10 20 AOD50 18 30 Sample Number APCCB 2004

9. Error Detection with AOD • Error detection of bias can be measured as the number of delta values (samples with previous results) required to detect changes in bias with specified certainty. • N90 and average first detection for a range of values for Cva/Cvwi and NAOD are shown in figure 3. • The following conclusions can be drawn: • Earlier error detection occurs with lower values of Cva/Cvwi • Error detection generally varies with NAOD in a “U-shape” with an optimal range of values depending on CVa/CVwi . • Examples of actual values for CVa/CVwi are in the table. APCCB 2004

10. A Figure 3 • Number of samples required for average 1st firing (A) and N90 (B) for detection of a shift of 2.5 x CVa for various values of CVwi/CVa and NAOD. • Earlier error detection with lower CVwi/CVa • Optimal error detection with NAOD 5-20 N90 Average 1st Firing CVwi/CVa B CVwi/CVa APCCB 2004

11. Table. • Examples of Cvwi / CVa . • CVwi from Westgard Website (www.westgard.com) • CVa from SydPath Laboratory (Olympus AU2700) APCCB 2004

12. Discussion • The limitation of AON is the ratio of group biological variation to analytical CV. • AOD may outperform AON if: • CVwi is small compared to between-person biological variation (a low Index of Individuality). • The frequency of samples with previous results is high. • Clinics or weekends affect AON results. Note than AOD should not be affected by change in patient mix as it uses patients as their own control. • AON may complement standard QC if it can: • Detect smaller errors than standard QC • Detect errors before standard QC • Allow less frequent use of standard QC APCCB 2004

13. Conclusions • Average of Delta may allow improved error detection without additional QC testing. • The process would most suit tests as follows: • A low within-individual biological variation compared to the analytical variation. • A high frequency of repeat testing. • Software programs must be written to further evaluate and this tool and allow for use in the routine environment. APCCB 2004