calibration in environmental analysis issues and proposals for improvement richard burrows l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Calibration in Environmental Analysis Issues and Proposals for Improvement Richard Burrows PowerPoint Presentation
Download Presentation
Calibration in Environmental Analysis Issues and Proposals for Improvement Richard Burrows

Loading in 2 Seconds...

play fullscreen
1 / 55

Calibration in Environmental Analysis Issues and Proposals for Improvement Richard Burrows - PowerPoint PPT Presentation


  • 238 Views
  • Uploaded on

Calibration in Environmental Analysis Issues and Proposals for Improvement Richard Burrows. FSEA May 2008. ©2007, TestAmerica Analytical Testing Corp.  All rights reserved. TestAmerica & Design TM are trademarks of TestAmerica Analytical Testing Corp. Premise 1.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Calibration in Environmental Analysis Issues and Proposals for Improvement Richard Burrows' - onofre


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
calibration in environmental analysis issues and proposals for improvement richard burrows

Calibration in Environmental AnalysisIssues and Proposals for ImprovementRichard Burrows

FSEA May 2008

©2007, TestAmerica Analytical Testing Corp.  All rights reserved.

TestAmerica & Design TM are trademarks of TestAmerica Analytical Testing Corp.

premise 1
Premise 1
  • The impact of calibration models on the ability to detect and quantify analytes is substantial.
    • Nonetheless, use of the appropriate calibration models is poorly understood and poorly controlled, and in many cases we are instructed to use calibration models that produce false positives, false negatives, and wildly inaccurate quantitation.
instrument calibration
Instrument Calibration
  • What do we want from the calibration?
    • Accurate translation of instrument response to analyte amount
    • Minimize the errors introduced by the calibration itself
relative vs absolute errors
Relative vs. Absolute Errors
  • How does the system behave?
  • Which kind of error are we measuring with our QC?
  • Which is more important from the risk standpoint?
what do we care most about
What do we care most about?

Calibration curve 1-100ppb

  • Do we prefer and expect:
    • +/- 5ppb at all levels (Absolute error)
    • +/- 10% at all levels (Relative error)
slide6
Risk

The difference in risk level between a concentration of 100 and 110 is small, but the difference between 0 and 1 may be very large.

characteristics of variance
Characteristics of Variance
  • Method 3520/8270, 8 replicates prepared and analyzed at 100ppb, 10ppb, 1ppb

Average of 84 analytes

Unweighted regression is valid onlyif the standard deviation is constant across the range.

what type of error
What type of Error?

We are concerned about relative error

We need a:

  • Calibration model that minimizes relative error
  • Measure for the calibration curve that evaluates how much relative error exists
epa calibration criteria
EPA calibration criteria
  • Linear regression is an option for almost all EPA methods
    • For linear regression, unweighted is usually the first option
    • Correlation coefficient is always the measure for a linear regression
  • Some methods have an option for average response factor
least squares curve fitting
Least squares curve fitting

Assumption: Constant Variance

weighted regressions
Weighted Regressions
  • Unweighted Σ [(predicted – actual)2]
  • 1/X Weighting Σ [[(predicted – actual)/ conc]2]
  • 1/X2 Weighting Σ [[(predicted – actual)/ (conc)2]2]
  • Weighted regressions tend to minimize relative error as opposed to absolute error
  • Weighted regressions are appropriate when variance is not constant (for example most environmental analysis method have fairly constant relative standard deviation (RSD) across the calibration range)
unweighted linear regression
Unweighted Linear Regression
  • Unweighted regressions minimize the absolute residuals
    • In a calibration from 1-100, an error (residual) of 5 at the 1.0 point has the same weight as an error of 5 at the 100 point.
  • 1/ (Conc)2 weighted regressions minimize the relative residuals
    • In a calibration from 1-100, an error (residual) of 5% at the 1.0 point has the same weight as an error of 5% at the 100 point.
calibration approaches
Calibration Approaches
  • SW-846 Method 8000B
    • ..begin with the simplest approach, the linear model through the origin, and progress through other options until calibration criteria are met
      • If RSD is < 20%, linearity through the origin may be assumed, and the average calibration or response factor may be used to determine sample concentrations
linear regression
Linear Regression
  • SW-846 Method8000B
    • If the RSD is > 20% then linearity through the origin cannot be assumed. In this case, the analyst may employ a regression equation that does not pass through the origin.
  • 8000C
    • Linear least squares regression may be employed based on past experience or at the discretion of the analyst.
weighting
Weighting
  • 8000B
    • The analyst may also employ a weighted least squares regression if replicate multi point curves have been performed 1/SD2
  • 8000C
    • Weighting may significantly improve the ability of the regression to fit the linear model to the data.
      • The mathematics used in the least squares regression has a tendency to favor numbers of larger value over numbers of smaller value. Thus the regression curves that are generated will tend to fit points that are at the upper calibration levels better than those points at the lower calibration levels.
method 8000c
Method 8000C
  • Weighting
    • Examples of weighting factors which can place more emphasis on numbers of smaller value are:
    • wi = 1/yi or wi = 1/yi2
    • These weighting factors are recommended if weighting other than wi = 1 is to be used
method 1631 guidance
Method 1631 Guidance
  • Weighting
    • “An unweighted regression is incorrect for nearly all instruments and analytical systems.”
    • “The calibration included a data point at the Method 1631 MDL (0.2 ng/L). The RSD for the CF/WR approach was 7.8 percent. The coefficient of determination (r2) for the unweighted approach was 1.000, indicating no error in calibration. The reason for the indication of zero error is that the low calibration points are, essentially, unweighted. Therefore, the unweighted regression is equivalent to a single-point calibration at the highest calibration point. We do not believe that this form of calibration is consistent with the best science.”
calibration criteria
Calibration criteria
  • Acceptance criteria
    • RSD must be less than 20 (15)%

or

    • r, COD or r2 must be greater or equal to 0.99
second premise
Second Premise

The Correlation coefficient (and the coefficient of determination) are pretty much useless for evaluating the suitability of a calibration curve.

correlation coefficient
Correlation Coefficient
  • Correlation Coefficient Pros
    • Included in most EPA methods
  • Correlation coefficient Cons
    • Not technically justified
    • Does not establish the best fit for environmental analysis
    • Cannot be compared to the RSE of an average RF
correlation coefficient26
Correlation Coefficient
  • For most applications, and calibration curves in particular, the correlation coefficient must be regarded as a relic of the past
    • Meier and Zund, Statistical Methods in Analytical Chemistry, 2000
correlation coefficient27
Correlation Coefficient
  • “The correlation coefficient in the context of linearity testing is potentially misleading and should be avoided”
    • Royal Society of Chemistry, Technical brief
  • “The author has seen cases where a correlation coefficient of 0.997 was believed to be a better fit than 0.996 of a 5 point calibration curve. One can even find requirements in quality assurance plans to recalibrate if the correlation coefficient is less than 0.995!”
    • Taylor, Statistical Techniques for Data Analysis, 1990
iupac
IUPAC
  • Guidelines for calibration in Analytical Chemistry, 1998
    • “The correlation coefficient which is a measure of relationship of two random variables, has no meaning in calibration….because the values x are not random quantities in the calibration experiment”
correlation coefficient29
Correlation Coefficient
  • “One practice that should be discouraged is the use of the correlation coefficient as a means of evaluating goodness of fit of linear models”
    • Van Arendonk and Skogerboe, Anal. Chem. 53, 1981, 2349-2350
calibration objectives
Calibration Objectives
  • The calibration model should minimize relative error
  • The calibration measure(s) should determine how well this objective is met
one additional requirement
One Additional Requirement
  • If we accept different ways of evaluating the curve, we want some consistency.
  • We don’t want one measure to say a curve is good, and another measure to say that it is bad
relative standard deviation
Relative Standard Deviation
  • RSD Pros
    • Simple
    • Evaluates relative error
    • Reasonable criteria established and included in many SW-846 methods
  • RSD Cons
    • Can only be applied to the average response factor type calibration – not suitable for linear or quadratic regressions
rsd and rse

= predicted response from curve

RSD and RSE

C = curve coefficient

x = Concentration

y = response

RSE = RSD when calculated for the average

COULD USE THE SAME CRITERIA

epa clarification memo on the use of sw 846 methods aug 7 1998
EPA clarification memo on the use of SW-846 methods, Aug 7 1998
  • “Further, the Agency recognizes that the relative standard error (RSE) is a useful measure of the goodness of fit of a calibration model that the Agency had not previously considered. The RSE is useful for both linear regression models as well as non-linear models, as it considers the error at each point in the calibration model as a function of the concentration of that standard.”
epa clarification memo on the use of sw 846 methods aug 7 199837
EPA clarification memo on the use of SW-846 methods, Aug 7 1998
  • “Using the RSE as a metric has the added advantage of allowing the same numerical standard to be applied to the calibration model, regardless of the form of the model. Thus, if a method states that the RSD should be <20% for the traditional linear model through the origin, then the RSE acceptance limit can remain 20% as well. Similarly, if a method provides an RSD acceptance limit of 15%, then that same figure can be used as the acceptance limit for the RSE.”
the calibration curve that can t fail a digression
The Calibration Curve that Can’t Fail! (A Digression)
  • “We really want to make sure we carefully define the low end of the curve.”
the impact of calibration models on analyte detection and accuracy at low concentrations
The Impact of Calibration Models on Analyte Detection and Accuracy at Low Concentrations
  • Example GC/MS Data
  • Example ICP/MS Data
  • Example IC data Data
gc ms data
GC/MS Data
  • Three calibration models,
    • Average response factor
    • Linear regression with no weighting
    • Linear regression with inverse square weighting.
  • If a sample gave the same response as our low standard, what would we detect and report?
icp ms data
ICP/MS Data
  • Compare Continuing Calibration Blank results using two different calibration models, linear regression without weighting and linear regression with 1/X weighting.
the test
The test
  • If the CCB result is greater than the MDL, you have a high risk of false positives
  • If the CCB result is less than the negative value of the MDL, you have a high risk of false negatives
weighted versus unweighted
Weighted versus unweighted
  • Unweighted – 50% fail test. CCB results are either > MDL or < -MDL
  • Weighted – 1.2% fail test. One high result for molybdenum
  • Same data, same instrument same sensitivity, only difference is calibration model
method 300
Method 300
  • In every case, the unforced, unweighted linear regression introduced error of over 200% for the low point of the curve.
  • Despite this, in every case the correlation coefficient for the unweighted unforced curve is better than 0.999!
  • The unweighted forced curve gives lower error
  • Unforced weighted curves (1/Concentration or 1/Concentration2) give the lowest error across the range of the calibration
  • The RSE does a good job of identifying which curves are acceptable for environmental analysis.
summary
Summary
  • The correlation coefficient and coefficient of determination are inadequate as QC measures for calibrations curves for environmental analysis
  • Relative Standard Error is one good measure that could be used and assesses relative error across the curve
  • Unweighted regressions are almost always bad – a good QC measure would indicate this
slide55

Questions?

richard.burrows@testamericainc.com

303-736-0100