Dynamic Calibration and Validation Testing

1. Dynamic Calibration and Validation Testing

2. Why is Dynamic Calibration Required • Factory calibration calibrates each detector so it is free of inter- element correlations and determines the factors in the A3 model that enhances the performance of Thermo Gamma-Metrics coal analyzers. • But the factory can not exactly match the field conditions; therefore fine tuning of the calibration is required in the field. • Sources used in the field are different from what was used in the factory • Coal stream is continuous through the unit in the field while it is a finite length (equal to the calibration standard length) in the factory. • The range of ash and sulfur are narrower in the field therefore the analyzer can be tuned to be more accurate over the narrower range. • The analyzer must be calibrated to the reference sample system.

3. Precision and Accuracy and Catch 22 Neither Precise nor Accurate Precise but not Accurate Precise and Accurate Bias Adjustment Needed The Catch 22 is we don’t know what the location of the bull’s eye is. We are all measuring it with a number of systems each with their variances. Typically an analyzer customer thinks his sample system and lab have no error and the analysis results from the sample system/lab is the bull’s eye for an analyzer.

4. The objective • During the field calibration and validation process, the objective is to calibrate the analyzer to match the results of the customer’s reference system, usually his sample system and lab. • The analyzer measures the quality of all the coal that passes through it. • The lab analyzes a very small subset of the material on the belt to determine the quality of a large lot (usually one hour) of coal flow. • It is critical that the sample analyzed in the lab is “representative” of the coal on the belt seen by the analyzer. • It is critical that the lab prepare and analyze samples with precise adherence to standard procedures,. with well maintained equipment and well trained people.

5. Sample collection • Collection of each gross sample is controlled by ASTM Standards D2234 (Collection of a Gross Sample of Coal) and D2013 (Preparing Coal Samples for Analysis) in the USA. • D2234 covers collection of a gross sample without any crushing. • D2013 covers the reduction and division of the gross sample collected according to D2234 with crushing. • To calibrate an analyzer generally requires 30 gross samples covering a range of coal quality. • At least 20 gross samples are required to validate the analyzer calibration.

6. Minute-by-minute sulfur and ash data for a unit train loading. Loading from High Sulfur Pile Dozer Pushing from High Ash Pile D 2234 defines how many increments are required to accurately represent the quality of this train.

7. The highlights of ASTM D 2234 collection of a gross sample. • Number of increment samples required to make up a single gross sample? • For coal with consistent quality (clean coal from a single seam) 15 increments up to a 1000 ton lot size. • For inconsistent coal (raw coal, partial wash coal, any blend coal) 35 increments up to a 1000 ton lot size. • For lot sizes greater than 1000 tons, the number of increments required is; N1 = minimum increments shown above N2 = number of increments required for lot size

8. Example of number of increments required • Loading a unit train • Blended Coal dozed from different stacking tube piles; N1 = 35 • The unit train capacity is 10,000 tons • Therefore • 111 individual increments are needed to represent the coal on the belt.

9. Impact of number of increments on the repeatability and how well the sample represents the total lot. As these graphs show, it is critical that enough increments are taken to accurately represent the quality of the lot of coal. If too few increments are taken, the random error in the sample collection process will negatively impact the calibration of the analyzer.

10. The highlights of ASTM D 2234 collection of a gross sample, continued. • When a sample increment is taken the sampling device must have an opening at least 2-1/2 to 3 times the top size of the coal, but never smaller than 1.25-inches. This applies to stop belt sampling, falling stream cutters or sweep samplers. Falling material must strike the sample device at right angles to the opening. • In addition the cutter speed is specified (18inches/second) and the minimum amount of sample that has to be taken, based on the material top size is also specified.

11. The highlights of ASTM D 2234 collection of a gross sample, continued. • If a very large sample increment is collected in the primary sample, it is often required that the primary sample be reduced in size with secondary sampling before a crushing step is undertaken. • At least six secondary increments are required for each primary increment. • All the rules concerning minimum increment weight, cutter speed and cutter opening apply to this secondary sampler. • Once the sample has passed through a crusher, ASTM D 2013 defines the sampling procedures.

12. ASTM D 2013 Preparing coal samples for analysis • D 2013 covers both the manual and the mechanical reduction and devision of samples collected according to D 2234. • Applies to sample reduction after crushing in the sample system • Applies to the lab manual reduction of the sample collected by a sample system • At least 60 increments are required from the sample taken by D 2234. Good practice recommends that at least one tertiary cut be taken for every primary cut if this is a larger number than 60. • The minimum sample size required is determined by how fine the coal is crushed and whether the coal is 100% clean coal from one seam or is it a variable coal quality (raw coal or blends).

13. ASTM D 2013 Preparing coal samples for analysis, continued Preparation of Laboratory Samples Table shows the minimum weight of the final sample depending on the material size following crushing. Group A = “coals that have been cleaned in all sizes” Group B = “All other coals including unknown”

14. ASTM D 2013 Preparing coal samples for analysis, continued • Good practice in a sample system • Cutter opening after crushing 1-¼ inch minimum opening • Maximum cutter speed 18 inches/sec. • Final sample weight collected by the sample system about 40 pounds. • ASTM D 2013 manual sample reduction procedures must be rigorously followed in the lab sample preparation process to insure the sample is not biased or contaminated in the process. • Errors made in sample collection or preparation will invalidate a sample.

15. Dynamic Calibration of the Analyzer • Dynamic calibration of the analyzer involves calibrating the analyzer so it matches the reference system (the sample system and the lab). • The Moisture, Sulfur, Ash Oxides and the Ash are calibrated by plotting the analyzer results against the reference system.

16. Dynamic Calibration of the Analyzer • As the X-Y plot of the sulfur calibration data showed, PGNAA responses are linear; therefore, the field calibration of the analyzer will be the equation of the linear regression fitting the analyzer raw results to the reference system results. • Before the linear regression is performed, the data should be screened for outliers (data that does not match the trend seen with the rest of the data). • Outliers need to be addressed before the analyzer is calibrated.

17. Handling Outliers in the Data • Errors can happen in the collection of samples, problems in the lab, improperly recording values, etc. therefore it is not unexpected to see outliers in a data set. • If an outlier is seen, an effort needs to be made to determine is there an attributable reason. In the data below the sample weight for all samples except the outlier were 40-pounds ± 4. The sample weight of the outlier was 8-pounds. Something must have happened to the sample system. Therefore this point was discarded and not used in the calibration. Outlier data point discarded from the data set because it was determined the collected sample weight was too small to be representative of the tons sampled.

18. Dynamic calibration The dynamic calibration is completed by applying the gain (in this case 0.8108) and offset (+0.0642) determined to the analyzer calibration. This is done for the ash and moisture too, completing the dynamic calibration of the analyzer.

19. Validation Test • Once the analyzer has been dynamically calibrated the validation test can be performed. • The validation test ideally consists of the collection of 30 samples, collected and analyzed in exactly the same manner as the dynamic calibration samples. • As the normal guarantees are only for ash, sulfur and often moisture, the ash oxides only need to be run if the customer is specifically interested in them. • The Root Mean Square Deviation (RMSD) of the analyzer to the lab data is calculated. n = number of data sets

20. RMSD example

21. Acceptance of Analyzer Performance • The analytical precision of the analyzer is guaranteed by Thermo. • RMSD measures the combined error of the analyzer and the sample system/lab reference system. • If the assumption is made that the analyzer and the sample system/lab have approximately the same precision (an assumption that has been proven to be often true). Then the precision of the analyzer can be estimated as:

22. Acceptance continued • If the measured RMSD of the analyzer is equal to or less than the precision guarantee, the analyzer has passed the acceptance test. • If the RMSD is larger than the guarantee, the analyzer precision may still satisfy the guarantee. • If the lab and analyzer precision assumed to be equal, an RMSD of 0.707 would indicate an analyzer precision of 0.50. • If the precision of the analyzer is in question, then ASTM D 6543 “Evaluation of Measurements Made by On-Line Coal Analyzers”, details the procedure to conduct a Grubbs Estimator measurement of the precision of the analyzer.

23. Grubbs Estimator • Requires dual ASTM sampling of the coal stream at two different locations. • Requires the collection of 60 samples from each sample location • Requires the samples to be processed completely by independent labs. • The result is an estimate of the precision of the three analysis systems, the coal analyzer, sample collection/lab one and sample collection/lab two. • The Grubbs estimate of the precision of the analyzer is what is compared to the guarantee performance. • Thermo’s guarantee is a one sigma guarantee (68% of the results will fall within plus or minus the guarantee value), and sukbject to statistical significance testing.

24. Precision of Sample Collection • How well does the sample collected represent the true quality of the lot? • What is the precision of the sample collected by the sample system? • For the example in slide 9 the measured ash had a precision of ±0.60 and the sulfur precision was ±0.036 at the ASTM recommended increment. • This number will be less for coal streams with less variability and more if the variability is greater. • The sample collected by the sample system goes back to the lab and is air dried, crushed and manually reduced to 500 grams that goes into the lab. • The lab then runs very small subsets of the prepared sample in a number of instruments.

25. Reproducibility Lab to Lab • Repeatability is the ability of a single lab to duplicate the same results when repeating the same analysis of the same sample. • Reproducibility is how well the results from two different labs should compare for the same sample. • As the coal analyzer is one “lab” we should look at the reproducibility of results on analysis system to another 95% of the time (2 sigma). • D 3173 Moisture Reproducibility - 0.3% for coals with less than 5% moisture and 0.5% for coals with more than 5% moisture. • D 3174 Ash Reproducibility – 0.32% on a dry basis • D 4239 Sulfur Reproducibility – 0.02 + 0.09(average sulfur between labs)