Quality Audits

1. Quality Audits

2. Bernhard, G., and G, Seckmeyer, “Uncertainty of measurements of spectral solar UV irradiance”, J. Geophys. Res., Vol. 104, No. D12, 14,321-14,345 (1999). Gardiner, B.G., “QA in ultraviolet spectroradiometry: the state of the art”, WMO QC/QC working group. Heikkilä, A., et al., “SUVDAMA data storage Edition 4”, Final SUVDAMA Report (1999). Seckmeyer, G., et al., “Instruments to measure solar ultraviolet radiation Part 1: Spectral instruments”, version 18 from the Final SUVDAMA Report (1999). Webb, A., et al., “Guidelines for site quality control of UV monitoring”, WMO/GAW Rep. 126, World Meteorological Organization, Geneva, WMO TD No. 884, 39 pp. Supporting documentation

3. Test audit

4. Objectives (1) • To provide assurance to data users that a certain level of quality has been achieved “The site operator should be able to persuade an interested visitor that the measurement and collection of data are carried out diligently and to a standard that justifies any claims made to the quality of the data.”

5. Objectives (2) • To provide site operators with information that will allow them to improve the standard of their QC and QA procedures, and thus the value of their data The final report will attempt to identify and promote best practice

6. Requisites (1) • Mission statement The site operator should give a description of the site objectives in relation to the quantity and quality of data collected.

7. Requisites (2) • The site operator should be able to present a detailed and comprehensive description of the calibration, measurement and data analysis procedures, together with the results of their customary application. • The site operator should be able to make a quantitative statement about the accuracy of the measurements and be able to justify this assessment.

8. Checklist 5.Measurement regime 5.1.     standard operating procedures; 5.2.     type and quantity of data collected; 5.3.     resolution and freq. of spectral measurements; 5.4.     typical downtime for repairs/maintenance; 5.5.     documentation; 5.6.     demonstrable repeatability; 6.Data Analysis 6.1.    documentation; 6.2.    analysis of errors; 6.3.   correction for errors; 6.4.     use of ancillary measurements; 6.5.     estimates of relative/absolute errors; 6.6.     application of WMO guidelines; 6.7.     measurements of slit functions; 6.8.     procedures for the application of retrospective corrections; 6.9.     data storage; 6.10.  QC tools available and applied; 7.Quality control 7.1.     documentation; 7.2.     standard procedures; 7.3.     details of past/future instrument intercomparisons; 7.4.     details of data version control; 3.Instrumentation 3.1.     list & description of instruments; 3.2.     documentation of instrument characteristics; 3.3.     Freq. of regular maintenance & characterisation; 3.4.     maintenance record; 3.5.     past changes in instrumentation; 3.6.     planned changes to instrumentation; 3.7.     measurement of angular response; 3.8.     measurement procedure for slit function(s); 3.9.     photographs of instruments; 4.Calibrations 4.1.     documentation; 4.2.     certificates; 4.3.     frequency of calibrations; 4.4.     definition of fiducial plane; 4.5.     measurement of lamp distance; 4.6.     method for wavelength calibration; 4.7.     typical instrument drift between calibrations; 4.8.     evidence that the calibration is reliable at the quoted level of accuracy; 4.9.     age of lamps; 4.10.  level of agreement between lamps (internal consistency); 4.11.  level of repeatability/consistency in calibration; 4.12.  steps taken to eliminate stray light; 4.13.  evaluation with the JRC calibration equipment; 1.Location 1.1.   the longitude, latitude and altitude of site; 1.2.  the source and accuracy of these coordinates; 1.3.  a description of the local environment; 1.4. availability & accuracy of a horizon/alt map; 1.5.  a description of nearby and tall objects; 1.6.  site representative to local environment; 1.7.  the local climate; 1.8. photographs of horizon and measuring site; 2.Operational matters 2.1. site ‘mission statement’,; 2.2. details of who is responsible for what; 2.3. time dedicated to measurements; 2.4. % time spent on maintenance etc.; 2.5. % of operator’s working week on measurements; 2.6. the experience of staff; 2.7. how often staff change; 2.8. the training of new staff; 2.9. details of how often procedures have changed; 2.10. description of instrument characteristics; 2.11. the records of past intercomparisons; 2.12 description of any publications read/adopted; 2.13.  photographs of the operators;

9. Presentation of results (1)

10. Presentation of results (2)

11. Formal statement (1) • Brief summary of the audit • Containing: an overview (i.e., site objectives and practices), major findings, conclusions • The institute will have the opportunity to comment on this statement, and to append further information or comments they consider appropriate, before it is circulated • Blank sections left for future updates • The formal statement will be in a format that can be used within the final report on the quality audits.

12. Formal statement (2) Introduction This document is intended to provide an informal summary of the quality audit performed at RIVM over the course of the three days Monday 19 to Wednesday 21 March 2001. This audit was t he first of some one dozen site visits that will be conducted over the next 18 months within the EDU CE project. It was planned as a ‘test’ audit, so that the audit procedures and timetable could be considered and refined. While I hope that the notes presented below will prove useful to both RIV M and the EDUCE coordinator, the trial nature of this first visit and the incompleteness of the audit mean that impressions and concl usions are necessarily preliminary. Tim Martin Overview This document is intended to provide an informal summary of the quality audit performed at RIVM over the course of the three days Monday 19 to Wednesday 21 March 2001. This audit was t he first of some one dozen site visits that will be conducted over the next 18 months within the EDU CE project. It was planned as a ‘test’ audit, so that the audit procedures and timetable could be considered and refined. While I hope that the notes presented below will prove useful to both RIV M and the EDUCE coordinator, the trial nature of this first visit and the incompleteness of the audit mean that impressions and concl usions are necessarily preliminary. In February, it was agreed that it would not be possible to observe a full calibration of a spectroradiometer during the course of the first visit to RIVM. For this reason, the sections of the audit rep ort and summary relating to instrument calibration are incomplete. A calibration is planned for the return visit in O ctober 2002, when RIVM also hope to be able to make use of the JRC travelling calibration kit. Three full days were provided for this visit. In future, only two days will be allowed. I am most grateful to all at RIVM for their time, hospitality and helpful advice during the course of the visit. This document is sent to: Best practice This document is intended to provide an informal summary of the quality audit performed at RIVM over the course of the three days Monday 19 to Wednesday 21 March 2001. This audit was t he first of some one dozen site visits that will be conducted over the next 18 months within the EDU CE project. It was planned as a ‘test’ audit, so that the audit procedures and timetable could be considered and refined. While I hope that the notes presented below will prove useful to both RIV M and the EDUCE coordinator, the trial nature of this first visit and the incompleteness of the audit mean that impressions and concl usions are necessarily preliminary. In February, it was agreed that it would not be possible to observe a full calibration of a spectroradiometer during the course of the first visit to RIVM. For this reason, the sections of the audit rep ort and summary relating to instrument calibration are incomplete. A calibration is planned for the return visit in O ctober 2002, when RIVM also hope to be able to make use of the JRC travelling calibration kit. Three full days were provided for this visit. In future, only two days will be allowed. Room for improvement This document is intended to provide an informal summary of the quality audit performed at RIVM over the course of the three days Monday 19 to Wednesday 21 March 2001. This audit was t he first of some one dozen site visits that will be conducted over the next 18 months within the EDU CE project. It was planned as a ‘test’ audit, so that the audit procedures and timetable could be considered and refined. While I hope that the notes presented below will prove useful to both RIV M and the EDUCE coordinator, the trial nature of this first visit and the incompleteness of the audit mean that impressions and concl I am most grateful to all at RIVM for their time, hospitality and helpful advice during the course of the visit.This document is sent to: Conclusions This document is intended to provide an informal summary of the quality audit performed at RIVM over the course of the three days Monday 19 to Wednesday 21 March 2001. This audit was t=he first of some one dozen site visits that will be conducted over the next 18 months within the EDU CE project. It was planned as a ‘test’ audit, so that the audit procedures and timetabl e could be considered and refined. While I hope that the notes presented below will prove useful to both RIV EDUCE coordinator, the trial nature of this first visit and the incompleteness of the audit mean that impressions and conclusions are necessarily preliminary. In February, it was agreed that it would not be possible to observe a full calibration of ort and summary relating to instrument calibration are incomplete. A calibration is planned for the return visit in O ctober 2002, when RIVM also hope to be able to make use of the JRC travelling calibration kit. Three full days were provided for this visit. In future, only two days will be allowed. Quality Audit at Institute – April 2002

13. Final report • The report will be circulated to all EDUCE PIs • The report will be made available at the database website (accessible only to registered users). • Some non-contentious information about the visit will also be placed at the public pages of the EDUCE web site.