Key outputs from the GSS Quality Task Force and how these can help you…

1. Key outputs from the GSS Quality Task Force and how these can help you… Jill Pobjoy (ONS), Emma Newman (ONS) and Nick Woodhill (DASA)

2. Background to Quality Task Force • A Task Force under GSS SPSC • Established in January 2010 to develop policies, standards and good practice on • - Quality management • - Quality assurance • - Quality reviews • - Quality measurement and reporting

3. Achievements • A good practice document and definitions to aid quality management • Quality training to understand and implement quality assurance • A tool (“QMHT”) for quality reviews • Good practice guidance on quality measurement and reporting • These are now described in more detail

4. GSS Quality Good Practice Document • To help producers of official statistics achieve the quality requirements of the Code of Practice • Concentrates in particular on Principle 4 ‘Sound methods and assured quality’ • Supplements the National Statistician’s Guidance on Quality Methods and Harmonisation

5. Key Definitions The task force decided on the key quality definitions which have been approved by GSS SPSC • Quality – ‘fitness for purpose’ • Quality Management - ‘an encompassing approach to quality work’ • Quality Assurance – ‘anticipating and avoiding problems’ • Quality Control – ‘responding to observed problems’

6. GSS Quality Training A training course has been developed that specifically addresses the requirements in Principle 4, Practice 4 of the Code of Practice for Official Statistics • Available at two levels - a high level overview for G7 and above - operational level for SEOs and below

7. Aims of the Quality Training Both training courses aim to provide: • An understanding of Quality Assurance and Quality Management in the context of Official Statistics • Various examples to enable best practice and useful techniques to be shared • An understanding of how this links with Principle 4 of the Code of Practice

8. Course content Both courses focus on: • Key definitions • Output & Process quality • Quality Assurance & Quality Control: the differences, including exercises and examples • Quality Management: background and examples • How it all fits together • Useful tools

9. Quality Measurement & Reporting: Implementation in DASA Nick Woodhill (DASA)

10. DASA’s Background Quality Reports • DASA used the GSS QIF to produce BQRs which provide: • an overview of the output quality • users with information to determine if fit for purpose • information against the 6 ESS dimensions of quality • DASA’s Pilot Process: • internal report produced by methodologists who spent time with the relevant business area • This was resource intensive in current financial climate time • DASA being assessed by UKSA in Autumn 2011

11. DASA’s Background Quality Reports • DASA’s Plan: • Assess families of outputs • Remove burden from producers • Take a flexible approach • DASA’s flexible approach: • The BQR focused approach • Multiple BQRs and 1 WEfA • The WEfA focused approach • Multiple WEfAs with BQRs to follow • The combined approach • Multiple WEfAs and Multiple BQRs

12. DASA’s Background Quality Reports • Outcomes and Benefits • BQRs produced for 15 families of output • Better understanding of quality & how to report it • New initiatives have improved products: • Better commentary for the Civilian sickness absence statistics • Engagement in customer consultation (defence inflation) • New outputs to better meet customer requirements (deployment statistics & stationed location statistics) • Helped prepare DASA for UKSA assessment • Users of DASA publications can now judge quality

13. Quality, Methods and Harmonisation Tool (QMHT) Emma Newman (ONS)

14. QMHT for quality reviews • Self-assessment questionnaire, covering all stages of the statistical process • Principle 4 Practice 5: continuous improvement, regular reviews • Suitable for survey and administrative data • Identifies areas for improvement and good practice • A way of documenting the methods used • Users will only complete sections relevant to their work • Useful as part of preparation for UK Statistics Authority assessments

15. Who is QMHT for? • Survey managers • Staff involved in various stages of statistical process e.g. sample design, data collection, analysis etc. • Anyone producing a statistical output from survey or administrative data • Offers a port of call before consultation with methodologists

16. Example screenshot

17. Testing and preparation • Cognitively tested across GSS • Piloted with different business areas in ONS • Endorsed by the National Statistician and ratified by GSS Heads of Profession • QMHT Version 2.0 is now finalised and available to download from StatNet and the ONS website • Visit the Quality Centre stand today for more information

18. How QMHT can benefit you • Helps producers of statistics to apply quality management and quality assurance • Promotes a culture of continuous improvement • Provides an effective way of meeting Principle 4 Practice 5, which is a requirement of the CoP • Has been developed purely to benefit producers of statistics – helps with assessment but not a requirement as such

19. Thank you…any questions? jill.pobjoy@ons.gsi.gov.uk nick.woodhill@dasa.defence.gsi.gov.uk emma.newman@ons.gsi.gov.uk