Effective National Statistical System Assessments: Heading Towards Open Frameworks

1. Effective National Statistical System Assessments: Heading Towards Open Frameworks Results of NSS Assessment Tools Study Mary Strode – Independent Consultant March 2017 – 49th UN Statistical Commission

2. Questions on NSS Assessments: raised by Study • What are we measuring? • Why are we measuring it? • What are we going to do with the results? • How can we develop a more harmonised approach to assessing National Statistical Systems in the light of new developments in data?

3. Early tools for NSS assessment World Bank – Time Series –10 years – 146 countries IMF Data Quality Tool PARIS21 Benchmarks Levels of DQAF

4. There are many more tools to chose from… 15 + and rising

5. What are they measuring? • They measure a variety of issues… some several in one tool • Data quality • Outputs, indicators, openness, timeliness and periodicity • Adherence to standards and principles • Prerequisites of quality (laws, resources, structure etc.) • Compliance with Codes of Practice • Monitor Progress of statistical improvement • Advocacy – for statistical support • Statistical strategies • Project design • SDG coverage • Do they measure statistical capacity? • Why do we want to measure it? Impact of TA, design appropriate TA, ability to support Development Goals, trust in the resulting statistics

6. Four Types of Tool Identified • 1. Tools used national authorities to inform statistical national planning and strategic processes • 2. Tools for partners who want to ‘invest’ in statistics to inform project design and to discuss with governments ( includes data quality) • 3. Tools to assess statistical performance internationally –Global public goods • 4. Assess compliance with codes of practice (e.g. European Statistics Code Of Practice)

7. Tools classified by type 1. National Planning 2. Project Design 3. Global Monitoring 4. Compliance

8. What do we mean by capacity?Statistics in the Development Context CAPACITY DEFINITIONS • “ability of people, organizations and society as a whole to manage their affairs successfully”. OECD. • An essential ingredient in the UNDP capacity development approach is transformation… it must bring about transformation that is generated and sustained over time from within. Transformation of this kind goes beyond performing tasks; • If something does not lead to change that is generated, guided and sustained by those whom it is meant to benefit, then it cannot be said to have enhanced capacity, even if it has served a valid development purpose. STATISTICAL CAPACITY • Statistical Capacity is a nation's ability to collect, analyze, and disseminate high-quality data about its population and economy. WBI • PARIS21 Capacity 4.0… to manage the Data Revolution

9. UNDP – GENERIC CAPACITY MODELAdapted by ISWE for Statistics (2016) National regulatory environment Home user demand Freedom of press Rule of law Strength of economy Statistical Laws Codes of Practice Funding Resources Management Business processes Dissemination Quality processes Training Attitude Skills

10. Tools use different capacity dimensions

11. Do we have a Theory of Change that works behind our tools? • They are all GOOD - but all DIFFERENT • Statistical development is related to development – economic, policy, social, regulatory – strength of institutions. SUPPORTING USE • Are statistics unique in development - having most of the demand external to the country? • We know surprising little about use of statistics in developing countries – why would governments invest in statistics if they don’t use the products? • ISWE (UN 2016) spoke of the disconnect between use and production of statistics. In countries with weak demand – we focus on needs of users (dismally small) rather than supporting it.

12. What do we need to know in order to build capacity to meet future needs? AUGMENTATION OF CAPACITY • Capacity is strongly enhanced by donor funding and activity – for TA, field costs etc. – outputs valuable - sustainable capacity? • How much more augmentation is viable for SDGs, Big Data etc.? Could this overload fragile systems? HOW WILL COUNTRIES MANAGE NEW DATA SOURCES • Should NSOs have stronger curation and quality role? – less data collection? • Transferring institutional processes from developed countries may be misplaced. How transferable are these practices? Isomorphic mimicry • How is this to be managed in the developing world

13. How will results of assessments contribute to the changed context for statistics? • To what end do we need to develop this capacity? What will be its purpose? • More Data? Is this overloading NSS? • More Capacity? Can this be done sustainably? • Whose capacities need to be developed? • NSS – Production? Curation? Quality? Interpretation? • Users – how to use this data • Other producers – to augment NSS capacity • What kinds of capacities need to be developed to achieve the broader development objectives? • Technical • Productive • Demand • Regulatory/Governance (e.g. laws for managing Dig Data) • Are our tools giving us information to answer these questions? • How can we move forward in a harmonised manner?

14. Moving Forward • We have different frameworks in use by the tools. • Different questions – often several hundred per tool – each with different formats and with implicit solutions to capacity problems. • Burden and transaction costs on countries significant. • Results are often not shared – inefficient & costly • How does the measurement support an agreed ‘theory of change’ to support the capacity changes we need. • Can this be agreed, globally or regionally?

15. Do we need a task team?