Putting the Evidence into Evidence Based Policy Maximising the Value of Statistics

1. Putting the Evidence into Evidence Based Policy Maximising the Value of Statistics Peter Harper Deputy Australian Statistician Australian Bureau of Statistics National Statistical Service Seminar 19 August 2011

2. Outline • Statistics and evidence-based policy • Statistical literacy • Statistical quality • What makes a good indicator • The misuse of statistics • The ABS and how we work with policy makers • The National Statistical Service

3. Statistics and evidence-based policy • Evidence-based policy is policy informed by rigorously established objective evidence • Evidence-based policy reduces idiosyncratic decisions and leads to better outcomes • Statistics are at the heart of evidence-based policy, but they must also be well used

5. Statistics and the policy cycle Government and non-government information sources

6. Statistical quality • A statistic is of high quality if it is fit for the purpose for which it is being used • therefore, the purpose determines the quality required, the same statistic can be high quality for one use, and low quality for another • Statistical quality has a range of dimensions, which must be understood to determine whether or not a statistics is fit for purpose

7. Statistical literacy • Data awareness • Understanding of statistical concepts • Ability to analyse, interpret and evaluate statistical information • Ability to communicate statistical information and understanding

8. Dimensions of statistical quality

9. What makes a good indicator? • Relevant/reflective of issue • Available as a time series • Meaningful and sensitive to change • Summary in nature • Able to be disaggregated • Easily interpreted • Able to be related to other indicators

10. The misuse of statistics “Torture numbers, and they’ll confess to anything” • The misuse of statistics can be extremely damaging • Statistics can be misused deliberately or inadvertently • Statistics can be presented to deceive • Statistics can be chosen for support, rather than illumination • Ignorance can cause misuse

11. Misuse arising from error in statistical methods • Discarding unfavourable data • Biased samples • Non-random samples • Self selection • Non-response bias • Error in reporting • Leading questions • Deliberate lying • Guessing • Aiming to give a favourable impression

12. Misuse arising from error in presentation • Misleading averages • Mean, median and mode • Hidden distributions “Then there is the man who drowned crossing a stream with an average depth of six inches” • Meaningless percentages • Overgeneralisation • False precision • Data dredging/selective presentation • Inappropriate comparisons • Misleading charts

13. Tons of rubbish per person per day 200 100 Green City Garbageville 0

17. Shape Colour Everything BUT colour

18. Misuse arising from inappropriate analysis • Drawing conclusions from inconclusive results • Sample error • “Signal to noise” ratio • Misunderstanding indexes • Missing the context • Extrapolating the trend • False causality

19. Preventing misuse • Be informed, talk with the experts • Watch out for advocacy and agendas • Probe behind the numbers • Look into what might be missing “Do not put your faith in what statistics say until you have carefully considered what they do not say” • Does it make sense?

20. The Australian Bureau of Statistics • Australia’s national statistical agency • Provides statistical services for the Commonwealth Government, the State and territory governments and the Australian people • Established in 1905 • Independence and integrity underpinned by legislation • ABS Act, Census and Statistics Act • Over 3000 staff in nine office across Australia • Headquarters in Belconnen, ACT • A world leader

21. ABS Mission We assist and encourage informed decision-making, research and discussion within governments and the community, by leading a high quality, objective and responsive national statistical service.

22. ABS Values • Integrity • Relevance • Service • Access for All • Professionalism • Trust of Providers

23. How the ABS can assist policy makers • Provide statistical information and analysis • Provide advice on how information requirements could be met • Early engagement is essential • Provide advice on the use of statistics • Provide advice on the development of administrative data • Transform administrative data • Produce synthetic and/or modelled statistical output • Conduct statistical collections • Provide methodological and other statistical support

24. The National Statistical Service • A community of government agencies, led by the ABS, building a richer statistical picture for a better informed Australia • an agreed set of statistical frameworks, principles, policies and resources • underpinned by a shared set of values and associated behaviours.

25. NSS Objectives • Deliver a high quality, up-to-date, comprehensive, coherent statistical picture of the economy, society and the environment to assist • Encourage informed decision making, research and discussion within governments and the wider community • Provide a world class official statistical service that retains the confidence and trust of the Australian society as both providers to, and users of, the resultant official statistics • Maximise the use for official statistical purposes of data available within government administrative systems

26. NSS Objectives (continued) • Minimise the burden of statistical reporting at all levels of the Australian community • Document and retain as an enduring national resource key statistical outputs and their underlying data sources.

27. The value of the NSS • Provide trusted statistics that answer important questions • A better life through better decisions • Enable Australians to assess the performance of their governments and hold them accountable for outcomes • Provide a foundation for evidence-based policy and, through feedback, drive innovation in service delivery

28. The value of the NSS (continued) • For producers of statistics, a more cost effective approach in line with best practice to meet their own information needs and the needs of others, and provide them with access to statistical information from others • Enable Australia to realise the full potential of its investment in statistical resources by overcoming institutional and jurisdictional fragmentation

29. NSS Foundations • Values - governments and the community value and support high integrity information • Capability - producers and users are able to manage data, make it accessible, and use it well • Content - public information sources are fully used to provide a statistical picture of the economy, society and the environment • Statistical infrastructure - infrastructure such as statistical standards, policies and tools are shared to maximise the value of investment and support integrated statistics

30. The ABS is leading a national statistical service. An informed Australia and a better place for our citizens. www.abs.gov.au www.nss.gov.au