Report on INTOSAI Working Group on Public Policies and Programs Evaluation

1. Report on the works of the INTOSAI Working Group on public policies and programs evaluation (WGEPPP) Rémi FrentzInternational Relations DirectorSAI France June 2019

2. A major Achievement in 2016 The endorsement of the “Guidelines on the Evaluation of Public Policies” by the INCOSAI XXII of December 2016 - INTOSAI GOV 9400 (cf. issai.org – changed into GUID 9030 at the next IFPP) • A new direction has been approved at the INCOSAI and the composition of the group is being reviewed to take it into account. • A survey was submitted in 2017 to all SAIs to identify the ones conducting or planning to conduct evaluations and interested in joining the group. • Two new members in 2019: the European Court of Auditors and the Comptroller and Auditor general Office of India. • Three technical workshops were announced Work plan 2017-2019 : a new direction for WGEPPP June 2019 - Report on the works of the INTOSAI Working Group on public policies and programs evaluation

3. Members and Observers of the WGEPPP – 2019 • Europe (13) • Belgium • Czech Republic (obs) • ECA • Finland • France • Georgia • Asia (4) • India • Pakistan (obs.) • Thailand (obs.) • South Korea • Germany • Hungary • Italy (obs.) • Lithuania • Poland • Russia • Switzerland 2016 adoption of the INTOSAI GOV 9400 • North America (1) • United States • Middle East (1) • Saudi Arabia (obs.) • Oceania (3) • New Zealand (obs.) • Papua New Guinea • Central and South Americas (4) • Chile • Costa Rica • El Salvador • Peru (obs.) • Africa (7) • Algeria (obs.) • Ivory Cost (obs.) • Gabon • Kenya • Libya • Morocco • Senegal (obs.) June 2019 - Report on the works of the INTOSAI Working Group on public policies and programs evaluation

4. September 2017 : first Workshop on qualitative and quantitative methods • This meeting was the first one taking place, It was held on the Paris on 25 and 26 September. • The Supreme Audit Institutions (SAIs) of 13 countries were represented. • A particular focus has been made on the optimal combination of qualitative and quantitative methods. SAIs of France, United States, Switzerland and Sweden presented their own experiences on the topic. • The members of the group adopted a conclusion : “Public policy evaluation conducted by SAIs could be strengthened by a more assertive use of quantitative techniques”. June 2019 - Report on the works of the INTOSAI Working Group on public policies and programs evaluation

5. June 2018 :a second workshop dedicated to evaluation of health policies • The next workshop (June 12th and 13th in Paris)was dedicated to sharing experiences, based on case studies and focusing on methodological aspects of evaluation on health policies. • After a external point of view on the topic, the French (Identification of progress from tobacco and alcohol policies evaluations), the Swiss (The effects of the "ambulatory medical rate“), the American (The case of « Medicaid Assisted Living Services »), the Lithuanian (A performance audit on "Quality of health services": How to move from a performance audit to an evaluation?) and the Algerian SAIs (first steps of the evaluation of the Algerian cancer plan) presented their own experiences of evaluation on health policies. June 2019 - Report on the works of the INTOSAI Working Group on public policies and programs evaluation

6. June 2019: A Third workshop on three topics (1) • The last workshop (May 20th and 21st in Vilnius) focused on three main topics: • 1/ Follow-up on the implementation of INTOSAI GOV 9400 Guidelines • A survey on the follow-up of this implementation was conducted among the WGEPPP members (14 answers). • The findings of this survey showed disparities in the conduct of EPPP. • However, while many of them do not conduct evaluations per se (but rather performance audits with an evaluative scope), the majority of SAIs pointed out that they still use the INTOSAI GOV 9400 guidelines as a reference framework. June 2019 - Report on the works of the INTOSAI Working Group on public policies and programs evaluation

7. June 2019: A Third workshop on three topics (2) • 2/ Methodological approaches on stakeholders’ involvement and use of qualitative tools. • Presentations were made by: • The Federal Control of Finances of Switzerland: Involving Stakeholders in Evaluation at the Swiss Federal Audit Office; • The Government Accountability Office of the United States: Advanced Technologies Effect on Workforce – An Evaluation Using Multiple Methodologies; • The European Court of Auditors: Overview of evaluation techniques on an EU Passenger rights audit. June 2019 - Report on the works of the INTOSAI Working Group on public policies and programs evaluation

8. June 2019: A Third workshop on three topics (3) • 3/ Practical cases of evaluations. • Presentations were made by: • The French Cour des comptes: Social housing policy evaluation: a French experience feedback; • The Algerian Court of Accounts: Evaluation of the maternal and perinatal health promotion policy; • The Italian Court of Accounts: Evaluation of sanitary housing and building policies and programs. June 2019 - Report on the works of the INTOSAI Working Group on public policies and programs evaluation

9. June 2019: A Third workshop on three topics (4) • This year meeting also saw the involvement of representatives from the academic community and evaluation organizations. • One representative from the OCDE gave two presentations: • on the results of the survey on the evaluation of public policies in the OECD countries; • on the inclusion of public policy evaluations results in the political decision-making process. • One representative from academia gave a presentation on the qualitative approaches in public policy assessment: practices, challenges, and ambitions. June 2019 - Report on the works of the INTOSAI Working Group on public policies and programs evaluation

10. 2019 Next steps • Last WGEPP meeting’s summary of decisions open for comment and review • Open call for proposals for hosting the next Working Group meeting in 2020. Members are free to come out with propositions before next INTOSAI Congress in Moscow in September • Adoption of the 2020-2022 work plan June 2019 - Report on the works of the INTOSAI Working Group on public policies and programs evaluation

11. 2020-2022 Work plan • The members agreed on reviewing the current version of the INTOSAI GOV 9400 by the 2022 INCOSAI, identifying gaps, difficulties, clarifications… • The members also agreed on several subjects that may be put forward as themes for the coming meetings: • Follow-up, impact and medialization of recommendations after evaluations; • Invitation of a past evaluation stakeholder to attend part of the next meeting, to give his insight on the added value of an advisory body; • Digitalization as a new methodological tool that can be used in the conduct of evaluations (citizens’ contributions, big data…); • Several themes of case study: Economics Education Employment Environment issues Health Sovereign topics (justice, defense, security, tax policy …) June 2019 - Report on the works of the INTOSAI Working Group on public policies and programs evaluation

12. our Website • Access to : • Minutes, presentations, conclusions of the meetings • INTOSAI GOV 9400 • http://program-evaluation.ccomptes.fr June 2019 - Report on the works of the INTOSAI Working Group on public policies and programs evaluation