Comments to the planned Eurochild scorecards

1. Eurchild Policy Steering Group meeting Progress Hotel, Brussels 1 September, 2010 Comments to the planned Eurochild scorecards István György Tóth – András Gábos TARKI Social Research Institute

2. Questions from Jana • Obstacles in the work of identifying/selecting indicators on child well-being? • Feasibility of the development of a scorecard for each member state? Pitfalls? • Reactions from member states/the EC? • Likelihood (& desirability) of the EU integrating some of these indicators into the Laeken indicators? • Usefulness of data on child well-being in influencing policy?

3. Main tasks within the project on „Child poverty and child well-being in the EU”: Task 1. Empirical analysis of child poverty Task 2. Assessment of the effectiveness of policies for combating child poverty Task 3. Recommendations for a limited set of indicators most relevant from a child perspective Commissioned by: DG Employment of the European Commission, Unit E2 Consortium: Tárki Social Research Institute, Budapest Applica sprl, Brussels

4. Domains of child poverty and well-being(EU Task-Force, TÁRKI/Applica reports) • Material well-being:factors relating to the materialresourcesof the household that the child has access to or lacks during his/her development, which include indicators of • (A1) income, • (A2) material deprivation, • (A3) housing, • (A4) labour market attachment. • Non-material dimensions of child well-being, which may reflect on both the resources a child has access or lacks during his/her development and outcomes in different stages of this development: • (B1) education, • (B2) health, • (B3) exposure to risk and risk behaviour, • (B4) social participation and relationships, family environment, • (B5) local environment.

5. Selection criteria for appropriate child well-being indicators (a)  Tocapture the essence of the problem, we need indicators reflecting - well-being, predicting future prospects - attention to life cycle elements and intergenerational aspects - the level anddistributionof well-being (social gapbetween the poorer and the more well-off) (b)  be robust and statistically validated - assessment of the statistical reliability (level of mesurement error) - cross country variance (c)  provide a sufficient level of cross countries comparability, - with use of internationally applied definitions and data collection standards (d)  be built on available underlying data, be timelyand susceptible to revision (e)  should be responsive to policy interventionsbut not subject to manipulation

6. Applied criteria for the selection of indicators • Statistical robustness: • 5: highly robust, • 4: caution is warranted, confidence intervals to be published, • 3: for majority of countries caution is warranted and conf intervals • to be published, for other the data cannot be published, • 2: for majority of countries a significant data improvement/sample size increase • is needed, and • 1: to have reliable data a new dataset is to be designed. • Level of cross country comparability: • 3: no comparability problems, • 2: unclear institutional or cultural specificities prevail, • 1: either institutional or cross/cultural problems or both hinder comparability • across countries or there are or major data harmonisation problems. Responsiveness to policy change(the length of the causal chain between policy interventions and measured outcomes): high ( short), medium (medium) and low (long).

7. Steps of identifying good indicators • a broad basedcollection of potentially relevant indicatorsin each dimension • work onindicator development(customising the selection criteria) • suggestions forbreakdownswherever possible • to fill out an indicator fichefor each and every indicators (example) • statistical validationof all material indicators (where data allows) • identifying data gaps • formulatingsuggestions

8. A sample indicator card with validation results:

9. Sample indicator charts with some tipical data problems B4.5a Figure 11-year-olds who have three or more close friends of the same gender A1.1a Figure At-risk-of-poverty rate by age of child, 2007 Robustness problems with the detail of the breakdown Very low cross-country variance B1.2a Figure Difference in average reading literacy between pupils whose parents have completed tertiary education and pupils whose parents have lower secondary education or below (PIRLS 2006) B2.6 Figure Breastfeeding, EU-27, proportion of children who were exclusively breastfed at various ages Good quality indicator, with some data gaps Serious data gaps for many countries

10. Conclusions: indicators Conclusion 1: There is a need for a comprehensive set of indicators to monitor child poverty and well-being Conclusion 2: Various child ages need to be reflected Conclusion 3:There is a need to monitor the social situation of children with migrant or ethnic (specifically Roma) background

11. Conclusions: data infrastructure • Conclusion 3: context information is needed on childand family related social expenditures, within the OMC reporting routines • Conclusion 4: further work on statistical validation necessitates opening up microdata aceess to some core datasets on non- material dimensions • Conclusion 5: incentives to support substitute or alternative datasets in national contexts is needed • Conclusion 6: to further investigate the potential for utilising national administrative datasets • Conclusion 7: to invest in panel surveys (national or EU level) to facilitate exploring causal relationships

12. Suggestions Suggestion 1: As an immediate action, new education, health and risk behaviour indicators should be introduced to fill in the reserved child well-being slot within the Social OMC portfolio of indicators Suggestion 2: To build-up a comprehensive and dedicated set of child well-being indicators to allow for monitoring their situation in a comparative way across the MSs Suggestion 3: To complement this portfolio with context indicators (e.g. institutional indicators or measures of intergenerational redistribution) Suggestion 4: To improve and adjust the data infrastructure accordingly

13. Suggestion 2: a full portfolio of child indicators and age breakdowns

14. Suggestions for the Eurochild PSG • Go ahead with the scorecards (it is important to have an independent evaluation of EU policies, on a comparative, harmonized basis) • No composit indicators, pls! • Focus on outcomes • Present distributional (inequality) aspects

15. Suggestions for the Eurochild PSG • Set clear guidelines/requirements for the scorecard excercise • Contact major data infrastructures, ensure continued supply of good quality data • Also lobby for data (access) • Maintain professional control of the production of annual scorecards (before entering the policy sphere) • Build ownership at EU and national levels (with the help of Eurochild national members)

16. Final report is available at: www.tarki.hu/en/research/childpoverty

17. The „Study on child poverty” project Commissioned by: DG Employment of the European Commission, Unit E2 Consortium: Tárki Social Research Institute, Budapest Applica sprl, Brussels Steering Committe: Terry Ward (chair) Applica Michael F. Förster OECD Hugh Frazer National Univ. of Ireland Petra Hoelscher UNICEF Eric Marlier CEPS/INSTEAD Holly Sutherland University of Essex István György Tóth TÁRKI

18. The EU policy context of the project • 2005: MarchEU Presidency Conclusions and Luxembourg Presidency initiative on “Taking forward the EU Social Inclusion Process” • 2006: Commission’s Communication ‘Towards an EU Strategy on the Rights of the Child, Communication from the Commission’ • Since 2006: streamlining of Social OMC, more systematic attention to children and reports and recommendations on tackling child poverty and social exclusion produced under PROGRESS by independent experts and anti-poverty networks • 2007: EU Task-Force on Child poverty and Child Well-Being • 2008: formal adoptionof the report and their incorporation into the EU acquis, National Strategy Reports of child poverty • 2009: „Study on child poverty and child well-being” • 2010: planned publication of a Commission staff working paper on child poverty.

19. How does this project add to the process? • Contributes to developing tools to regularlymonitorchild poverty and child well-being in the Member States • It aims at filling in the Social OMC „reserved slot”for child well being indicator(s) • Provides recommendations forimproving data infrastructure Starting point: Related projects:

20. Conclusion 1: There is a need for a comprehensive set of indicators to monitor child poverty and well-being • The new set could: • reflect most of thechild well-being dimensionsas set out in the EU Task-Force report • incorporate OMC indicators already having a0-17 age breakdown • include a few new material well-being indicators(educational deprivation and childcare) • include new breakdownsfor the already existing indicators • a whole range of non-material indicators • This suggestion • could be well basedon the existing indicator development work • would betimely in 2010(European year against social exclusion)

21. Surveyed datasets • The EU Statistics on Income and Living Conditions(EU-SILC) • The Labour Force Survey (LFS) • The Programme for International Student Assessment (PISA of OECD) • Progress in International Reading Literacy Study (PIRLS) • Trends in International Mathematics and Science Study (TIMSS) • Health Behaviour in School-aged Children survey (HBSC of WHO) • European School Survey Project on Alcohol and Other Drugs (ESPAD)

22. Sample indicator charts with some typical data problems A1.1a At-risk-of-poverty rate by age of child, 2007 Robustness problems with the detail of the breakdown

23. Sample indicator charts with some typical data problems B4.5a 11-year-olds who have three or more close friends of the same gender Very low cross-country variance

24. Sample indicator charts with some typical data problems B1.2a Difference in average reading literacy between pupils whose parents have completed tertiary education and pupils whose parents have lower secondary education or below (PIRLS 2006) Good quality indicator, with some data gaps

25. Sample indicator charts with some typical data problems B2.6 Breastfeeding, EU-27, proportion of children who were exclusively breastfed at various ages Serious data gaps for many countries