Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

1. Additionality and Public Sector Support to Irish Industry: Some Methodological Issues • Helena Lenihan (University of Limerick) • Mark Hart (Kingston University, UK) Research and the Knowledge Based Society - Measuring the Link European conference on good practice in evaluation and indicators May 24th 2004, NUI Galway Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

2. Key issue: ‘How’ to evaluate • Increased impetus to evaluate industrial policy interventions • For decades, significant resources have been allocated by the Irish government re: various types and measures of industrial policy interventions • Changes have taken place without insights of evaluation Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

3. Without reference to many of factors at work w.r.t. financial assistance: exploring the counterfactual (deadweight and displacement) • Some assessment of additional impact Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

4. Key findings from Irish experience • 2 development agencies - Shannon Development (Lenihan 1999; 2001; 2004) - Enterprise Ireland (Lenihan et al., 2003; Hart and Lenihan 2004)-Investor and consumer of research • Reflect on key methodological issues Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

5. Defining deadweight - Focus on ‘partial degrees’ of deadweight Evaluating the Effects of Shannon Development (SD)Industrial Support • 1995 • ‘Self assessment approach’ Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

6. Degree to which projects would have gone ahead anyway without assistance from SD in 1995. • Also incorporated other ‘indicators’ of deadweight into the study. • Adoption of direct and indirect questions to assess deadweight-considered imperative. Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

7. Key findings of Lenihan (1999) study on SD. • 53% of firms reported ‘pure’ deadweight • 10.4% ‘zero’ deadweight • Incorporating ‘partial’ deadweight categories---deadweight estimate rises to 78.4% • Confirmed by other ‘indicators’ of deadweight. Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

8. Table 1: Deadweight Estimates Derived in International Studies Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

9. Defining displacement • Displacement estimated at 4.2% • Are the results representative of Irish industrial policy interventions in general? Additionality of EI Financial Assistance (Lenihan et al 2003) Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

10. Table 2: Displacement Estimates Derived in International Studies Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

11. Key findings from EI study (Lenihan et al 2003) • Deadweight between 46.2% and 55.8% • Displacement estimates between 4.4% to 12.2% Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

12. Table 3: Deadweight Estimates Derived in other Irish Studies Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

13. Explaining Deadweight Estimates • No longer sufficient to merely derive estimates of deadweight and displacement and to discuss their consequences (Lenihan 2004). • Focus on specific firm characteristics that influence these estimates. • Logit (predictive) regression model for deadweight. Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

14. Do certain characteristics of grant-aided firms increase the likelihood of deadweight? • Useful for policymakers in terms of providing an ex-ante appraisal/evaluation template. Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

15. Key challenges and opportunities facing the Irish evaluation community • Indirect benefits (positive externalities) associated with financial assistance to firms. • Role of EI as evidence of due dilligence and its importance in financial leverage -Deadweight estimate adjusted downwards by between 3 and 7 percentage points. • Benefits of being part of EI international network • Account for fact that some financial assistance is repayable- complicates deadweight estimates. Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

16. Best-practice internationally • Econometric treatment models (e.g. 2-step Heckman models) which account for ‘selection’ and ‘assistance’ effects • Methodological minimum standards: full scale longitudinal set of case studies; control group analysis; selection and assistance modelling; predictive modelling . Additionality and Public Sector Support to Irish Industry: Some Methodological Issues

17. Conclusion • Importance of accounting for deadweight and displacement. • Financial assistance more amenable to evaluation. • Methodological approaches and lessons are transferrable. • Rigorous, structured but at same time user-friendly approaches. • No single best evaluation framework in same way there is no single best economic policy instrument. Additionality and Public Sector Support to Irish Industry: Some Methodological Issues