Evaluations of and reflections on university development cooperation – The Dutch case

1. Evaluations of and reflections on university development cooperation – The Dutch case Jolie Franke Team Coordinator Capacity Building Programmes Barcelona, 9 December 2013

2. Dutch Programme for Higher Education Capacity Building NPT and NICHE focus on the sustainable strengthening of Higher Education and TVET capacity in fifteen developing countries and fit in with the bilateral cooperation policies of the Dutch embassies in four selected policy priorities 1) Water, 2) Food Security, 3) Sexual and Reproductive Health and Rights and 4) Security and the Rule of Law (EUR 29 Million/year). Strengthened education and training capacity must respond to labour market and gender needs.

3. Evaluation practice • Regular internal project evaluations by project implementers: Requesting and Dutch organisations • Incidental external project evaluations commissioned by Nuffic • Regular external country programme evaluations commissioned by Nuffic • Incidental external thematic evaluation, e.g. gender review of NFP and NPT • Regular external programme evaluations commissioned by Netherlands Ministry of Foreign Affairs: 2007 NPT; 2012 NFP, NPT and NICHE.

4. 2012: Externalevaluation of NPT and NICHE • CommissionedbyMinistry of ForeignAffairs NL: • Accountability • Learning • Recommendations for successor programme • ExecutedbyRamböll: • Relevance of NPT & NICHE • Effectiveness of NPT • Efficiency of NPT & NICHE • Impact of NPT • Sustainability of NPT

5. Evaluation methodology • Desk study: comparison of nine International Education Development programmes (indepth: Sweden, Belgium, Finland and Norway) • Interviews and questionnaires of 16 selected projects • Visit to 6 countries and 34 requesting organisations • Statistical analyses: • capacity development index • Cost-efficiency analysis • Cost-effectiveness analysis • Adaptation of 5 Capabilities concept: evaluation grid • Capability to act • Capability to produce development results • Capability to relate • Capability to achieve coherence • Capability to adapt and self renew

6. Relevance • NPT and NICHE projects generally meet the respective needs on the partner country level and on the level of the requesting organisations. • Higher Education Sector: • On the sector level as they contribute to higher learning reforms (even more under NICHE). • On the level of the requesting organisations a sector-wide approach as undertaken by NICHE can strengthen the relevance of the projects by avoiding “project islands”. This can create synergies between different projects. • Economic Sector: • On the sector level NPT and NICHE are in line with the development strategies of the partners. • However, universities are sometimes not the main leverage to solve the challenges within a certain sector. Here, a focus on the TVET sector, as under NICHE, is more relevant.

7. Effectiveness on Southern Partner Organisations • Often second order learning is not reached • Top & middle management is often not involved • Training mechanisms are often not institutionalised • External networks are often not established NPT strengthens human resources, teaching capabilities and infrastructure NPT does not engage in wider organisational change processes NPT develops and revises curricula and courses successfully NPT does not always capacitate organisations to adjust to external change.

8. Efficiency • Regional stakeholders (e.g. other universities in the region) added value to an efficient project implementation and goal achievement • The roles distribution in the programme and project cycle are not clear and cause inefficiencies • The demand-driven design in both programmes • allows requesting organisations to articulate their demand independently . • causes inefficiencies regarding the time-involvement of the Dutch organisations. • The tendering procedure has had a modest effect on the efficiency of the programmes.

9. Impact: Employment According to Sector • The majority of NPT projects enable the requesting organisations to increase the availability of human power for the specific sector. • 96% of the NPT alumni are employed • 82% of the NPT alumni work in the sector for which they have been educated for • Case studies showed however that the absorption rate is not always given. • Case studies also confirmed that most graduates gain employment due to a shortage of adequate qualified staff in the sector. N=121

10. Sustainability in Organisations • The sustainability of the NPT programme is not always guaranteed because • in some cases there is a mismatch between the projects and external circumstances. • there is a lack of involvement of the higher management. • there is a lack of ownership. • a lack of attention towards reaching a critical mass in terms of training and a loss of trained personnel. • The investments into infrastructure are highly sustainable as they are used and maintained. • The general duration of the NPT projects is an impediment to reach sustainability.

11. Recommendations Continue the NICHE approach and further strengthen the connection to the Labour Market Approach capacity development at requesting organisations in a holistic manner Define roles and responsibilities more clearly and improve organisational and mutual learning mechanisms Ensure future interest of Dutch organisations in the programme and foster competition through improved incentive structures Coordinate more effectively with other programmes

12. The Perspective of the Dutch Organisations (Nikièma 2012) • Past (before 2002): stimulating factors • Prospects for long term cooperation • Diversity of projects • Opportunities for setting own agenda • Quite “lean and mean” procedures • Possibilities to input into processes from programme/project design up to implementation • Since 2002: hampering factors • Limited/no room for setting own agenda or impacting the project processes and directions • More complexity and costs for acquisition, implementation and administration of projects • Challenges for HE in realization of own ambitions

13. The Top 5 Motivation of Dutch Organisations to participate (Ramböll)

14. New NICHE phasestarted 1 July 2013 • Roles: Nuffic more at distance; implementationbasedon trust • Simplified procedures; lessreportingrequirements • HolisticCapacityDevelopmentthrough 5 Capabilitiesapproach • Bidders have more freedom to be creative in their bid: define outputs • Tender evaluation directly links price with quality • Focus on contribution to country programme • Trust - explicit notification - sanctions • Exploratory study on modalities of longer-term collaboration: pursuit of a shared agenda which leads to mutual benefits

15. More information: www.nuffic.nl • www.nuffic.nl/niche • http://www.rijksoverheid.nl/documenten-en-publicaties/rapporten/2012/05/17/final-report-evaluation-of-npt-and-niche.html • http://www.nuffic.nl/en/capacity-building/in-the-picture • Synergy in action: Coordination of cooperation programmes in higher education and research • Conference proceedings: Shaping the future: New pwespectives on Dutch cooperation in post-secondary education and training for sustainable development • Complexities of gender mainstreaming in higher education capacity development programmes • CD-ROM: Shared practice: Mainstreaming gender and labour market in capacity development

16. Thank you for your attention – Questions? Jolie Franke jfranke@nuffic.nl www.nuffic.nl