Handbook on HWF planning methodologies MODULE 5 Evaluation of the selected practices

2. Handbook on HWF planning methodologies MODULE 5 Evaluation of the selected practices Annalisa Malgieri & Paolo Michelutti WP5 team _____________________________ Turin, 18 – 19 September 2014

3. 7 Consolidated planning experiencesasstartingpoint…solid foundations for the selection Quantitative projections Bothsupply and demand Europeanmodels

4. The first step to the evaluation…the checklist 5 SECTIONS 32 ITEMS 3 OPTIONS FOR THE ANSWERS SHARED AND VALIDATED BY PARTNERS IN JULY 2014 THOUGHT AS A «SELF-EVALUATION»TOOL 7 COMLETED CHECK LISTS RECEIVED HARMONIZATION OF THE EVALUATION

5. Before starting some important clarifications • The check list has not the ambition to be exhaustive of all the indicators useful for the evaluation of a planning system • The purpose of the check list is not to drawn up a classification of the goodness of different aspects of the planning systems in the analyzed Countries, but to furnish the most objective and standardized interpretation for the individuation of the good practices • Related evaluations are only referred to the current situation and to the practices today in use. This means that we haven’t taken into account practices no more in use and practices not still consolidated

6. How to use the check list? • The check list allows for different kinds of evaluations: • an evaluation and identification of strengths and weaknesses common to the different planning systems analyzed • an evaluation and identification of the specific good practices within each planning system

7. 2 dimensions to explore for each item … Each item is identified by coordinates (importance, performance) ITEM E.6 IMPORTANCE ……….. ITEM B.1 ITEM A.3 ITEM A.1 ITEM A.2 PERFORMANCE

8. 4 different areas identified MAP OF STRENGHTS AND WEAKNESSES

9. Let’s take a look at the results…(1/4) AREA OF EXCELLENCE

10. Let’s take a look at the results…(2/4) AREA OF IMPROVEMENT

11. Let’s take a look at the results…(3/4) AREA OF MAINTENANCE

12. Let’s take a look at the results…(4/4) AREA OF INDIFFERENCE

13. On the road for the goodpractices’ selection IMPORTANCE FOCUS ON PERFORMANCE DIMENSION 1 2 3 PERFORMANCE Not just High (+) and Low (-) levelbut an exactvalue of performance for each item (score) Synthesis and analysis of results: Per item Per section Per planning model

14. Synoptic table of performance PAY ATTENTION: IT’S NOT A RANKING LIST BUT JUST A GUIDE TO HELP THE SELECTION OF GOOD PRACTICES FOR EACH SPECIFIC ASPECT OF A HWF PLANNING SYSTEM

15. The selected good practices BELGIUM Good practice on involvement of stakeholders, elaboration of a common proposal, reporting and advising the policy makers (and how it's important to have the right staff to manage this process). With examples of reports to the policy makers. Good practice on data collection of the supply side, with an in-depth analysis on how the process works, how data on "practicing" and "active" are collected, how FTE is calculated. Good practice in monitoringthe appliedplanning and enabling policy discussion. DENMARK Good practice on the supply side forecasting "easy and flexible", main aspects of the model, available tools, etc.

16. The selected good practices ENGLAND Good practice on to define explicit, transparent and specific goals on HWF planning Good practice on stakeholder involvement: the experiences of the "call for evidence" and the process of "triangulation” Good practice on data collection process FINLAND Good practice on HWF planning harmonized with social and economic development and planning of other related sectors

17. The selected good practices NETHERLANDS Good practice on how to assess the current imbalances: a concrete example (maybe on general practitioners) of which indicators and standard used to estimate current imbalances Good practice on forecasting the demand side: in particular for dentist Good practice on data collection, in particular on FTE estimation (examples, detailed description of the process, tools, etc). Good practice on self-evaluation of the effectiveness of the planning model (monitoring process and consumer satisfaction survey): examples and tools.

18. The selected good practices NORWAY Good practice on forecasting the HWF demand SPAIN Good practices on multi regional planning: organization aspects and forecasting model with special attention on in-land migration impacts. Good practice on how HWF planning system takes into account budget constraints in its scenarios and it allows to evaluate alternative allocation of resources.

19. Methodology for Strengths/Weaknesses MAP (1/2) PERFORMANCE DIMENSION Translation of the evaluation given in number values (see table) 1 2 3 4 Sum of the score obtained by each item in all 7 planning models (absolute score) Relative score calculation (absolute score/max score in percentage) i.e. ITEM B.1 • Items with relative score > 75% have been defined with high performance (+) • Items with relative score <= 75% have been defined with low performance (-)

20. Methodology for Strengths/Weaknesses MAP (2/2) IMPORTANCE DIMENSION PRINCIPLE: inclusion/not inclusion in a “minimum” health workforce planning model As per “minimum” model it is considered a “start-up” model containing the elements necessary for an efficient human resources planning, with particular attention to its use in those Countries/Regions that for the first time approach to the theme of health workforce planning in quantitative terms • CRITERIA: • Items considered essential to be included in a minimum model have been evaluated with high importance (+) • Items considered not to be essential to be included in a minimum model have been evaluated with low importance (-)

21. Methodology for good practices’ selection (1/2) PURE MATHEMATICAL CRITERIA Translation of the evaluation given in number values (see table) 1 2 3 4 Sum of the score obtained by the item for each section and for each model Relative score calculation (absolute score/max score in percentage) i.e. BELGIUM Comparison of the results among system planning models analyzed for each section to identify the good practice

22. Methodology for good practices’ selection (1/2) ADDITIONAL CRITERIA Good performance on items placed in “improvement area” 1 2 3 4 • Good performance on items considered very important based on Milan and Florence workshop evidences • i.e. budget constraints • i.e. FTE calculation Good performance on specific aspect of the planning system which are useful for the implementation and the improvement Unique experience on a specific and interesting aspect

23. Conclusion THE OBJECTIVE OF THIS WORK IS NOT TO MAKE A RANKING OF THE ANALYZED HWF PLANNING SYSTEMS THEY ALREADY REPRESENT GOOD EXPERIENCES ON HWF PLANNING THE METHODOLOGY APPLIED HAS BEEN STUDIED IN ORDER TO: - identify common strengths and weaknesses - develop a tool in order to define a prior area of intervention (improvement) - take into account all aspects to be included in a minimum planning model (implementation) THE FINAL SCOPE is taking advantage from the experience of someone to grow up all together, thanks to the exchange of the good practices

24. THANK YOU