Development Assessment

1. Development Assessment Dr. LokNathBhusal

2. Content • Active Learning - Group Work • Defining Development and Its Assessment • Determining indicators • Ways to assess development at the output and outcome levels • Development assessment practices • Issues and Suggestions for Improvement • Summary, Conclusion and Review

3. Objectives of the Session • Explain the ways to assess development • Describe the techniques of periodic and regular assessment

4. Active Learning Discuss these questions and Answer them in a Group: What is development? What is development Assessment? How do we assess development? Why should we assess development? What are the common problems in development Assessment? How to overcome them? Note down your answers and make a short and sweet presentation – You have 20 minutes for this all.

5. What is Development • Growth + Change = Development • Growth = Increases in quantity in desirable direction • Change = Desirable Qualitative Improvements • Examples: Growth = Economic Growth Rate Change = Economic/political/social transformation

6. Development is … • Collective intervention • In collective affairs • According to standards of improvement • Which vary by historical context, class, culture and relations of power

7. Meanings of development

8. Dimensions of development theories

9. Global hegemony & theories

10. Current actors… current views

11. Trends in development thinking

12. Trends…

13. Summary: Development • Development is a multidimensional phenomenon. It is defined and understood differently by scholars and practicioners coming from different time period, discipline and orientation • In nutshell, it is desirable growth and development in the desirable direction

14. Development Assessment Dictionary (Cambridge) Meaning Theact of judging or deciding the amount, value,  quality, or importance of something, or thejudgment or decision that is made. • Would you say that is a fair assessment of the situation? • Boththeirassessments of productioncosts were hopelesslyinaccurate.

15. Development Assessment -2 • Development Assessment is all about measuring/identifying/understanding the output, outcome and impacts of development interventions • It is a process of determining/obtaining/controlling desired results of development • It is an outcome of monitoring and evaluation of development interventions • We need to understand Development Result to understand Development Assessment

16. Development Results • A Result is a describable or measurable change resulting from a cause and effect relationship • Results are consequences of actions taken to meet certain purposes. • “ An effect arising from something” • “ The success or benefit obtained from a course of action” 3. Two major elements to remember: The notion of CHANGE which involves a visible transformation in the group, the organization or the society or country. The notion of CAUSALITY illustrating the cause and effect relationship between an action and the results achieved.

17. Knowing the Results The statement of results should illustrate the type and level of human transformation that occurs from intervention in a given context, such as: • An improvement (in the health conditions) • An increase (of the revenues of a given group or community) • An increase (in the Gross National Product) • A strengthening (of the capacities of local NGOs) • An increase (in the girls’ scholarship rate) • A reduction (in the infant mortality rate) Or it can illustrate a transformation in attitudes, practices or behaviour of a given group.

18. Result Chain A results chain is a logically linked set of results, some immediate, others more distant. Results at each level aggregate to produce the results at the next higher level. The results chain includes: • Immediate results called OUTPUTS that are the consequences of completed activities. • End-of-project results called OUTCOMES, which are the consequence of the achievement of a set of outputs. • A long-term result called IMPACT, that is the logical consequence of the achievement of the outcomes.

19. Some Key Terms • Inputs: the raw materials that provide a basis for conducting a program. Inputs can include money, technical expertise, relationships and personnel. • Activities: the actions of staff and their partners that are designed to meet a project/programme/policy/plan objectives • Impacts: higher level strategic goals, such as increased access to justice or improvements in public safety. • Logical/Results Framework: A project planning and oversight tool consisting of indicators and milestones for key inputs, activities, outputs, outcomes and impacts. • Outcomes: the benefits that a project or intervention is designed to deliver. • Outputs: the tangible and intangible products that result from project activities. • Theory of change: a set of assumptions about the relationship between project activities and goals.

21. Example of Improving Supply of Potable Water and Sanitation Facilities

22. Results Like… Focus Timeframe more Impact HIV incidence reduced Human! 5-10 yrs then if then Outcome Response brought to scale Institutions/ Behaviours 5 yrs Collective Accountability if Outcome Leadership empowered Institutions/ Behaviours 5 yrs then Output Skills of NAC strengthened Knowledge, skills, abilities, services <3 yrs if then less Activity Train 250 district AIDS officers <1 yr if

23. Results at the Core

24. An Illustration of Results Framework

25. Determining Indicators for DA • An indicator is a “pointer” that helps you to measure progress towards achieving results • A quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor/intervention • There are two types of indicators: quantitative indicators and qualitative indicators

26. Quantitative vs Qualitative Indicators Quantitative • Number of women in the decision-making positions • Comparative (women and men) rate of employment in the public sector • Economic Growth, unemployment, poverty rate Qualitative • Level of satisfaction of the beneficiaries • Quality of the service provided • Perception of men on women’s participation in the local Committee • Coherence between the management tools developed and the absorptive capacity of the beneficiaries.

27. Why do we need indicators? • to make changes from abstract results towards observable and measurable results • to filter a large amount of information to a few key elements to make strategic decisions • to make the assessment of progress less subjective

28. Indicators Hierarchy Impact Indicators OutcomeIndicators OutputIndicators

29. Types of indicators

30. Which Indicator to Select? 1. Validity: Does the indicator allows you to be precise in measuring the results (quantity, quality, time bound) 2. Reliability: Do the indicators measure trends over time? To be reliable, the information must always be collected at the same time period. 3. Representativity: Do the indicators provide a desegregated information by sex, age group, etc.)? 4. Simplicity: Is the information available and will it be feasible to collect and analyse it? 5. Affordability: Can we afford to collect and analyse this information?

31. Steps in selecting good indicators 1. Clarify the output/outcome/impact statement • Can people understand it? Can partners agree on it? • Review precise wording and keep it simple 2. Develop a list of possible indicators • Usually many possible indicators, but some are more appropriate and useful than others • Don’t settle too quickly on the most obvious and most convenient indicator • Tools: a) internal brainstorming, b) consultations with experts, c) experience of other operating units with similar indicators 3. Assess how “good” each indicator is • see check-list

32. 4. Select “best” indicator(s) • Select optimum set that meets the need for management at reasonable costs • Select only those that represent the most basic and important dimension of the objective (one, two or maximum three) • Remember: it’s an art, not science 5. Ensure each indicator is measurable in terms of quantity, quality, time, target group(s) and location 6. Determine baseline, targets, milestones • If you can’t find data for the baseline, use a different indicator. If you can not agree on targets/milestones, redesign the project.

33. Common Problems with Indicators 1. UNCLEAR RESULTS • an indicator can only be as good as the output or outcome statement it refers to • clear, simple, precise wording; avoid buzzwords or ambiguous statements EXAMPLE.: “ Appropriate capacity building services passed on to all levels of the local government service delivery system” 2. MULTIPLE INDICATORS • indicators with more than one dimension need to be separated • the word “and”, a comma or a slash often point to a compound indicator EXAMPLE.: “MC/PM adopted in all DDCs / municipalities and piloted in selected VDCs”

34. Common Problems with Indicators 3. DESCRIPTION vs. DATA • The descriptionof the indicators can be as long, specific and detailed as possible EXAMPLE.: % of annual work plans by all VDCs in Nepal which are prepared on the basis of a participatory planning process etc. • The actual data set (indicator baseline, indicator target, indicator status) can only be a single number or a single word. EXAMPLE.: 64% 4. LIMITED SET OF INDICATORS • a single indicator often does not sufficiently capture the various dimensions of a complex result • more than 5 indicators often diminish their value • generally, a set of 2 or 4 indicators is ideal to keep monitoring meaningful yet simple and manageable • usually, outcomes require more indicators than outputs, because they are often more complex EXAMPLE: “Citizens and communities engaged actively with local governments and hold them accountable” (LGCDP Outcome 1)

35. Indirect (Proxy) Indicators Direct indicators are preferable. However, they do sometimes not exist, are too expensive or inefficient to obtain. In such cases, indirect indicator have to be used. Indirect measure or sign that approximates or represents a phenomenon in the absence of a direct measure or sign

36. Exercise: Indirect (Proxy) Indicators

37. Assessing Development at the Output and Outcome levels Output level: Outputs are lower-level tangible and intangible products that result from project/plan activities We need to determine output level indicators to assess development at this level

38. Assessing at the Output Level What does your project produce ? Output indicators describe the delivery of products, including, but not limited to: the providing training and technical assistance; creating standards and legislative documents; investing in buildings and infrastructure; and hiring staff required to implement a project. When combined with measures of inputs and activities, output indicators can provide measures of economy and efficiency, describing the relationship between investments in a project and products.

39. Check the Indicators • The Results Framework of the 13th Plan • MDGs • SDGs • Any Development Project • Check the levels of the indicator – Output to Impact

40. Assessing at the Output level - 2 • It is usually important to track output indicators at regular intervals over the life course of an initiative, as a way of assessing progress towards project goals and detecting delays. • While achieving project outputs offers no guarantees that your project will be successful, without achieving your outputs the chances of success may be slim.

41. Output Indicators Checklist • Do you have indicators for all of the outputs that you expect to contribute to project success? • Do you have some measures which describe the quality of outputs, as well as the quantity? • Do your output indicators describe the outputs produced in different project sites and who participated in their production (where relevant)? • Do your output indicators provide information necessary for replication (e.g. by providing a clear connection between project activities and outputs)?

42. Assessing at the Outcome Level Well-designed outcome indicators are important mechanisms for ensuring transparency and accountability, describing the return on investments and the benefits that a project delivers. Whereas output indicators often rely on De Jure measures, describing things like the creation of legislation, provision of training or purchasing of equipment, outcome indicators should be De Facto – describing the real world changes that these outputs will produce. For example, creating legislation or purchasing medical equipment is rarely the end goal of a project; outcome indicators should measure the protection of rights or improvements in health that result from these activities and outputs

43. Assessing at the Outcome Level - 2 • Effective outcome indicators typically combine quantitative and qualitative measures, describing the number of people benefitting from a project and the nature of those benefits. • Because they are designed to measure the ultimate results of your project, it is often important to include the perceptions and experiences of the intended beneficiaries

44. Outcome indicators checklist • Do your outcomes indicators draw on existing data wherever available? • Do they describe the perceptions and experiences of program participants and other beneficiaries? • Do they only include what is to be measured (and not information on targets or benchmarks)? • Are your outcome indicators relevant, measurable and realistic? • Do they include information that is important to key stakeholders, funders and intended beneficiaries? • Do your indicators describe issues that are important to vulnerable groups, and can you disaggregate the results to describe the experiences of women, girls and other vulnerable groups? • Are they pro-poor? • Do your indicators include De Facto measures of project outputs (describing changes in practice or experiences) in addition to De Jure measures?

45. Development Assessment Practices • Supervision • Monitoring • Evaluation • Performance Review • Joint Review

46. In Nepal - 1 • Supervision and Monitoring at the project, programme and policy level • Time-bound progress reporting • Ministry-level Progress Reviews • Quarterly Progress reviews • Review at the National Level – National Development Problem Solution Committee • Half-yearly reviews • Annual Reviews • Mid term and final evaluation of policies, programmes and plans.

47. In Nepal - 2 • Donor assessment – IMF/WB/ADB • Independent Assessment • Public Hearing • Poverty/health Monitoring

48. Why Assessment? • To know the progress • To improve performance • To solve problems • To provide feedbacks • To celebrate success • To avoid future failure – lesson learning • To comply with donors’ requirements • To improve accountability and transparency

49. Issues in Development Assessment • Content of assessment – output or outcome or impact • Time of Assessment – regular, irregular or at convenience • Professionalism in Assessment • Participation in Assessment – RRA, Customer Surveys • Accuracy of Data and methodology • Utilisation of feedback

50. Suggestions for improvement • Clarify the Content of assessment – output or outcome or impact • Carry out Regular Assessment • Enhance Professionalism in Assessment • Ensure maximum participation • Ensure the Utilisation of Assessment feedback • Use accurate Data and methodology for Assessment