Outline of the presentation

Monitoring and Evaluating Community System Strengthening Current CSS Indicators and Approaches Round 11 regional Workshop, Nairobi, KenyaAugust 19, 2011Monitoring & Evaluation

Outline of the presentation • Overview of CSS indicators review; rational, objectives, process and approach; • The indicator set; • Developing M&E frameworks for CSS programs and linking these to the Global Fund proposal; • Program review and evaluations

Review rational and process • 27 indicators from May 2010 release not incorporated into RD 10 proposals; • Preparing for Round 11 launch; • Solicit implementer’s feedback • Assess applicability of the indicators at regional and country level; • To assess the comprehensiveness in measuring community systems strengthening; • Determine whether existing alternatives can be recommended; • Assess existing or anticipated challenges measuring the recommended indicators; • Coordinated by TGF but conducted by a partner representative Technical working group • Regional consultations with constituencies

Results overview • A set of 29 indicators – all process and output level • Two categories - CORE and additional • While many are process and output level, core indicators will indicate improvements in the system over time as proxy measures of system/service quality • Shows the linkage to health outcomes as presented in a holistic health system approach • Clearly identified gaps in the CSS M&E framework for further development; • Field testing and Validation • QoS framework development – Programs and Community system

CSS Output/Process Level Indicators by SDA

Developing M&E frameworks for CSS programs • In country consultations to include the CSS programs and M&E into national strategies – national M&E plan • View CSS programs as part of the overall health system Refer to HSS/CSS complementarity • Link CSS interventions to health results and therefore have a comprehensive results framework refer to - CSS results framework • Show direct linkage to health service delivery • Link M&E framework to the Global Fund grant through the proposal Performance Framework (select from the national M&E plan or an equivalent)

INPUTS (HSS/CSS) ACTIONS OUTPUT OUTCOME IMPACT Example of the complementarity between the Health and community System % of facilities with capacity (ready) to provide ACT treatment to malaria cases (drugs, trained staff & guidelines, & diagnostic s) Facilities reporting no stock outs of ACTs/LLINs Increase “in-patient & outpatient visits” (e.g., people accessing ACTs) HSS SDA: Health Workforce -Health worker retention Training & education % of children under five/pregnant women that slept under a LLIN the previous night % of uncomplicated malaria cases receiving appropriate treatment Annual parasite incidence rate HSS SDA: Procurement & Supply Chain Management -Distribution of ACTs, LLINs and laboratory equipment & diagnostics CSS SDA: Service availability, use and quality -Demand creation through community mobilization for LLIN use, involvement in PMTCT, follow up of mothers and babies in communities and creating awareness of available PMTCT services

Outcome Level Indicators Malaria Tuberculosis Case notification rate Treatment success rate • Suspected malaria cases that have laboratory diagnosis/ Children under-5 with fever (household survey) that received laboratory diagnosis • Uncomplicated malaria cases receiving appropriate treatment • Women who received two or more doses of IPT during their last pregnancy

Outcome Level Indicators Maternal, New-born and Child Health HIV/AIDS Proportion of adults/children with advanced HIV infection receiving ART HIV-positive pregnant women who receive ART for PMTCT • Proportion of women attending antenatal care • Proportion of institutional deliveries • Children who received DTP3 • Children fully immunised • Drop out between DTP1 and DTP3 coverage (%) • Equity in immunisation coverage (Difference in DTP3 coverage in lowest wealth quintile and coverage in highest wealth quintile) (%)

Impact Level Indicators Health Status Financial Risk Protection The ratio of household out-of-pocket as payments for health to total expenditure on health • Neonatal mortality • Mortality due to major cause of death • Child mortality • Maternal mortality ratio • ART survival rate • PMTCT transmission rate • Confirmed malaria cases • Inpatient confirmed malaria cases • Future deaths averted

CSS indicators recommended data sources • Administrative records: organizational routine source documents. • The records will defer by indicator and organisation and therefore need to be defined at the planning stage. • Examples: Activity reports, policy documents, monitoring reports, supervision checklists, client registers, training records. • Institutional Surveys/Assessments: periodic data collection exercises • gather information on defined aspects of the organisation usually related to performance and or quality standards. • These are commonly implemented by the national level designated entity with oversight responsibility for community based activities and services. • either as part of routine supervision or as an exclusive exercise. • Require predefined service standards and checklists • Population-based survey - representative sample from population • commonly implemented by the national level designated program authority with the involvement of all partners. • CSS implementers are encouraged to participate as much as possible in the planning and execution of the surveys.

Program reviews and evaluation • CSS programs need to plan and conduct periodic program evaluations • CSS programs encouraged to include impact outcome indicators to demonstrate contribution to health results • Tracking of relevant core indicators to track improvement in the community system • For Periodic reviews, the core indicators will be assessed along with the grant impact outcome indicator performance

Outline of the presentation