Data Quality Assessment of PEPFAR ART Sites In Nigeria Final Report February 17, 2006

1. Data Quality Assessment of PEPFAR ART SitesIn NigeriaFinal ReportFebruary 17, 2006 Nigeria/Monitoring and Evaluation Management Services in collaboration with MEASURE Evaluation

2. What is Data Quality? Data Quality fundamentally means that the information collected as part of a program’s monitoring and evaluation system adequately represents the program’s activities.

3. Adequately Represents? The information is accurate and reliable. Accurate information is interpreted as measuring what we intend to measure (that the information is correct).

4. Role of Data Quality in the Emergency Plan • PEPFAR emphasizes data quality because the Emergency Plan is explicitly evidence based and results oriented • Good data are needed to inform the design of interventions and monitor and evaluate the plan’s quantitative progress toward pre-determined treatment, prevention, and care targets • Hence, if the effectiveness of the emergency plan is evaluated on the basis of numbers, then any doubt about those numbers makes the entire program vulnerable to criticism

5. Data Quality Assessment of PEPFAR ART Sites in Nigeria

6. Assessment Purpose • Determine quality of data used to report on PEPFAR ART indicators • Examine data collection, management and reporting procedures • Ascertain site capacity for M&E • Document degree of data sharing and harmonization among PEPFAR, GON, and other HIV/AIDS Programs • Assess cross-agency data comparability

7. Methods and Facilities • Field Visits – August 30 – September 2 2005 • Seven out 9 PEPFAR’s targeted States covered • 16 facilities out 23 assessed • Qualitative and Quantitative techniques • Mapping of patient flow within facility • Structured interview • Observations • Accuracy checklist • Geographic Information System (GIS) coordinates

8. ART Assessments took place: Nnewi

9. Data Quality Assessments Measure • Validity: Does data adequately represent performance? • Reliability: Are data collection and management procedures consistent? • Precision: Do data have sufficient detail to present a fair picture of performance? • Timeliness: Are data current and collected with required frequency? • Integrity: Are data free of manipulation for political, professional or personal reasons?

10. DQA Methods • Mapping • Questionnaire • Interviews • Participant observation

11. DQA Findings: Successes • M&E personnel were well trained and highly motivated • Written/posted M&E guidelines and flow charts at many sites • All sites had or were about to implement an electronic data reporting system Well coordinated ART patient administration, testing, treatment, counseling and tracking systems

12. DQA Findings: Challenges • ART eligibility guidelines not uniform among sites • Physicians complained about number and redundancy of required forms • Forms and databases not uniform across IPs, preventing unified reporting system • Tensions between PEPFAR IPs providing services at same site

13. DQA Findings: Challenges (cont.) • Outsourcing of lab tests hampered document monitoring • Lack of coordinated supervisory system led to fragmented data monitoring and processing delays • Inadequate Staff and equipment

14. Conclusion • The provision of ART services was a ‘work in progress.’ • Most sites had only just started operating when assessments occurred –two clinics had not yet opened • PMM harmonization across all ART sites in progress For the complete ART Assessment report, log on to:www.nigeriamems.com

15. Updates after Assessment • USG-SI-Advisor convened SI-working group • USG through IPs have engaged GON on needs for harmonization of tools • USG-supported GON in the harmonization process • Stakeholders’ meeting held • Current PMM tools under review and awaiting approval by GON • New harmonized tools to be piloted in scale facilities

16. DISCUSSION and QUESTIONS