Introduction to citizen implementation monitoring tools and methodologies

1. Introduction to citizen implementation monitoring tools and methodologies CEGAA-OSI EE Partners Workshop March 2009 Teresa Guthrie

2. Why citizen monitoring and advocacy? • Citizens fund government expenditure through taxes • Citizens have rights, governments have to deliver on these rights • Civil society is a component in a transparent and accountable state. CSOs and individual citizens have to participate in decision making and planning processes affecting their lives • Civil society has to monitor how their public finances are utilised to advance human rights and development and to reduce poverty

3. Different implementation monitoring and auditing tools • Social Auditing (e.g. MKSS, India) • Citizen Report Cards (PAC, India) • Public Procurement Monitoring (e.g. PWI, Philippines) • Physical Verification/ Infrastructural Development Monitoring (CCAGG, Philippines) Materials adapted from the source document by Vivek Ramkumar, IBP: “Our money, our responsibility: A citizen’s guide to monitoring government expenditures”

4. Social Auditing

5. Introduction • A social audit is a community driven participatory auditing of government implementation of programmes • It complements financial audits performed by government agencies • By verifying quality of service provided, performance of the state, and strengthening the financial audits • It promotes citizen participation in governance issues • Resulting in better transparency and accountability

6. Differences Between Social & Financial Audits

7. Social Auditing Methodology

8. Social Auditing: Methodology • A proper social audit has 7 steps which recognise local citizens as experts • CSOs can only provide importance assistance to the community undertaking a social audit • Training of citizens on the social auditing process • Assisting with accessing information to conduct the audit • Assisting in collating and disseminating information to the whole community • Documenting the social audit findings and following up with government officials to demand action

9. Methodology: 7 Steps of the AuditStep 1: Identify the Scope of the Audit • Identify specific programme for auditing • Identify specific state agencies relevant to the audit • Stipulate period of investigation (how many years do you want to audit?) • Ask questions for information about: • Access to information • How much information is available? Does it cover the whole audit period? • How much community involvement is expected?

10. Methodology: 7 Steps of the AuditStep 1: Identify the Scope of the Audit • What and how much resources are available from the coordinating CSO for this audit? • resources may include office space in the community, volunteers, transport, etc • What is the relationship between the coordinating CSO and government officials • Good relationships assist in accessing information more easily and in seeking urgent corrective action after the audit

11. Methodology: 7 Steps of the AuditStep 2: Identify and understand the management structure of the programme • Which level of government is responsible for the programme to be audited? • Develop a map that identifies all programme implementers and stakeholders, funding flows • Coordinating CSO to identify community members who have worked in programme for firsthand information, e.g. volunteers, project managers, etc

12. Methodology: 7 Steps of the AuditStep 3: Obtaining information on Programme under Audit • Coordinating CSO to secure access to relevant documents such as • Accounting documents (cashbooks, wage rolls, purchase orders, invoices, signed receipts etc) • Technical project records (book of standards, measurement books, contract specifications, etc) • Managerial records (stock registers, progress reports, spending certificates, etc) • Information requests likely to face resistance from local officials

13. Methodology: 7 Steps of the AuditStep 4: Collating information once received • Coordinating CSO to sort out with assistance from volunteer citizens collected information and preparing project files • Creating simplified charts of data basically summarising information obtained

14. Methodology: 7 Steps of the AuditStep 5: Distributing information • Coordinating CSO to make copies of project documents with summaries and share with community members through door to door visits by volunteers • Organising meetings with residents who have worked or participated somehow in the programme to verify accuracy and truthfulness of detail on the records • The social audit team or coordinating body should physically verify project site and visit beneficiaries to verify benefits received as reflected on the project documents • Findings from these exercises should be recorded and filed in relevant project files • The team should also invite everyone providing information to come testify at the forthcoming public hearing

15. Methodology: 7 Steps of the AuditStep 6: Holding public hearing(s) • The data collection phase should be used to publicise the upcoming public hearings • Publicise the hearings as widely as possible • Ensure that the hearing location is accessible to all residents • Provide necessities for the hearing: water, sufficient seating arrangement, sanitation etc • Provide basic location/ venue – extravagant looking venue may affect participation and attendance • Key roleplayers need to be identified and have to be present at the public hearing: coordinating team, eminent community members, local media, local government officials (“to receive community feedback and to respond to it”)

16. Methodology: 7 Steps of the AuditStep 7: Follow-up to the hearing • Transforming audit findings into a strong and broader advocacy campaign • Preparing formal report from the hearing(s) and submitting to senior government officials, media and other interest groups • Developing recommendations for corrective actions and policy changes to improve services • The coordinating team should put pressure for changes to take place

17. Citizen Report Cards (CRC)- measures of public satisfaction -

18. Methodology: 6 Phases of the CRCPhase 1: Identification of Scope, Actors & Purpose • Clarifying scope of the CRC: which public services will be assessed, how findings will be used and shared • Building a coalition for credibility: determine legitimacy of participating groups (are these local or are these seen as illegitimate outsiders? • Prepare to answer the questions:“Whose political interest are they serving”? And “Who is funding you?”)

19. Methodology: 6 Phases of the CRCPhase 2: Designing a questionnaire • Identify users to be surveyed – those who have used the service within the past 6 months or year and run focus groups • Organise focus group meetings with service providers, solicit their views and ask questions emanating from users discussions • Use information from users and service providers to design an informed questionnaire • Define the structure and size of the questionnaire • Questionnaire length, interview duration, rating scales etc • Always include a timeframe when asking questions so that responses apply to that specific period of interest • Pre-test the questionnaire on similar focus groups and make appropriate modifications • Translations into local language(s) may be necessary

20. Methodology: 6 Phases of the CRCPhase 3: Sampling • Identify geographical area from which respondents will be drawn • Determine survey sample size • Choose sample respondents • “proper sampling is no easy task”

21. Methodology: 6 Phases of the CRCPhase 4: Execution of survey • Select and train a team of research assistants/ enumerators • Training on the purpose of the survey, interviewing skills, etc is compulsory prior to fieldwork • Perform random checks of interviews to ascertain that collected data is genuine for credibility issues • Enumerators may sometime mis-interpret the respondent, and record inaccurate data • Some fraudulent enumerators fill information on questionnaire without any respondent present !

22. Methodology: 6 Phases of the CRCPhase 5: Analysing the data • Statistical skills are essential in analysing quantitative data on questionnaires • Different data analysis / computing tools could be used, such SPSS • Use findings to compile a report card

23. Methodology: 6 Phases of the CRCPhase 6: Dissemination • Engage relevant officials, and acquire their responses to be incorporated into the final report • Engage the media, in a press conference or similar medium. Different media could be used to disseminate information to increase coverage • Facilitate engagement between service providers and users • This may happen in a form of community meetings, discussion, or event a more effective public hearing • Present the message • Presentation of final results, both positive and negative, with conclusions and recommendations for fixing the problems

24. Public Procurement Monitoring- the PWI Approach, Philippines -

25. Introduction • Procurement monitoring involves analysing procurement documents and holding government accountable for all procurement transactions • Large amounts of government funds are spent through procurement processes and this is where civil society can make a big difference: by tracking down original transaction documents from the planning phase to the final stage of payment to service providers, and looking for inefficiencies and irregularities • PWI have developed the so-called Differential Expenditure Efficiency Measurement (DEEM), a tool to measure corruption and inefficiency in public procurement

26. The DEEM Methodology DEEM • Measures efficiency both in cost and time • Measures wastefulness in public spending • Identifies causes of inefficiency in procurement/ spending • Assumes competition in free market • Measures for competitive bidding & comparable prices • Examines government procurement documents • Enters data into ten forms that collect relevant information about the procurement

27. The DEEM Methodology: • Form I : The first form provides an overview of the transaction, including information on a cheque issued in payment of an invoice and the corresponding disbursement voucher • Also check: signatories, names, titles, signatures, dates, amounts • Form II : Description of items procured and summary of information pertaining to that procurement • Form III : Addressing the purchase request form, check for number, date, requesting department/section/ person, requested items, estimated costs, purpose, authorised signatures etc • Form IV : Addressing purchase order, check for consistency with information on purchase request form, corresponding disbursement voucher and supplier details • Form V : Collects information on the invoice, checks for consistency of information in the disbursement voucher and purchase order, authorised signatures and dates. • Other forms: cover other stages of the procurement process, i.e pre-bidding process, assessment of bid received, inspection reports, annual procurement planning, minutes of procurement meetings etc

28. The DEEM Methodology: Summaries The DEEM team then analyses summary sheets to identify inconsistencies and irregularities in the procurement process, asking questions such as: • Is the purchase request form dated after the purchase order form? • Does the purchase order form show a higher cost for a procured item than the bid document does? • Does the payment invoice show a higher amount paid to a vendor than the purchase order does? • Does the purchase order contain a different quantity than the payment invoice does? • Does the delivery date (as recorded in the goods inspection and acceptance form) the same as the date mentioned in the contract or purchase order, and is any delay accounted for?

29. Physical Verification- the Abra experience –Concerned Citizens of Abra for Good Government (CCAGG) Philippines

30. The CCAGG Approach • CCAGG monitors government projects in Abra province of Philippines, increasing community participation in development and public works programmes • Specialty in monitoring infrastructure projects, using local monitors to verify that construction projects are executed according to contract norms and standard in the “blue book”

31. CCAGG assumptions • Information of official government documents is not always accurate • Verification of government information is key to improving accountability • Public participation has to happen at community level where projects are implemented • Projects declared ‘complete’ may have never had happened

32. Methodology for Physical Verification • Community monitors including housewives, students, out-of-school youth, observe construction projects and report findings to senior or specialist colleagues such as engineers or accountants • Monitors are equipped with all essentials, measuring tapes, record books, cameras. Amd voice recorders to assist them in monitoring • CCAGG uses legal framework and other provisions to access information and to monitor projects: • The “Blue Book” – technical reference guide with specifications, or standards for public works • Approved plans and drawings for the project under investigation • Program of work containing general information on the project under study – location, source of funding, contract amounts, work schedule, project officer who approved the document etc

33. Methodology for Physical Verification • The monitoring team compares information on original project [planning] documents against both the information obtained by community monitors from their physical inspection and the financial and accounting documents and other technical reports submitted by the contractor upon the project completion • The team looks for evidence of corruption ot poor performance , including the use of sub-standard materials or fraud in contracting procedures (such as rigged contracts or tenders) • A final report form includes a summary of the project with all specifications, monitors’ findings from physical verification and recommendations for action by relevant government officials

34. The end, more work back home!!