1 / 32

Citizen Report Card Methodology

Citizen Report Card Methodology. THE BUILDING BLOCKS. Overview of CRC Methodology. Defining Research Objectives. Development of Instrument. Sampling Design. Data collection. Analysis. Interpretation. The CRC Journey. Defining Research Objectives. Development of Instrument.

ashton
Download Presentation

Citizen Report Card Methodology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Citizen Report Card Methodology THE BUILDING BLOCKS Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  2. Overview of CRC Methodology Defining Research Objectives Development of Instrument Sampling Design Data collection Analysis Interpretation Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  3. The CRC Journey Defining Research Objectives Development of Instrument Sampling Design Data collection Analysis Interpretation Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  4. Examples of Research Objectives General- • Getting feedback from people on quality of public services Specific - • Finding out satisfaction with quality of service, satisfaction with behaviour of staff, extent of corruption, problem resolving capabilities of the agency Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  5. The CRC Journey Defining Research Objectives Development of Instrument Sampling Design Data collection Analysis Interpretation Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  6. Designing the Instrument Basic structure of a schedule/ questionnaire • Investigator information • Introduction • Filter questions • Demographics • The body of the schedule - sections if needed - sub- schedules if required Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  7. Thumb Rules for a Schedule • All research objectives to be covered • Shortest length possible • Language - simple and convenient based on respondents’ and investigator’s level of understanding • Clear instructions for the investigator Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  8. Basic Tools in Instrument Design • Different types of questions • Open-ended and closed questions / combinations • yes/ no questions • Coding - pre and post coding • Use of show cards • Scaling - 5 pt or 2 pt • Skips • Grids Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  9. CRC Process Chart Defining Research Objectives Development of Instrument Sampling Data collection Analysis Interpretation Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  10. Sampling Design • Why sampling? • Importance of representativeness • How to assure representativeness • Fixing quotas by area or category or both • Choosing appropriate sample size • time constraints and cost constraints • standard error • level of sub classification required for analysis and conclusions Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  11. SAMPLING DESIGN • BMP: • 606 general households • 812 slum households • MAIN SAMPLE: • 2000 households from city and suburban areas: • 800 general households • 1200 slum households • General Households: • 10 wards from each of 3 zones selected • Number of households allocated according to population of ward • Mains and crosses selected from each ward • Proportionate number of households selected from each street • Slum Households: • Slums selected from each region using PPS • Proportionate allocation according to slum type & size within regions • Households selected using right-hand rule in each slum through systematic sampling Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  12. The Research Journey... Defining Research Objectives Development of Instrument Sampling Data collection Analysis Interpretation Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  13. Field Issues... • Method of data collection • Right field team • Training/Briefing of investigators • Regular reporting • Spot checks and back checks • Time schedule • Adherence to budget Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  14. The Right TeamHow do we decide? • Familiarity with locality, language • Able to relate to target audience • Intelligent • Available for the entire duration of fieldwork • And, INTEGRITY Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  15. The Right TeamHow many? • f {Sample size, Timeline, Productivity } • Sample size : 2000 • Timeline : 20 days • Productivity per person/ day : 5 • Team size = [(2000/20) / 5] = 20 Investigators Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  16. The Final Team... Field Manager Field Coordinator Field Supervisor Field Supervisor Field Supervisor Field Supervisor 5 Interviewers 5 Interviewers 5 Interviewers 5 Interviewers Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  17. Data Collection • Random sampling • Starting points selection • Sample Size : 1000 • 100 starting points • 10 listings around each starting point • Right hand rule • Boosters-- phone call, exit interviews Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  18. Quality Checks At all stages • Before launching fieldwork • During fieldwork • After fieldwork Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  19. Quality Checks Before fieldwork • Pilot exercise • 10-15 interviews • Check for • Length • Comprehension • Flow • Sensitiveness Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  20. Quality Checks During fieldwork • Accompanied interviews • Sample of each interviewer’s work spot checked by field supervisor • 10% back checks by the field coordinator • 100% scrutiny of schedules Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  21. Quality Checks After fieldwork • Sample sizes • Quotas • Representation Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  22. Cost Heads in Fieldwork • Cost is a function of • Nature of project • Type of respondent • Selection criteria • Sample size, sampling • Two broad types • Overhead • Expenses Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  23. Cost Heads in Fieldwork • Overheads • Time cost, personnel • Communication: telephone, computers • Courier / mailings Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  24. Cost Heads in Fieldwork • Expenses • Printing of questionnaires • Briefing and piloting • Interviewer fees • Supervisor fees • Local conveyance • Outstation travel : Field officer, interviewers • Others: Venue, equipment, moderation, gifts, transcriptions • Translation Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  25. The Research Journey... Defining Research Objectives Development of Instrument Sampling Data collection Analysis Interpretation Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  26. Data Analysis • Preliminary analysis by research agency • Further analysis by PAC Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  27. Framework of Analysis • Select computer package - Foxpro, SPSS • Run basic demographics • Check if this is in line with population • If yes, proceed with further analysis • If no, weight data to represent population Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  28. Framework of Analysis • What is the depth of analysis required? • What are the relevant analysis heads? • By demographics • By agency • By responses to other questions : • satisfied vs. dissatisfied • had problem vs. did not have problem • paid bribe vs. did not pay bribe • Simple measures used- percentages and mean scores. Recently regression models Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  29. The Research Journey... Defining Research Objectives Development of Instrument Sampling Data collection Analysis Interpretation Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  30. Interpretation of Data • Compare across agencies - rank them on satisfaction, responsiveness and corruption- using percentages and means • Within agencies - specific information - timings of water supply, quality of water - again using percentages and means • Two ways of interpreting same information - eg - number of visits to agency Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  31. Interpretation of Data • Assimilation of scattered findings into cohesive whole • Translation of findings into diagnostic statements • Conversion of diagnosis into conclusions • Formulation of recommendations based on conclusions Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

  32. From Findings to Recommendations • Satisfaction with behaviour of staff • Objective statement of finding - percentage satisfied, dissatisfied, ambivalent. • Conclusion - general / high/low satisfaction/dissatisfaction • Recommendation - training for staff to improve their behaviour Ms. Trina Vithayathil, Public Affairs Foundation, Bangalore

More Related