Multiple Indicator Cluster Surveys Survey Design Workshop

1. Multiple Indicator Cluster SurveysSurvey Design Workshop Preparing for Fieldwork MICS Survey Design Workshop

2. Session Objectives To review the MICS recommendations for: • Questionnaire translations • The Pre-test • The size, composition, roles and responsibilities of the fieldwork team • The fieldwork training • Preparing communities and households for MICS

3. Questionnaire Translations • Questionnaires need to be translated into each and every language they will be administered in. • Challenges exist for non-written local languages • In such cases attempts should be made to standardize the way all questions will be asked • Translated questionnaires should be back-translated into the original language • Back-translations should be done by person(s) with no former knowledge of the questionnaires Use people who are familiar with the language(s). Not only those in academia

4. Objectives of the Pre-test Objectives of the pretest is to test; • Respondents understanding of questions • Appropriateness of the questions. • Response categories are comprehensive. • How the non-MICS add-ons “work” • The flow of the questionnaires • The data entry application

5. When should a pre-test typically be organized? • “Final” questionnaires to be pre-tested are ready. • Data entry application that is similar to the paper questionnaires with all necessary customization done. • Conducted at least 2 months before the main training and beginning of fieldwork. Have enough time for necessary corrections in questionnaires and data entry application; prepare for the main training and fieldwork

6. How is a pre-test typically organised? • Preceded by a comprehensive training (2-3 weeks) • Often used as a training-of-trainers so that a sufficient number of trainers are available for fieldwork training. • Pre-test participants may also go on to be supervisors/ Editors

7. What is the typical scope of a pre-test? • Carried out in urban and rural areas but, could also include other population groups if necessary. • Need a sufficient number of households to draw conclusions (100 HHs minimum) • A pre-test report should be produced • After the pre-test review and conclusions nothing else should be added to the questionnaire

8. Pre-test considerations for Tablet/PDA based surveys 2 pretests are required: • The standard pretest of the paper questionnaires. • After finalization of the paper questionnaire and the tablet application is prepared, a second pretest on tablets/PDAs should be conducted (scheduled 1 month prior to the main training). • Emphasis on the testing of the application, but may also reveal additional problems in the questionnaire that were not caught during the first pretest.

9. MICS Fieldwork Team

10. Size and Composition of the Field Teams Number of interviewers depends on: • The sample size • Number of interviews that can be conducted per day (information provided from pre-test) • Logistics • Desired length of fieldwork (allowing for travel time from house to house and cluster to cluster) 2-3 months fieldwork should be the minimum in order to allow for mistakes in field to be identified & corrected

11. The fieldwork team composition MICS Recommendation: 1 Supervisor 1 Editor (not required when using tablets/PDAs) 1 Measurer 4 Interviewers (driver depending on country context) The MICS Fieldwork Duration and Staff Estimate Template can be used to assist in planning

12. Recruiting Field Staff Field staff should be: • Educated (secondary school or higher) • Fluent in the language(s) of respondents • Female (interviewers) but, female supervisors and editors are an advantage. • Some male if male questionnaire is being used • If tablets/PDAs are used field staff need to be “computer literate” and have some prior knowledge or experience in using computer, tablet, PDA or smart phone. Recruit and train more field personnel than needed (one fifth more) to take care of attrition

13. Supervisors’ role and responsibilities • Link between survey management team and field team • The well-being and safety of team members • Keeping up team morale • The maintenance of good data quality from their team

14. Supervisors’ role and responsibilities (cont.) • Manage the preparations for the fieldwork and moving between clusters • Assist interviewers in locating selected households • Coordinate the movement of interviewers and measurers in the field (perform HH spot checks including random rechecks of HH listing) • Support and monitor performance of the editor (Not applicable in CAPI): • Check the work of the field editor and assist with editing as needed • Spot-check the questionnaires that have been “edited” • Provide feedback to the editor • Organize additional training of interviewers if needed

15. Field Editors’ Role and Responsibilities • Monitor interviewer performance: • Observe several interviews every day • Edit all completed questionnaires before leaving the cluster • Conduct regular review sessions with interviewers • Compile completed questionnaires from a cluster and pack them up to be sent to the central office • Maintain a suitable mechanism to monitor the flow of questionnaires • Assist the supervisor as needed

16. Measurers' Role and Responsibilities • Measure the weight and height of all children under five • Maintenance of the equipment - Calibrate of the scale and measuring boards every morning

17. Interviewer’s Role and Responsibilities • Locate structures and households in the sampled clusters and complete the questionnaires • Check completed questionnaires • Revisits for households and/or women/mothers/caretakers/ men (at least 3 times) • Assist the measurer in conducting height and weight measurements of children • Complete control sheets

18. MICS Fieldwork Training

19. The MICS Training Recommendation For paper based surveys: • Minimum 14 day training (not including days off) Factor in adequate field practice days to build interviewer’s confidence • Additional 1-2 days for supervisors and editors • Additional 2-3 days training if anthropometry is included • One centralized training bringing all fieldworkers together • Facilitated by a group of core trainers that were involved in the pre-test and will be present throughout whole training • Participation of data entry personnel • Immediately followed by fieldwork

20. Training considerations when using tablets/PDAS The same as the paper based survey training but duration of training extended by: • at least 1 additional week for the tablet/PDA part • 4-5 additional field practice days. No participation of data entry personnel required

21. When the ideal is not possible • Sequential training Pros: The same trainers will do all training. Can do in different locations in the country. Cons: Immediate fieldwork is difficult to achieve if the trainers are also senior survey personnel Beware – difficult to standardize! • Training that happens at the same time in one venue, in parallel sessions Pros: All fieldwork starts at the same. Cons: Need a large venue and larger number of trainers.

22. Example Content of Training Course • Day 1: Objectives and survey instruments • Day 2-11: Survey procedures and questionnaires • Day 12-14: Pilot test (field exercise) with de-briefings • Day 15-21: Tablet data collection application • Day 21-25: Pilot test (field exercise with tablets/PDAs) with de-briefings

23. Example Content of Training Course (cont.) • Supervisor and editor training (2 days extra) • Household selection and map reading • Quality control • Practice • Anthropometry training (2-3 additional days for measurers)

24. Materials for Training • Large projector for questionnaire • Sufficient printed questionnaires and manuals • Final translated questionnaire (after pretest) • Enough copies for practice and for pilot test • Vaccination cards for practice (can be duplicated) • ORS packages, Vitamin A capsules, salt, contraceptives, bednetsetc • Equipment (measuring board, scales, salt test kits, GPS) • Sample maps and household listing forms • Control sheets (supervisors and editors)

25. Content of the Training Setting the agenda for interviewer training • Time to go over all questionnaires in detail (some modules require a lot more time than others) • Invite sector experts to introduce different modules • Include a lot of practice time • Use participatory techniques for training • Classroom practice should also include practice on “real” respondents before the pilot • Trainers need to always observe practice and go through completed modules and questionnaires to give feedback • Field work roles do not need to be assigned until towards the end of training

26. Practice! Practice!! Practice!!! • Role playing on approaching the household, handling refusals • Role playing on interview techniques, getting privacy • Front-of-class interview with real respondent • Using vaccination cards • Salt testing • Anthropometry • Sampling points (various ages, random tables etc) on the questionnaire, pictures etc.

27. Pilot Study • Rehearsal for fieldwork • Test all field procedures • Practice for interviewers, supervisors, editors and measurers • Allows trainers to see field staff in action • Do use rural and urban samples • Don’t use MICS sample clusters • Don’t use pilot study data for final sample • Do use questionnaires for data entry practice

28. Evaluation and testing of trainees Written tests: • One after the introduction of each questionnaire; one towards the end of training • Quizzes Observation • Careful observation of practice and reviewing completed questionnaires

29. Sensitizing communities and households to MICS Certain measures can be used to increase response rates: • Proper identification for interviewers (ID cards, letters of introduction from implementing agency) • National/regional/district broadcasts TV/radio/newspaper articles • Meeting with community/administrative leaders • Beginning the process of sensitizing at the HH listing stage • This can commence much earlier in the survey process

30. Other issues to plan for • Re-training of field personnel if necessary esp. Anthropometric measurement • Terms for the payment of salary/ per diems • Continuous monitoring of fieldwork – Critical at the beginning, in the middle and towards the end of fieldwork • Accommodation for field personnel Every decision  Implication on quality and cost

31. THANK YOU