Quality Indicators in Data Collection

1. Quality Indicators in Data Collection Q2010 Helsinki Gustav Haraldsen Øyvin Kleven, Anne Sundvoll gha@ssb.no, kle@ssb.no, asu@ssb.no Department of Data Collection Statistics Norway

2. Outline • Strategy for data collection – main objectives • Quality management system (check lists and indicators) • Components/indicators - examples • Challenges in implementing the information system

3. Main objectives – Strategy for data collection • Motivated respondents • Data collection of higher quality • Identify strategic focus areas and key process indicators • Measure effects of actions taken

4. Reduce perceived response burden Improve Data collection instruments Full-time accessible systems for electronic reporting Reduce actual response burden Improve register quality Reduce nonresponse Improve data reception systems and checks Standardisation of data collection routines Strategic focus areas Professional support service Me Motivated respondents Increased product quality Adapt data collection to respondents’ Information system and preferences Reduce errors in processing Develop and utilize quality indicators Develop information system for data collection

5. Key process indicators Actual response burden Respondent helpdesk system data Infrastructure performance Indicators Perceived response burden Register quality indicators Contact rate Cooperation rate Paradata Percentage electronic reporting Me Motivated respondents Increased product quality Rapid data reporting indicator Check list/ process indicators Complete data reporting indicator Data collection specification form Standard and new product quality indicators Customer evaluation form

6. Examples – key process indicators Infrastructure performance indicator

7. Examples – key process indicators

8. Rapid data indicators ReminderImposed fine Amount of units reported before reminder (week 9): 51% Amount of units imposed compulsory fine (week 19): 18% Amount of units followed up by the state agency for the recovery of fines: 4%

9. Project planning Data Collection Information System (DCIS) Expectations Specification form Customer evaluation Documentation Sample management Checklist Product Response burdens and gratifications Response process Feedback Accomplishment Quality Design Paradata Satisfaction

10. Order specification Specification form • Title, objectives & data collection • Client information • Budget • Sampling information • Design • Questionnaire development • Post survey tasks • Deliveries

11. Checklist Activities • Declaration of the work being done in the steps of developing and collecting the data. • Comprises 70 questions related to work done in the following processes: • Questionnaire design • Sample design • Training of staff (interviewers and support services) • Data collection procedures • Data reception • 1st line data editing

12. Perceived response burden Response burden

13. Product Product quality • Response rate • R-indicator • S-indicator • Item nonresponse • Validity indicators • Reliability indicators

14. Response rate

15. R-indicator

16. Response rate, net sample result and trend line. Perceived health

17. Customer evaluation Customer evaluation • The delivery compared with objectives stated in the order • Satisfaction with the communication during the project • Satisfaction with the work done

18. Project planning Data Collection Information System (DCIS) Expectations Specification form Customer evaluation Documentation Sample management Checklist Product Response burdens and gratifications Response process Feedback Accomplishment Quality Design Paradata Satisfaction