Immigrant respondents and quality in population surveys
1 / 14

Immigrant respondents and quality in population surveys: - PowerPoint PPT Presentation

  • Uploaded on

Immigrant respondents and quality in population surveys: . sampling, non-response & questionnaire design. Q2014, Vienna , 2.6.2014 Liisa Larja, Ada Kotilainen. Content of the presentation. Survey design Field work Questionnaire design Conclusions.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Immigrant respondents and quality in population surveys: ' - adonai

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Immigrant respondents and quality in population surveys

Immigrant respondents and quality in population surveys:

sampling, non-response &

questionnaire design

Q2014, Vienna, 2.6.2014

Liisa Larja, Ada Kotilainen

Content of the presentation
Content of the presentation

  • Survey design

  • Fieldwork

  • Questionnaire design

  • Conclusions

Problem 1 immigrants are underrepresented in the general population surveys
Problem 1: immigrantsareunderrepresented in the general populationsurveys

Problem 2 low response rate among immigrants decreases the quality of general population surveys
Problem 2: Lowresponserateamongimmigrantsdecreases the quality of general populationsurveys

Problem 3: Disaggregating results according to immigrant status is not possible when there are too few immigrants in the general population survey samples

  • No results: problems go unnoticed. Separate studies: comparison?

  • EU immigrant integration indicators derive from Labour Force Survey (LFS), Survey on Income and Living Conditions (SILC) and PISA

  • LFS ad hoc module 2014 on Labour market situation of immigrants and their descendants  in the Finnish LFS the number or immigrants would be very low (~900).

  • Oversampling – expensive

  • Pooling of data from several years – timeliness and quality problems, not possible for LFS ad hoc module 2014

The uth study survey on work and well being among persons of foreign origin
The ”UTH”-study collected within 1 interview(Survey on work and well-being among persons of foreign origin)

  • N = 5 400 individuals of ”foreign origin” (= both parents foreign-born)  all languages, all areas, citizens, foreigners, 2nd generation….

  • Sampling from population register. At least 6 % overcoverage: moved out from Finland

  • Computer assisted personal interviews (CAPI). Normally done in respondents home, but 50 % of the respondents wished for some other place.

  • Standardized interview technique

2 field work
2. Field work collected within 1 interview

  • For the first 7 reference weeks the response rates have been over 70 % , which is slightly better than in corresponding general population surveys.

  • Translating questionnaires and programming them into Blaise also “monolingual” interviewers were able to use language versions

  • Translating cover letters and info sheets on data secrecy

  • Recruiting 9 multilingual interviewers (through NGO’s) in addition to 120 permanent interviewers

Field work
collected within 1 interviewfieldwork

  • Training, motivating & encouraging interview staff (sensitive questions, interpretation, motivating different types of respondents)

  • Investing in searching for missing contact information (different registers, home visits & notes, neighbours, googling, social media, e-mail, SMS). Average time spent per sampling unit 3,5 hours (interview 0,7 h).

  • Non-response due to same reasons as for general populations: non-contacts, no time, no interest.

3 questionnaire testing an iterative process
3. collected within 1 interviewQuestionnairetesting: an iterativeprocess

  • 21 cognitive interviews (incl. one proxy). Focus on new questions. How do the respondents interpret the questions, how do they choose their response, do they leave something unsaid?

  • 6 pilot interviews (incl. one ad hoc translation). Testing the entire questionnaire. Length and flow of the interview, spontaneous reactions to the questions.

  • 21 pilot interviews. Testing of languages versions, testing of distant translation by phone and video (+ length and spontaneous reactions)

Questionnaire design comprehension problems
Questionnaire design: Comprehension problems collected within 1 interview

  • Linguistic problems (difficult words and sentence structures, Likert scales)  incorrect interpretation of the questions  reformulation of the questions

    • Many misinterpretations emerged only in the cognitive interviews.

  • Standardized scales  “easy Finnish” versions + help-text, to decrease need to deviate from standardized interview technique

  • Unfamiliarity with the interaction pattern of structured interview

     The length of the questionnaire was cut by 40 %

     Investing in language versions.

Questionnaire design translations
Questionnaire design: translations collected within 1 interview

  • Finnish, Swedish and English + 9 additional languages.  68 % had language version in their mother tongue

  • The number of language versions was prioritized over quality of translation:

    • Professional translation, but no back-translation, committee translation etc.

    • Interviewers proof-read the questionnaires and only obvious mistakes were corrected.

  • With only a few (rigorously tested) language versions there would be even more misunderstanding due to broken Finnish/English as compared to linguistic differences due to differences in translations.

Questionnaire testing trust
Questionnaire collected within 1 interviewtesting: trust

  • Questions on reasons for migration, language skills, cultural identity, labour market obstacles, health problems, experiences of violence, pregnancies etc. were found sensitive, especially with proxy respondent or when a friend was assisting with translation

     routing was changed to skip some of these questions for proxy respondents and when there were other persons assisting in the interview

     response cards were added to allow responding without speaking

  • The respondents expressed concerns on whether the information they give may have an effect on decisions on their social benefits etc.  info sheet on data secrecy

  • The respondents admitted, that the imagined opinion of the interviewer influenced option selection  interviewer training on sensitivity

Conclusions collected within 1 interview

  • Oversampling immigrants can be done cost-effectively by collecting data for multiple surveys during the same interview

  • High response rates can be achieved by investing in quality in field work (language version, working time in contacting)

  • Questionnaire testing is essential and it is best done iteratively

  • Finding solutions to the problems of comprehension, and trust is essential as they affect response behaviour and produce bias in the results.