immigrant respondents and quality in population surveys n.
Skip this Video
Download Presentation
Immigrant respondents and quality in population surveys:

Loading in 2 Seconds...

play fullscreen
1 / 14

Immigrant respondents and quality in population surveys: - PowerPoint PPT Presentation

  • Uploaded on

Immigrant respondents and quality in population surveys: . sampling, non-response & questionnaire design. Q2014, Vienna , 2.6.2014 Liisa Larja, Ada Kotilainen. Content of the presentation. Survey design Field work Questionnaire design Conclusions.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Immigrant respondents and quality in population surveys:' - adonai

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
immigrant respondents and quality in population surveys

Immigrant respondents and quality in population surveys:

sampling, non-response &

questionnaire design

Q2014, Vienna, 2.6.2014

Liisa Larja, Ada Kotilainen

content of the presentation
Content of the presentation
  • Survey design
  • Fieldwork
  • Questionnaire design
  • Conclusions
problem 2 low response rate among immigrants decreases the quality of general population surveys
Problem 2: Lowresponserateamongimmigrantsdecreases the quality of general populationsurveys

Problem 3: Disaggregating results according to immigrant status is not possible when there are too few immigrants in the general population survey samples

  • No results: problems go unnoticed. Separate studies: comparison?
  • EU immigrant integration indicators derive from Labour Force Survey (LFS), Survey on Income and Living Conditions (SILC) and PISA
  • LFS ad hoc module 2014 on Labour market situation of immigrants and their descendants  in the Finnish LFS the number or immigrants would be very low (~900).
  • Oversampling – expensive
  • Pooling of data from several years – timeliness and quality problems, not possible for LFS ad hoc module 2014
the uth study survey on work and well being among persons of foreign origin
The ”UTH”-study (Survey on work and well-being among persons of foreign origin)
  • N = 5 400 individuals of ”foreign origin” (= both parents foreign-born)  all languages, all areas, citizens, foreigners, 2nd generation….
  • Sampling from population register. At least 6 % overcoverage: moved out from Finland
  • Computer assisted personal interviews (CAPI). Normally done in respondents home, but 50 % of the respondents wished for some other place.
  • Standardized interview technique
2 field work
2. Field work
  • For the first 7 reference weeks the response rates have been over 70 % , which is slightly better than in corresponding general population surveys.
  • Translating questionnaires and programming them into Blaise also “monolingual” interviewers were able to use language versions
  • Translating cover letters and info sheets on data secrecy
  • Recruiting 9 multilingual interviewers (through NGO’s) in addition to 120 permanent interviewers
field work
  • Training, motivating & encouraging interview staff (sensitive questions, interpretation, motivating different types of respondents)
  • Investing in searching for missing contact information (different registers, home visits & notes, neighbours, googling, social media, e-mail, SMS). Average time spent per sampling unit 3,5 hours (interview 0,7 h).
  • Non-response due to same reasons as for general populations: non-contacts, no time, no interest.
3 questionnaire testing an iterative process
3. Questionnairetesting: an iterativeprocess
  • 21 cognitive interviews (incl. one proxy). Focus on new questions. How do the respondents interpret the questions, how do they choose their response, do they leave something unsaid?
  • 6 pilot interviews (incl. one ad hoc translation). Testing the entire questionnaire. Length and flow of the interview, spontaneous reactions to the questions.
  • 21 pilot interviews. Testing of languages versions, testing of distant translation by phone and video (+ length and spontaneous reactions)
questionnaire design comprehension problems
Questionnaire design: Comprehension problems
  • Linguistic problems (difficult words and sentence structures, Likert scales)  incorrect interpretation of the questions  reformulation of the questions
    • Many misinterpretations emerged only in the cognitive interviews.
  • Standardized scales  “easy Finnish” versions + help-text, to decrease need to deviate from standardized interview technique
  • Unfamiliarity with the interaction pattern of structured interview

 The length of the questionnaire was cut by 40 %

 Investing in language versions.

questionnaire design translations
Questionnaire design: translations
  • Finnish, Swedish and English + 9 additional languages.  68 % had language version in their mother tongue
  • The number of language versions was prioritized over quality of translation:
    • Professional translation, but no back-translation, committee translation etc.
    • Interviewers proof-read the questionnaires and only obvious mistakes were corrected.
  • With only a few (rigorously tested) language versions there would be even more misunderstanding due to broken Finnish/English as compared to linguistic differences due to differences in translations.
questionnaire testing trust
Questionnairetesting: trust
  • Questions on reasons for migration, language skills, cultural identity, labour market obstacles, health problems, experiences of violence, pregnancies etc. were found sensitive, especially with proxy respondent or when a friend was assisting with translation

 routing was changed to skip some of these questions for proxy respondents and when there were other persons assisting in the interview

 response cards were added to allow responding without speaking

  • The respondents expressed concerns on whether the information they give may have an effect on decisions on their social benefits etc.  info sheet on data secrecy
  • The respondents admitted, that the imagined opinion of the interviewer influenced option selection  interviewer training on sensitivity
  • Oversampling immigrants can be done cost-effectively by collecting data for multiple surveys during the same interview
  • High response rates can be achieved by investing in quality in field work (language version, working time in contacting)
  • Questionnaire testing is essential and it is best done iteratively
  • Finding solutions to the problems of comprehension, and trust is essential as they affect response behaviour and produce bias in the results.