immigrant respondents and quality in population surveys n.
Download
Skip this Video
Download Presentation
Immigrant respondents and quality in population surveys:

Loading in 2 Seconds...

play fullscreen
1 / 14

Immigrant respondents and quality in population surveys: - PowerPoint PPT Presentation


  • 127 Views
  • Uploaded on

Immigrant respondents and quality in population surveys: . sampling, non-response & questionnaire design. Q2014, Vienna , 2.6.2014 Liisa Larja, Ada Kotilainen. Content of the presentation. Survey design Field work Questionnaire design Conclusions.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Immigrant respondents and quality in population surveys:' - adonai


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
immigrant respondents and quality in population surveys

Immigrant respondents and quality in population surveys:

sampling, non-response &

questionnaire design

Q2014, Vienna, 2.6.2014

Liisa Larja, Ada Kotilainen

content of the presentation
Content of the presentation
  • Survey design
  • Fieldwork
  • Questionnaire design
  • Conclusions
problem 2 low response rate among immigrants decreases the quality of general population surveys
Problem 2: Lowresponserateamongimmigrantsdecreases the quality of general populationsurveys
slide5

Problem 3: Disaggregating results according to immigrant status is not possible when there are too few immigrants in the general population survey samples

  • No results: problems go unnoticed. Separate studies: comparison?
  • EU immigrant integration indicators derive from Labour Force Survey (LFS), Survey on Income and Living Conditions (SILC) and PISA
  • LFS ad hoc module 2014 on Labour market situation of immigrants and their descendants  in the Finnish LFS the number or immigrants would be very low (~900).
  • Oversampling – expensive
  • Pooling of data from several years – timeliness and quality problems, not possible for LFS ad hoc module 2014
the uth study survey on work and well being among persons of foreign origin
The ”UTH”-study (Survey on work and well-being among persons of foreign origin)
  • N = 5 400 individuals of ”foreign origin” (= both parents foreign-born)  all languages, all areas, citizens, foreigners, 2nd generation….
  • Sampling from population register. At least 6 % overcoverage: moved out from Finland
  • Computer assisted personal interviews (CAPI). Normally done in respondents home, but 50 % of the respondents wished for some other place.
  • Standardized interview technique
2 field work
2. Field work
  • For the first 7 reference weeks the response rates have been over 70 % , which is slightly better than in corresponding general population surveys.
  • Translating questionnaires and programming them into Blaise also “monolingual” interviewers were able to use language versions
  • Translating cover letters and info sheets on data secrecy
  • Recruiting 9 multilingual interviewers (through NGO’s) in addition to 120 permanent interviewers
field work
…fieldwork
  • Training, motivating & encouraging interview staff (sensitive questions, interpretation, motivating different types of respondents)
  • Investing in searching for missing contact information (different registers, home visits & notes, neighbours, googling, social media, e-mail, SMS). Average time spent per sampling unit 3,5 hours (interview 0,7 h).
  • Non-response due to same reasons as for general populations: non-contacts, no time, no interest.
3 questionnaire testing an iterative process
3. Questionnairetesting: an iterativeprocess
  • 21 cognitive interviews (incl. one proxy). Focus on new questions. How do the respondents interpret the questions, how do they choose their response, do they leave something unsaid?
  • 6 pilot interviews (incl. one ad hoc translation). Testing the entire questionnaire. Length and flow of the interview, spontaneous reactions to the questions.
  • 21 pilot interviews. Testing of languages versions, testing of distant translation by phone and video (+ length and spontaneous reactions)
questionnaire design comprehension problems
Questionnaire design: Comprehension problems
  • Linguistic problems (difficult words and sentence structures, Likert scales)  incorrect interpretation of the questions  reformulation of the questions
    • Many misinterpretations emerged only in the cognitive interviews.
  • Standardized scales  “easy Finnish” versions + help-text, to decrease need to deviate from standardized interview technique
  • Unfamiliarity with the interaction pattern of structured interview

 The length of the questionnaire was cut by 40 %

 Investing in language versions.

questionnaire design translations
Questionnaire design: translations
  • Finnish, Swedish and English + 9 additional languages.  68 % had language version in their mother tongue
  • The number of language versions was prioritized over quality of translation:
    • Professional translation, but no back-translation, committee translation etc.
    • Interviewers proof-read the questionnaires and only obvious mistakes were corrected.
  • With only a few (rigorously tested) language versions there would be even more misunderstanding due to broken Finnish/English as compared to linguistic differences due to differences in translations.
questionnaire testing trust
Questionnairetesting: trust
  • Questions on reasons for migration, language skills, cultural identity, labour market obstacles, health problems, experiences of violence, pregnancies etc. were found sensitive, especially with proxy respondent or when a friend was assisting with translation

 routing was changed to skip some of these questions for proxy respondents and when there were other persons assisting in the interview

 response cards were added to allow responding without speaking

  • The respondents expressed concerns on whether the information they give may have an effect on decisions on their social benefits etc.  info sheet on data secrecy
  • The respondents admitted, that the imagined opinion of the interviewer influenced option selection  interviewer training on sensitivity
conclusions
Conclusions
  • Oversampling immigrants can be done cost-effectively by collecting data for multiple surveys during the same interview
  • High response rates can be achieved by investing in quality in field work (language version, working time in contacting)
  • Questionnaire testing is essential and it is best done iteratively
  • Finding solutions to the problems of comprehension, and trust is essential as they affect response behaviour and produce bias in the results.