translation of survey instruments l.
Skip this Video
Loading SlideShow in 5 Seconds..
Translation of survey instruments PowerPoint Presentation
Download Presentation
Translation of survey instruments

Loading in 2 Seconds...

play fullscreen
1 / 38

Translation of survey instruments - PowerPoint PPT Presentation

  • Uploaded on

Translation of survey instruments. Alisú Schoua-Glusberg, Ph.D. Research Support Services DC/AAPOR Presentation – 7/7/04. Translation methods for survey instruments. This presentation will focus on: problems involved in questionnaire translation

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Translation of survey instruments' - tamarr

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
translation of survey instruments
Translation of survey instruments

Alisú Schoua-Glusberg, Ph.D.

Research Support Services

DC/AAPOR Presentation – 7/7/04

translation methods for survey instruments
Translation methods for survey instruments

This presentation will focus on:

  • problems involved in questionnaire translation
  • methods and approaches for the translation of survey instruments,
  • use of qualitative methods to assess translation quality and the performance of the translated instrument,
  • providing survey translators the information they need.

Research Support Services - 2004

when is questionnaire translation necessary
When is Questionnaire Translation Necessary?
  • Cross-national studies
  • Cross-cultural studies
  • National studies by Federal Statistical Agencies
  • Studies of special populations
  • Studies where sample falls in areas with significant concentrations of speakers of other languages

Research Support Services - 2004

factors to consider to define the target audience of translated instruments
Factors to Consider to Define the Target Audience of Translated Instruments
  • Language minorities
  • Immigrant populations
  • Monolingual (or at least not


  • Language different from designers’
  • Lower level of education
  • Age at immigration
  • Hybrid culture
  • Different degrees of acculturation

Research Support Services - 2004

source target question characteristics to consider
Source & Target Question Characteristics to Consider
  • Meaning
  • Style
  • Complexity
  • Source flexibility
  • Cultural aspects
  • Existence of equivalent realities

Research Support Services - 2004

using previous translations of questions vs translating anew
Using Previous Translations of Questions vs. Translating Anew


  • Maintaining longitudinal comparability vs. improving questions
  • Is priority to compare with past translations or with English questionnaire?

Research Support Services - 2004

questionnaires are a complex text type
Questionnaires are a Complex Text Type
  • Little context to clarify ‘intended meaning’
  • Ambiguous
  • Measurement properties lead to ‘surveyspeak’ and ‘scalespeak’ (Harkness 1996)
  • Translators thus often not versed in questionnaire discourse ‘rules’
  • Special kind of conversation

Research Support Services - 2004

q uestion equivalence is expected in translation
Question ‘Equivalence’ is Expected in Translation

Expectation that the translated question…

  • Says the same thing
  • Means the same thing
  • Measures the same thing
  • Measures equivalently (e.g. scales)
  • Imposes same burden on respondents
  • Meets reliability and validity requirements

Research Support Services - 2004

translators job
Translators make decisions about:


Syntax (structure)


They need to understand:

Intended “meaning” in order to translate

Covert (measurement) intention and requirements

Translators’ Job

Research Support Services - 2004

translators therefore
Have little context and co-text

Have little supporting documentation

Often have no questionnaire ‘author’ to consult

May experience uncertainly about what to “match”

May make decisions based on their experience with other types of text

Translators therefore…

Research Support Services - 2004

risks of overly close translations
Risks of Overly Close Translations
  • Focus on meaning of words rather than meaning of questions
  • Rs are inadvertently asked a different question
  • Processing is more complex
  • Translated questionnaire sounds unidiomatic

Research Support Services - 2004

steps in producing and testing quex translations
Steps in producing and testing quex translations
  • Translation
  • Translation review
  • Decisions/Adjudication
  • Quality control
  • Qualitative research
  • Pretesting
  • Documentation

Research Support Services - 2004

survey translation approaches
Survey Translation Approaches
  • One translator - one translation (direct translation)
  • Multiple translators – one translation (split committee)
  • Multiple translators – multiple parallel translations (parallel committee)

Research Support Services - 2004

committee approach
Committee Approach
  • Three translators prepare translations independently (split or parallel)
  • Reconciliation meeting with referee
  • Qualitative research with monolinguals (focus groups and/or cognitive interviews)

Research Support Services - 2004

committee approach reconciliation meeting
Committee Approach: Reconciliation Meeting
  • Question-by-question review
  • Reaching consensus when possible
  • Providing alternatives if no consensus possible
  • Identifying terms/items for qualitative research

Research Support Services - 2004

committee approach referee s role
Committee Approach: Referee’s Role
  • Resolves style disagreements
  • Manages interaction
  • Brings survey researcher perspective
  • Keeps an eye on source version
  • Pushes for global decisions
  • Liaison with research team for consultation and documentation

Research Support Services - 2004

committee approach cognitive interviews
Committee Approach: Cognitive Interviews
  • Allow to administer all or part of instrument
  • Give a glimpse into thought processes
  • Allow to see how different alternative terms work

Research Support Services - 2004

committee approach focus groups
Committee Approach: Focus Groups
  • Permit to distinguish what is idiosyncratic
  • Allow us to listen to how Rs. use language
  • Allow to include more people in a shorter time
  • Allow to include different national origin Rs and see if they reach consensus

Research Support Services - 2004

advantages of committee team approaches
Advantages of Committee/ Team Approaches
  • Group process benefits
  • Include different varieties of language in translation team
  • Qualitative research that follows allows to incorporate the target population into process
  • Relatively low cost
  • Relatively quick

Research Support Services - 2004

selected committee translations 2000 2004
NSFG (U. Mich.)



ICARIS (Battelle)

NMHS (Battelle)

SLAITS Asthma Q. (Abt)

Job Corps Student Q. (Battelle)

Survey of Consumer Attitudes (U. Mich.)

Catholic Voters Political Attitudes (BRS)

McNair Program Evaluation (DIR)

L.A. Latino Eye Study (USC)

Women’s Health Initiative Q. (U. Mich.)

WHO Health & Performance Q. (Harvard)

PHDCN (Harvard)

Project Bread (UMass-Boston)

California Safe Schools (RAND)

CAHPS Dialysis Center (RAND)

National Children’s Study Pilot (Battelle)

Survey of Bioterrorism Pre-paredness (N.Y. Academy of Medicine)

Selected Committee Translations: 2000-2004

Research Support Services - 2004

assessing translations s ome procedures harkness pennell schoua glusberg asa 2003
Assessing Translations: some procedures(Harkness, Pennell, Schoua-Glusberg – ASA, 2003)
  • Textual assessments
    • Translation appraisal
    • Holistic approaches, e.g., TRAPD, committee
    • Back translation
  • Pretesting with bilinguals, e.g.,
    • Splits
    • Double administrations
  • Debriefings
  • Probe interviews
  • Think alouds with Respondent or Translator
  • Focus groups with sample population

Research Support Services - 2004

  • One person translates from source into target language.
  • A second person translates the target language version back into the source language.
  • A third person compares the original and the backtranslated source language versions.
  • Discrepancies are investigated.

Research Support Services - 2004

what is wrong with back translation
What is wrong with back- translation?

It is a black box: we know what went in, we know what came out, yet we know nothing about the adequacy of the target language version.

Research Support Services - 2004

why give q by q specs to translators
Why give Q-by-Q specs to translators?

In the absence of question-by-question specifications, translators make their own decisions…

  • to resolve ambiguities
  • to figure what “they” mean

These are most often not documented or even explicit. They happen in the translator’s mind.

Research Support Services - 2004

research conducted
Research Conducted
  • Goal: to examine translators’ decision processes
  • Vehicle: Recorded a translators’ Committee meeting
  • Method: Listened to tape searching for decisions made during discussions

Research Support Services - 2004

why this design
Why this design?
  • Committee discussion involves:
          • Review
          • Translation
          • Adjudication
  • Makes decisions verbally explicit

Research Support Services - 2004

transcription detail
Transcription Detail:
  • 2 hours of committee meeting
  • 11 pages of questionnaire
  • 110 questions

Research Support Services - 2004

example 1
Example 1

How would your parents feel if they found out you drank alcohol sometimes?

  • Not at all upset
  • A little upset
  • Pretty upset
  • Very upset

Research Support Services - 2004

discussion about upset
Discussion about “upset”

Did “they” mean…


Perturbed / Bothered?

Given context, committee decided to translate upset as angry.

Research Support Services - 2004

example 2
Example 2

I think sometimes it’s okay to cheat at school.

Strongly Agree



Strongly Disagree

Research Support Services - 2004

discussion about cheat
Discussion about ‘cheat’

Original Item: I think sometimes it’s okay to cheat at school.

First translation:

  • Creo que a veces está bien hacer trampa en la escuela.

Referee: “¿Hacer trampa? Everyone agrees?”

Orig. Tr.: To me it refers to copying in exams.

Tr2: “You can also copy when you are doing homework or other schoolwork, that is cheating too, not only in exams.” “And you could cheat in sports, so we need to be specific. In this case this refers to school work, not to playing volleyball.”

Resolution: Creo que a veces está bien copiar o hacer trampa en mis exámenes y tareas escolares.

(I think sometimes it’s okay to copy or cheat in my exams and schoolwork.)

Research Support Services - 2004

example 3
Example 3

How do you feel about your ability to care for your teen when they are sick or upset?

Discussion about ‘upset’.

In this case, the discussion centered around whether ‘upset’ meant angry, not feeling well, or bothered by something.

The committee went with the latter option.

(‘upset’: molesto)

Research Support Services - 2004

example 4
Example 4

How do you feel about your ability to discipline your teen?

Discussion about ‘discipline’. Do “they” mean:

  • Punish?
  • Teach them to behave properly?
  • Set rules for them?

Checked with client who opted for ‘punish’.

Research Support Services - 2004

example 5
Example 5

How do you feel about your ability to obtain needed resources for your teen?

Discussion about ‘resources’. Did “they” mean...

Material resources?


Client asked to preserve the ambiguity as much as possible. Translated as ‘recursos’.

Research Support Services - 2004

example 6
Example 6

During the past week, how often did you let this teen know you really care about him/her?

Discussion about ‘care’. Did “they” mean…



Committee decide to include both:

… how much you love him/her and are concerned about him/her

Research Support Services - 2004

providing specifications ideal model
Providing Specifications:Ideal Model

Three steps:

  • Deliver question-by-question specifications
  • Review original text with translation team member
  • Ask translators to make their decisions explicit and submit them for review.

Research Support Services - 2004

format for documentation
Format for Documentation

Research Support Services - 2004

recent current efforts to standardize translation procedures
Recent/Current Efforts to Standardize Translation Procedures
  • U.S. Census Bureau translation guidelines
  • International Social Survey Programme’s translation methods’ work
  • European Social Survey Implemented Procedures in 20+ countries
  • European Social Survey and Survey of Health, Aging and Retirement in Europe analyzing translation and documentation outputs
  • CAHPS Cultural Comparability Task Force
  • NCHS Translation Issues Forum
  • CSDI (Comparative Survey Design and Implementation Translation Task Force)

Research Support Services - 2004