Test Construction in Multilingual Contexts Edwin Klinkenberg Department of Social Sciences

1. Test Construction in Multilingual ContextsEdwin KlinkenbergDepartment of Social Sciences

2. Presentation • Background • Measurement of latent variables • Test construction • Appropiate construction procedures • Examples

3. Background • Sociometrics/ Psychometrics • Test Construction • Multilingual Experience • Flanders/ Wallonia • Germany • Poland • United Kingdom • Fryslân

4. International Relevance • Scientific Research • Cross – Cultural Psychology • Clinical research • Commercial • Increasing mobility of workforce • Application domain, globalisation • Scientific Associations • APA, EAPA, AERA • ITC

5. Key Researchers • R. K. Hambleton • F. J. R. van de Vijver • N.K. Tanzer • D. Bartram

6. Tests • Test - Measurement instrument, Scale or Questionnaire • - Set of stimuli for which a set of responses is collected • Object of measurement • Not (directly) observable • Hypothetical construct • θ (latent trait)

7. Manifest versus Latent • Manifest : Directly observable • Height • Weight • Latent: Not directly observable • Vocabularly • Language Attitude • Arithmetic Ability • Memory • Political Preference

8. Key Measurement Aspects • Reliability • Measurement Error • Validity • Did the test measure what it intended to measure • Construct/Content Validity • Chain of evidence

9. Test construction (Monolingual) • General Procedure • Assume Latent Trait (e.g. Self-confidence) • Item construction: indicative for the latent trait (e.g. 10 statements about Self-confidence) • Data Collection • Psychometrical Analysis • Reliability & Internal Structure • Validity Research

10. Key Principles Multiple Groups (Age, Etnicity, Sex) • Construct Equivalence • Depression/Mood across age groups • Personality characteristics • Memory: direct and delayed recall • Measurement Equivalence • Goal: Valid comparisons across groups

11. Test Construction (Multilingual) • Different Procedures • Application • For populations with same cultural and linguistic background (US to UK) • Translation • For populations with different mother tongues (e.g. nonverbal IQ tests) • Adaptation • For adaptation to different language/ culture (e.g. to replace idiosyncracies) • Assembly • Construction of a new test

12. Key Questions • Does the latent construct exist in the target culture? • Is the latent construct equally valid in the target culture? • Are the items appropriate across groups? • Can measurement equivalence be achieved?

13. General Procedure • Cross-Lingual comparison? • Simultaneous test construction • Different experts/psychologist • Creation of a common scale • Contact your local Psychometrician!

14. Example 1 • EUROHARPID – The European Hamonization Project for Instruments in Dementia (2003). • Harmonization of Alzheimer Disease Assessment Scales • Adaptation of CAMDEX to 6 different cultures

15. Example 1 • Item adaptation for cognitive screening battery • Long term memory • UK NL F? G?

16. Example 2 Personality Assessement • Latent Construct: Assertiveness • Dutch: Assertiviteit • German/Polish: Rudeness? • Idiosyncracies • “I will not allow other people to eat the cheese of my bread” • “I call a spade a spade”

17. Example 3 • Schlichting Test for Language Production • Target group: children form age 4 • Item: Picture of crocodile • Dutch: Acceptable and appropriate • Flemish: Item too scary • Explanation: familiarity with item through children’s literature

18. Example 4 • IQ test: subtest Arithmetic • Source: Dutch (NL) • Target: Polish (P) • Problem: reverse item difficulties • Arithmetic items: Easy (NL), Difficult (P) • Mathematical items: Difficult (NL), Easy (P) • Explanation: Different focus on Arithmetics in primary and secondairy education

19. Thank you for your attention