survey design and cross cultural issues l.
Skip this Video
Loading SlideShow in 5 Seconds..
Survey Design and Cross cultural Issues PowerPoint Presentation
Download Presentation
Survey Design and Cross cultural Issues

Loading in 2 Seconds...

play fullscreen
1 / 30

Survey Design and Cross cultural Issues - PowerPoint PPT Presentation

  • Uploaded on

Survey Design and Cross cultural Issues. Avinish Chaturvedi Carlos Torelli. Agenda. Brief Review of the readings Methods and Data Analysis for Cross-Cultural Research Discussion. Heine et al., (2002).

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Survey Design and Cross cultural Issues' - omer

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
survey design and cross cultural issues

Survey Design and Cross cultural Issues

Avinish Chaturvedi

Carlos Torelli

  • Brief Review of the readings
  • Methods and Data Analysis for Cross-Cultural Research
  • Discussion
heine et al 2002
Heine et al., (2002)
  • Reference group effect: people from different cultural groups use different referents in their self-reported values (i.e., compare with different others)
  • Low Individualism -------------------- High Individualism 
  • ------------- Japanese High (7) ------ American High (7) 

An apparent no difference is actually a significant difference

wong et al 2003
Wong et al., (2003)
  • Problems with Likert scales that contain a mixture of positive-worded items (PWI) and reverse-worded items (RWI) in cross-cultural research.
  • Why use RWI?:
    • Reduce acquiescence bias.
  • General problems with RWI?
  • May negatively impact internal consistency.
  • May disrupt dimensionality (RWI loading in a separate factor).
brislin chapter
Brislin chapter:
  • Why do we need to change/modify the survey instrument in a different culture
  • Linking it with previous discussion
  • What factors change across cultures
  • Why not to use existing scales
types of cross cultural studies van de vijver chapter
Types of cross cultural studies: Van de Vijver chapter
  • Generalizability studies:

The design of a generalizability study is usually a replication of the original study.

- When the goals of cross cultural studies are defined as both delineating and explaining cross cultural differences, generalizability studies capitalize on first and often ignore the second.

types of cross cultural studies van de vijver chapter7
Types of cross cultural studies: Van de Vijver chapter
  • Studies of psychological differences

These are often based on a less elaborate theoretical framework than are generalizability studies

types of cross cultural studies van de vijver chapter8
Types of cross cultural studies: Van de Vijver chapter
  • Theory driven studies

- These studies test a theory of cross cultural differences. Theory driven studies are more systematic. Such studies are designed and carried out to critically test a hypothesis. These studies provide powerful tets of theories of cross cultural differences, which is one of the main goals of cross cultural research.

types of cross cultural studies van de vijver chapter9
Types of cross cultural studies: Van de Vijver chapter
  • External validation studies

- These studies take observed cross cultural level or structural differences as their starting point and scrutinize these differences either by exploring their antecedents or by testing interpretations of these differences.

cross cultural issues
Cross cultural issues
  • Researchers run the risk of missing aspects of a phenomenon as viewed by people in other cultures.
  • They risk imposing conclusions based on concepts which exist in their cultures but which are foreign, or at least partially incorrect, when used in other culture.
emic and etic distinction
Emic and Etic distinction
  • How to determine what attributes of a construct are static, i.e., unchangeable across cultures
  • What a priori measures can be employed in this regard
  • Example from Brislin chapter: Authoritarian and conservatism.
  • What kind of questions are more sensitive to cultural variations?
method and data analysis
Method and Data analysis
  • In cross cultural studies what kind of sampling measures should be taken
  • Convenience Sampling:

Studies adopting this sampling scheme often fall into the category of psychological differences studies

method and data analysis13
Method and Data analysis
  • Systematic Sampling:

Cultures are selected in a systematic, theory guided fashion. These studies usually fall into the categories of theory driven or generalizability studies. Cultures are selected in this procedure because they represent different values on a theoretical continuum

  • Random Sampling:

It involves the sampling of a large number of cultures randomly. This strategy is preferable for generalizability studies, in which a universal structure or a pan cultural theory is evaluated.

method and data analysis15
Method and Data analysis
  • Choosing Cultures:

How does choice of cultures in a research design changes when:

a. we are looking for similarities

b. we are looking for differences.

method and data analysis16
Method and Data analysis
  • Administration of instruments:



interaction between two

response procedures

stimulus materials

method and data analysis17
Method and Data analysis
  • Remedies:

A priori and post priori techniques.

Similarly prior and post hoc remedies can be used to alleviate problems of sample incomparability.

Application of a monotrait multimethod matrix in order to examine the influence of response procedures is useful

What are the other ways through which this effect can be minimized??

method and data analysis18
Method and Data analysis
  • Changing the survey instrument




In which scenario, each of above three technique becomes useful?

method and data analysis19
Method and Data analysis
  • Validity Enhancement

The major criterion in the choice of application, adaptation and assembly is the type of bias expected. If there are serious concerns that construct bias could play a role, adaptation or assembly should be chosen.

Indigenization which aims at maximizing the appropriateness of psychological theories and instruments to local cultures, will often amount to the assembly of new instruments

method and data analysis20
Method and Data analysis
  • Obtaining linguistically equivalent instruments:

Translation and back translation

Committee approach

method and data analysis21
Method and Data analysis
  • Translation

Translation is more than producing text in another language. Translators should know or be made aware of the linguistic and cultural differences that could influence responses to translated or adapted instruments.

  • Guidelines for translating and adapting psychological and educational instruments:
  • What are the other translation issues involved
study 1 experts ratings
Study 1 – Experts’ Ratings
  • Sample: Japanese specialists living in North America
  • What were the questions?
  • How much items in Singelis (1994) scale characterize Japanese or Americans
  • Findings:
    • Experts were in agreement with the common view for all 23 items (high face validity).
  • These findings suggest that the use of different reference groups can obscure cross-cultural differences (Americans use other Americans, and East Asians other East Asians).
study 2 manipulation of reference groups
Study 2 – Manipulation of Reference Groups
  • Sample: Canadian biculturals with knowledge about Japan and Canada.
  • Instruments:
    • Singelis (1994) in standard form.
    • Answers in comparison with most Japanese.
    • Answers in comparison with most Americans.
  • Findings:
  • Weak support for the common view using standard format.
  • When comparing with people from the other culture: strong support for the common view.
  • When comparing with people from their own culture: evidence from acculturation.
study 3 within culture
Study 3 – Within-culture
  • Sample: Asian Canadians vs. European Canadians, and returnee Japanese vs. Japan-bound Japanese
  • Common referent for each group.
  • Findings:
  • European Canadians scored higher in independence and lower on interdependence than Asian Canadians
  • Returnee Japanese scored higher on independence and lower on interdependence than Japan-bound Japanese
  • Likert scales capture one’s feelings relative to a comparison group or shared norm?
    • Is it always the case? The case of self-esteem?
    • When can Likert scales assess one’s feelings without a referent? When do people use less social comparison for answering (i.e., introspection)?
  • Remmedial measures:
    • Measure at the cultural level
    • Prompt comparisons with an standard (average peer)
    • Responses to information that is consistent/inconsistent with implicit theories (new information)
    • Use of behavioral measures
    • Items with concrete, objective response options
  • Implications from the reference-group effect:
  • Use of subjective Likert scales most valid for identifying differences within rather than between groups  What do you think?
the mvs
  • The MVS in cross-cultural settings:
    • Good reliability and validity in the U. S.
    • Questionable reliability and validity abroad.
  • Why the problems outside the U. S.?
  • Negations and contradictions can be confusing
  • Languages can employ different ways of making negation or contradiction.
  • Differences in how PWI and RWI are interpreted (cultural norms regarding agreeableness?)
mvs across cultures study 1
MVS across cultures – Study 1
  • Countries: the U. S., Singapore, Japan, Korea, and Thailand.
  • Prediction:
    • Negative correlation with life satisfaction in wealthy countries only.
    • Positive correlation with individualism across all countries
  • Findings:
  • MVS specified as a 2nd. Order factor with 3 dimensions (happiness, centrality, and success) showed a poor fit  systematic error?
  • PWI and RWI correlated in the U.S., but weakly or uncorrelated elsewhere RWI being responded differently?
  • Divergent correlations for materialism and life satisfaction
  • Revised Model: include two method factors  fit comparable to those obtained in the U.S., but construct validity still an issue
  • Support for the notion that RWI might have led to the poor initial performance
mvs across cultures study 2
MVS across cultures – Study 2
  • Objectives:
  • Assess conceptual meaning of materialism.
  • Show that other scales that use PWI and RWI have the same problems.
  • Evaluate alternative scales  interrogative format
  • Predictions:
    • Show conceptual equivalence by using adjectives
    • Other scales that include RWI should show two-factors structures
    • Inquiring about a respondent’s position on an issue would minimize “agreement for the sake of being polite”
  • What do you think about the way conceptual equivalence was assessed?
  • Does it assess the 3-dimensions?
  • What does the evidence that other scales show two factors mean?
  • Does it prove that agreeableness is the underlying issue? What about the findings in the general discussion?
  • Acquiescence vs. “substantive cultural differences”?
    • Are the results driven by acquiescence or not?
    • Are the meanings of the MVS items different across cultures due to religion and beliefs? If so, what is the explanation for the results with the other 4 scales?
    • Can there be any referent group effect?
  • Recommendations:
    • Use only PWI
    • Use interrogative formats
    • Which one is better?