Misresponse to reversed and negated items in surveys a review
Download
1 / 24

Misresponse to Reversed and Negated Items in Surveys: A Review - PowerPoint PPT Presentation


  • 74 Views
  • Uploaded on

Misresponse to Reversed and Negated Items in Surveys: A Review. Bert Weijters Hans Baumgartner. Motivation. Should reversed items be included in multi-item summative scales?

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Misresponse to Reversed and Negated Items in Surveys: A Review' - cody-daniels


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Misresponse to reversed and negated items in surveys a review

Misresponse to Reversed and Negated Items in Surveys: A Review

Bert Weijters

Hans Baumgartner


Motivation
Motivation Review

  • Should reversed items be included in multi-item summative scales?

    • The use of reversed items in surveys often leads to problems (low reliabilities, poor fit, distorted factor structure);

    • Reversed items control for acquiescence, serve as cognitive “speed bumps” and may encourage more complete coverage of the construct’s domain of content;

  • If reversed items are to be used, does it matter whether the reversal is achieved through negation or through other means?

  • What’s the link between reversal and negation, what types of MR result, what psychological mechanisms are involved, and how can MR be avoided?


Item reversal vs item negation
Item reversal vs. item negation Review

  • Authors often fail to draw a clear distinction between reversals and negations and use ambiguous terms such as ‘negatively worded items’, which makes it unclear whether they refer to reversed or negated items, or both;

  • Examples from the Material Values scale (Richins and Dawson 1992):

    • It sometimes bothers me quite a bit that I can't afford to buy all the things I’d like.

    • I have all the things I really need to enjoy life.

    • I wouldn't be any happier if I owned nicer things.


Empirical data
Empirical data Review

we analyzed items from volumes 1 through 36 of JCR (1974 till the end of 2009) and volumes 1 through 46 of JMR (1964 to 2009);

we included all Likert-type scales for which the items making up the scale were reproduced in the article and factor loadings or item-total correlations were reported;

total of 66 articles in which information about 1330 items measuring 314 factors was provided;

of the 1330 distinct items in the data set, 608 came from JCR and 722 from JMR;


Item negation
Item negation Review

  • Items can be stated either as an assertion (affirmation) or as a denial (disaffirmation) of something (Horn 1989);

  • Negation is a grammatical issue;

  • Classification of negations in terms of two dimensions:

    • what part of speech is negated (how a word is used in a sentence: as a verb, noun/pronoun, adjective, adverb or preposition/conjunction);

    • how the negation is achieved (by means of particle negation, the addition of no, the use of negative affixes, negative adjectives and adverbs, negative pronouns, or negative prepositions);


Item reversal
Item reversal Review

  • an item is reversed if its meaning is opposite to a relevant standard of comparison (semantic issue);

  • three senses of reversal:

    • reversal relative to the polarity of the construct being measured;

    • reversed relative to other items measuring the same construct:

      • reversal relative to the first item

      • reversal relative to the majority of the items

    • reversal relative to a respondent’s true position on the issue under consideration (Swain et al. 2008);


Item reversal cont d
Item reversal (cont’d) Review

in our data set of 1330 items, between 83 and 86 percent of items were nonreversed (depending on the definition of reversal);

the proportion of factors (or subfactors in the case of multi-factor constructs) that do not contain reversed items was 70 percent;

only 8 percent of factors (out of 314 factors) were composed of an equal number of reversed and nonreversed items (i.e., the scale was balanced);




Misresponse to negated and reversed items
Misresponse to negated and reversed items Review

MR → within-participant inconsistency in response to multiple items intended to measure the same construct;


Theoretical explanations of mr comprehension
Theoretical explanations of MR: ReviewComprehension

  • Careless responding (Schmitt and Stults 1985):

    • respondents fail to pay careful attention to individual items and respond based on their overall position on an issue [RMR];

    • more likely when a reversed item is preceded by a block of nonreversed items;

    • Remedies:

      • alert Rs to the presence of reversed items, avoid tedious surveys, and be careful with involuntary Rs;

      • increase Rs’ ability and avoid distractions;

      • use balanced scales, alternate the keying direction, and disperse the items;


Theoretical explanations of mr comprehension cont d
Theoretical explanations of MR: ReviewComprehension (cont’d)

  • Reversal ambiguity:

    • Rs may not view antonyms as polar opposites [POMR];

    • contradictories vs. contraries:

      • Antonyms can be contradictories or contraries, depending on whether they are bounded or unbounded (Paradis and Willners 2006);

      • Negations of the same core concept are clearly contradictories, but things are more complicated when the core concept differs;

    • simultaneous disagreement is more likely when items are worded extremely (McPherson and Mohr 2005);

    • “Buddhism’s ontology and epistemology appear to make East Asians relatively comfortable with apparent contradictions” (Wong et al. 2003, p. 86) [RMR];


Theoretical explanations of mr comprehension cont d1
Theoretical explanations of MR: ReviewComprehension (cont’d)

  • Remedies:

    • use more sophisticated procedures to identify appropriate antonyms (formulate linguistic contrasts in two stages; see Dickson and Albaum 1977);

    • may be particularly useful in cross-cultural research;

    • bounded antonyms have to be pretested and unbounded antonyms have to be used with care;

    • extreme statements should be avoided;


Theoretical explanations of mr retrieval
Theoretical explanations of MR: ReviewRetrieval

  • Item-wording effects:

    • Confirmation bias (Davies 2003; Kunda et al. 1993);

    • Directly applicable to antonymic reversals;

    • For negation reversals, confirmation bias can lead to MR if a non-negated polar opposite schema is readily available (Mayo et al. 2004);

    • Remedies:

      • Use polar opposite reversals to get richer belief samples, even though they may increase apparent MR;

      • Negation reversals have few retrieval benefits;


Theoretical explanations of mr retrieval1
Theoretical explanations of MR: ReviewRetrieval

  • Positioning effects:

    • Dispersed PO items reduce carryover effects and can increase coverage, but the task is more taxing for Rs and internal consistency may suffer;

    • Item similarity may determine whether Rs engage in additional retrieval when items are grouped together;

    • Remedies:

      • The use of dispersed antonyms should encourage the generation of distinct belief samples;

      • Avoid very similar (negated) statements when items are grouped;


Theoretical explanations of mr judgment
Theoretical explanations of MR: ReviewJudgment

  • Item verification difficulty (Carpenter and Just 1975; Swain et al. 2008):

    • MR is a function of the complexity of verifying the truth or falsity of an item relative to one’s true beliefs, which depends on whether the item is stated as an affirmation or negation [NMR];

    • Remedies:

      • Negations are problematic because they increase the likelihood of making mistakes (remember there are many types of negations);

      • Negated polar opposites are most error-prone;

      • Mix of regular and PO reversals should be best;


Item verification difficulty
Item verification difficulty Review

MR

Negation

Affirmation

Truth value

False

True


Theoretical explanations of mr response
Theoretical explanations of MR: ReviewResponse

  • Acquiescence: Rs initially accept a statement and subsequently re-consider it based on extant evidence; the first stage is automatic, the second requires effort (Knowles and Condon 1999) [RMR];

  • Remedies:

    • Although response styles are largely individual difference variables, situational factors may be under the control of the researcher (e.g., reduce the cognitive load for Rs);

    • Problems with online surveys;


Theoretical explanations of mr response cont d
Theoretical explanations of MR: ReviewResponse (cont’d)

  • Asymmetric scale interpretation: the midpoint of the rating scale may not be the boundary between agreement and disagreement for Rs (esp. if the response categories are not labeled; cf. Gannon and Ostrom 1996) [RMR];

  • Remedies:

    • Use fully labeled 5- or 7-point response scales;


Consequences of mr
Consequences of MR Review

  • Effect on scale reliability: [based on 4 studies]

    RG . .68

    nRG .60

    PO .57

    nPO .49

  • Effect on correlations between items measuring the same construct:

    • when content saturation is high, social desirability is low, and endorsement frequencies are moderate, the correlation between the four types of items can be quite high, although the findings showed evidence of POMR (Jackson and Lay 1968);

    • Ahlawat (1985) showed that the correlations between nPO and other items were particularly low;


Consequences of mr cont d
Consequences of MR (cont’d) Review

  • Effect on factor structure: [based on three studies]

    • The factor structure can emerge clearly when the items are high in content saturation and free of social desirability, but RMR and POMR are apparent;

    • In Schriesheim and Eisenbach (1995), RG items performed best and nPO items did very poorly;

  • Effect on correlations with other scales:

    • Holden et al. (9185) showed that item negation was more detrimental to criterion-related validity than item reversal;


Example item with 4 negations
Example item with 4 negations Review

Top management in my company has let it be known in no uncertain terms that unethical behaviors will not be tolerated.