1 / 34

Rounding Behavior of Respondents in Household Surveys

Rounding Behavior of Respondents in Household Surveys. Dr. des. Oliver Serfling University of Basel Presentation November 11, 2005 Swiss Statistical Meeting, Zürich. Agenda. Types of Survey Measurement Errors The Rounding Phenomenon Theoretical Issues & Literature Research Goals

Download Presentation

Rounding Behavior of Respondents in Household Surveys

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Rounding Behavior of Respondents in Household Surveys Dr. des. Oliver Serfling University of Basel Presentation November 11, 2005 Swiss Statistical Meeting, Zürich

  2. Agenda • Types of Survey Measurement Errors • The Rounding Phenomenon • Theoretical Issues & Literature • Research Goals • Literature on rounding behavior • Our Data: SHP • Empirical Strategy • Rounding Patterns • Conclusion Survey Response Rounding Swiss Statistical Meeting, Zürich

  3. Introduction & Motivation

  4. Types of Survey Measurement Errors Generally, measurement error occur if the reported value (Z) is not identical with the „true“ value (X): INR Item Nonresponse True value X is not reported, Z=? MME: Measurement Error Continuous X is reported with error as continous Z: Z=X+ Continuous X is reported as a discrete interval with midpoint Z where X lies in  Rounding MRE: Misreporting Error MCE: Misclassification error Discrete X is reported as wrong but discrete Z Swiss Statistical Meeting, Zürich

  5. The Rounding Phenomenon Rounding as a data coarsening: • Loss of information and data quality • Small changes in the variable become unobservable •  Problem for sensitivtiy analysis • Variance is upward biased Rounding as a response phenomenon: • Rounding may indicate motivation of respondent. Therefore, it may be a precursor of item or unit nonresponse • Rounding may be a strategy of the respondent to avoid/reduce disclosure of privacy Swiss Statistical Meeting, Zürich

  6. Literature: Rounding as coarsening • Sheppard (1898): • Examines grouping effects on normal distribution • Effect on mean is negligible • Variance is upward biased by 1/12w with w=rounding interval • Sheppards correction: calculate unbiased estimator of variance • Eisenhart (1947): • analyzes the effects of rounding with different sample sizes • Tricker (1984): • analyzes rounding on non-symmetrical dist.: gamma, log-normal • Rounding error in mean and variance is positively related to skewness of distribution and rounding degree Swiss Statistical Meeting, Zürich

  7. Three types of rounding • Presented literature deals only with same rounding behavior on every observed value • ... but in survey interviews every respondent may have its own degree of rounding, which can be: • at random or systematic • Under the assumption that respondents round correctly: (A1) • And the rounding error is uniformly distributed in the rounding interval: (A2) e ~ U[-w/2 ; w/2] • 3 types of rounded data can be distinguished: (R1) every value is rounded to same degree of rounding (w): Z = X + e with e ~ U[-w/2 ; w/2] (R2) degree of rounding (w) differs over individuals (i): Z = X + e with e ~ U[-wi/2 ; wi/2] (R3) degree of rounding (w) is a function of X: Z = X + e with e ~ U[-w(X)/2 ; w(X)/2] Swiss Statistical Meeting, Zürich

  8. R1 effects on distribution Simulated right-skewed distribution of „money“ amounts Swiss Statistical Meeting, Zürich

  9. R1 effects on distribution Simulated distribution of „money“ amounts rounded to 10s Swiss Statistical Meeting, Zürich

  10. R1 effects on distribution Simulated distribution of „money“ amounts rounded to 100s Swiss Statistical Meeting, Zürich

  11. R1 effects on distribution Simulated distribution of „money“ amounts rounded to 1000s Swiss Statistical Meeting, Zürich

  12. R2 effects on distribution Simulated distribution, individual rounding intensity at random Swiss Statistical Meeting, Zürich

  13. R3 effects on distribution Simulated distribution, rounding intensity dependent on absolute value Swiss Statistical Meeting, Zürich

  14. R1-R3 effects on moments Deviance (%) of rounded moments from their population counterpart Swiss Statistical Meeting, Zürich

  15. Research goals • Q1.) Find an appropriate rounding intensity measure • Q2.) Occurrence of rounding and correlation of rounding with similar respondent behavior • Q3.) Is there heterogeneity in degree of rounding, and how can it be explained? • Characteristics of respondent (Respondent Effects) • Person of the interviewer (Interviewer Effects) • Interview type and interview situation (Situation Effects) • Q4.) Is the degree of rounding driven by the value of concerned variable? • Q5.) Is there a panel duration effect? Swiss Statistical Meeting, Zürich

  16. Results from literature Rounding as respondent behavior

  17. Literature: Rounding as resp. behav. Schweitzer, Severance-Lossin (1996): • 71% of all reported earnings in CPS (Current Population Survey) March 1994 are multiples of $1,000 • Rounding behavior is highly systematic and correlated with respondents‘ earnings level • Systematic nature substantially affects some common used measures on earnings data: • Inequaltity summary measures (Gini-coefficient) • Earnings quantiles • Kernel density estimates • In particular, statistics are sometimes altered at levels of annual change and/or standard errors. Swiss Statistical Meeting, Zürich

  18. Literature: Rounding as resp. behav. Schräpler (1999): • Data: Gross income question of waves 1-12 of GSOEP • Roundings to 100, 500, 1000 in 67-77% of income statements • Method: Multinomial Logit estimation • categories of dependent var: exact, 10, 100, 500/1000 • Results: • Sex: Men have higher rounding propensity (5-7% higher probability of choosing 500/1000; Female interviewers provoque extreme rounding intensities (exactness and 500/1000 rounding). Male I‘s provoque middle rounding intensity. • Age of respondent and precision of statement seem to be correlated • Interview duration: positively correlated with presicion – it takes time to provide exact values • Interview mode: in self administered quest. low rounding, higher in face-to-face interviews • Experience: of respondents with interview provoques rounding • Income: low roundings in first quartile, high in fourth quartile Swiss Statistical Meeting, Zürich

  19. Literature: Rounding as resp. behav. Hanisch (2003): • Data: Finish sample of ECHP • Roundings after 1 or 2 significant digits: • 80% of gross wage statement • 95% of net disposable income question • Method: ordered probit on number of significant digits • Results: • Sex: males provide higher precision (scandinavian artifact) • Foreigners have lower roundings • Interview mode: CAPI leads to highest precision, longer interview duration produced more precision • Job effects: some professions are more precise than others • Panel participation does not have a monotone effect on rounding behavior. Swiss Statistical Meeting, Zürich

  20. Literature: Rounding as resp. behav. Kroh (2004): • analyses interview effects on rounding with self-reported body weight • Data: body weight of GSOEP 2002 • Method: Binary Probit on the event of rounded weight statement • Results: • Sex: Women provide rounded weights more often • Lower educated interviewees and singles provide rounded weights more frequently • Overweighted people tend to stronger roundings! Swiss Statistical Meeting, Zürich

  21. Our Data The Swiss Household Panel

  22. The Swiss Houeshold Panel (SHP) • SHP is an annually collected comprehensive survey • Comprises information on: • housing, living standard, income and ist components • socio-demographics, education, employment, • politics, values, and leisure. • Three separate questionnaires: • grid • personal • household • Personal questionnaire has to be answered by every household-member who reached the age of 14 • SHP is completely surveyed by CATI (Computer Assisted Telephone Interviews) • Sample size: 7,799 persons (1999) to 5,220 (2003), (refresh: 2004) Swiss Statistical Meeting, Zürich

  23. SHP Interviewer Survey • Additionally, in second wave (2000): survey of the interviewers with 24 questions on: • Socio-demographics • Interviewer experience and occupation • Opinions towards the survey • From 53 interviewers worked for SHP in 2000: • 45 participated • 41 filled in questionnaire completely • No information on interviewers in 1999, and 2001-2003 • Therefore, missing interviewer information on • 1,211 out of 7,799 cases in 1999 • approx. 700 cases in 2001, 2002 and 2003 Swiss Statistical Meeting, Zürich

  24. Own analysis

  25. Research goals revisited • Q1.) Find an appropriate rounding intensity measure • Q2.) Occurrence of rounding and correlation of rounding with similar respondent behavior • Q3.) Is there heterogeneity in degree of rounding, and how can it be explained? • Characteristics of respondent (Respondent Effects) • Person of the interviewer (Interviewer Effects) • Interview type and interview situation (Situation Effects) • Q4.) Is the degree of rounding driven by the value of concerned variable? • Q5.) Is there a panel duration effect? Swiss Statistical Meeting, Zürich

  26. Rounding Decision Model • Hypothesis: • The respondent is free to decide about his rounding intensity (RI) • … which is determined by the costs and benefits of precision: • i.e. cognitive burden, disclosure of privacy • The respondent chooses the RI which maximizes his utility: • If the cost and benefit components are attributed to the characteristics of the respondent, his interviewer and the interactions thereof, the latent rounding intensity (RI*) is: • With: αt =baseline cost-surplus in answering the question at time t, Rit are the characteristics of the respondent i, Ij are the characteristics of the interviewer j, (R*I) are the interaction of both and εit is white noise Swiss Statistical Meeting, Zürich

  27. Rounding measures Which measure reflects the latent rounding intensity? • NRD: Number of rounded digits (discrete absolute measure) • NSD: Number of significant digits (discrete absolute measure) • RQ: Rounding–Quotient = rounding digit / number of digits (discrete relative measure) • RSM: Rounding strain measure = NRD-(NSD-1) • Relative rounding error (%)(continous relative measure) Swiss Statistical Meeting, Zürich

  28. Empirical strategy • Regression of rounding measure on possible determinants: • Respondent characteristics: sex, age, education, employment status, satisfaction, health status, language, experience, nationality • Interviewer characteristics and interview experience • Interviewer-Respondent interactions • Interview situation effects: panel duration • The value of rounded variable, log amount-splines, higher polynomials of variables value • Using: • Ordered Probit modelwith a set of fully interacted covariates (RHS Var * NoD-dummies) • Dependent variable: • Number of Rounded Digits for the first income statement in the SHP questionnaire Swiss Statistical Meeting, Zürich

  29. Correlation Rounding <-> Nonresponse • large autocorrelations of rounding measures • small positive correlation of rounding with Item-Nonresponse Swiss Statistical Meeting, Zürich

  30. Respondent Effects … on Rounding Intensity (NRD): Swiss Statistical Meeting, Zürich

  31. Interviewer Effects Weak but significant effects, since SHP is conducted via CATI (telephone interviews) No significant Interviewer-Respondent Interaction / Social Distance effects! Swiss Statistical Meeting, Zürich

  32. NoD or Income Effect? • Model is augmented with log-income splines for 2,3,5, and 6 digits (4 digits as reference) • (robustness check: estimation of 5th order income polynomial) • We find different slopes of the income effect by NoD • with a negative effect for 6-digit incomes • no log-linear income effect or • additional NoD-Effect Swiss Statistical Meeting, Zürich

  33. Conclusion • Rounding in income data of the SHP is a rule, rather than an exception • Rounding intensity differs over respondents • There are robust patterns of influences on rounding behavior by respondents characteristics, interviewers characteristics, but non for interviewer-respondents interactions • Rounding intensity is also driven by the amount of considered variable, but its magnitude seems to be relatively decreasing Swiss Statistical Meeting, Zürich

  34. The End Thank you for your attention ! Paper will soon be available at: http://www.wwz.unibas.ch/stat/team/serfling Swiss Statistical Meeting, Zürich

More Related