Rounding Behavior of Respondents in Household Surveys

1. Rounding Behavior of Respondents in Household Surveys Dr. des. Oliver Serfling University of Basel Presentation November 11, 2005 Swiss Statistical Meeting, Zürich

2. Agenda • Types of Survey Measurement Errors • The Rounding Phenomenon • Theoretical Issues & Literature • Research Goals • Literature on rounding behavior • Our Data: SHP • Empirical Strategy • Rounding Patterns • Conclusion Survey Response Rounding Swiss Statistical Meeting, Zürich

3. Introduction & Motivation

4. Types of Survey Measurement Errors Generally, measurement error occur if the reported value (Z) is not identical with the „true“ value (X): INR Item Nonresponse True value X is not reported, Z=? MME: Measurement Error Continuous X is reported with error as continous Z: Z=X+ Continuous X is reported as a discrete interval with midpoint Z where X lies in  Rounding MRE: Misreporting Error MCE: Misclassification error Discrete X is reported as wrong but discrete Z Swiss Statistical Meeting, Zürich

5. The Rounding Phenomenon Rounding as a data coarsening: • Loss of information and data quality • Small changes in the variable become unobservable •  Problem for sensitivtiy analysis • Variance is upward biased Rounding as a response phenomenon: • Rounding may indicate motivation of respondent. Therefore, it may be a precursor of item or unit nonresponse • Rounding may be a strategy of the respondent to avoid/reduce disclosure of privacy Swiss Statistical Meeting, Zürich

6. Literature: Rounding as coarsening • Sheppard (1898): • Examines grouping effects on normal distribution • Effect on mean is negligible • Variance is upward biased by 1/12w with w=rounding interval • Sheppards correction: calculate unbiased estimator of variance • Eisenhart (1947): • analyzes the effects of rounding with different sample sizes • Tricker (1984): • analyzes rounding on non-symmetrical dist.: gamma, log-normal • Rounding error in mean and variance is positively related to skewness of distribution and rounding degree Swiss Statistical Meeting, Zürich

7. Three types of rounding • Presented literature deals only with same rounding behavior on every observed value • ... but in survey interviews every respondent may have its own degree of rounding, which can be: • at random or systematic • Under the assumption that respondents round correctly: (A1) • And the rounding error is uniformly distributed in the rounding interval: (A2) e ~ U[-w/2 ; w/2] • 3 types of rounded data can be distinguished: (R1) every value is rounded to same degree of rounding (w): Z = X + e with e ~ U[-w/2 ; w/2] (R2) degree of rounding (w) differs over individuals (i): Z = X + e with e ~ U[-wi/2 ; wi/2] (R3) degree of rounding (w) is a function of X: Z = X + e with e ~ U[-w(X)/2 ; w(X)/2] Swiss Statistical Meeting, Zürich

8. R1 effects on distribution Simulated right-skewed distribution of „money“ amounts Swiss Statistical Meeting, Zürich

9. R1 effects on distribution Simulated distribution of „money“ amounts rounded to 10s Swiss Statistical Meeting, Zürich

10. R1 effects on distribution Simulated distribution of „money“ amounts rounded to 100s Swiss Statistical Meeting, Zürich

11. R1 effects on distribution Simulated distribution of „money“ amounts rounded to 1000s Swiss Statistical Meeting, Zürich

12. R2 effects on distribution Simulated distribution, individual rounding intensity at random Swiss Statistical Meeting, Zürich

13. R3 effects on distribution Simulated distribution, rounding intensity dependent on absolute value Swiss Statistical Meeting, Zürich

14. R1-R3 effects on moments Deviance (%) of rounded moments from their population counterpart Swiss Statistical Meeting, Zürich

15. Research goals • Q1.) Find an appropriate rounding intensity measure • Q2.) Occurrence of rounding and correlation of rounding with similar respondent behavior • Q3.) Is there heterogeneity in degree of rounding, and how can it be explained? • Characteristics of respondent (Respondent Effects) • Person of the interviewer (Interviewer Effects) • Interview type and interview situation (Situation Effects) • Q4.) Is the degree of rounding driven by the value of concerned variable? • Q5.) Is there a panel duration effect? Swiss Statistical Meeting, Zürich

16. Results from literature Rounding as respondent behavior

17. Literature: Rounding as resp. behav. Schweitzer, Severance-Lossin (1996): • 71% of all reported earnings in CPS (Current Population Survey) March 1994 are multiples of $1,000 • Rounding behavior is highly systematic and correlated with respondents‘ earnings level • Systematic nature substantially affects some common used measures on earnings data: • Inequaltity summary measures (Gini-coefficient) • Earnings quantiles • Kernel density estimates • In particular, statistics are sometimes altered at levels of annual change and/or standard errors. Swiss Statistical Meeting, Zürich

18. Literature: Rounding as resp. behav. Schräpler (1999): • Data: Gross income question of waves 1-12 of GSOEP • Roundings to 100, 500, 1000 in 67-77% of income statements • Method: Multinomial Logit estimation • categories of dependent var: exact, 10, 100, 500/1000 • Results: • Sex: Men have higher rounding propensity (5-7% higher probability of choosing 500/1000; Female interviewers provoque extreme rounding intensities (exactness and 500/1000 rounding). Male I‘s provoque middle rounding intensity. • Age of respondent and precision of statement seem to be correlated • Interview duration: positively correlated with presicion – it takes time to provide exact values • Interview mode: in self administered quest. low rounding, higher in face-to-face interviews • Experience: of respondents with interview provoques rounding • Income: low roundings in first quartile, high in fourth quartile Swiss Statistical Meeting, Zürich

19. Literature: Rounding as resp. behav. Hanisch (2003): • Data: Finish sample of ECHP • Roundings after 1 or 2 significant digits: • 80% of gross wage statement • 95% of net disposable income question • Method: ordered probit on number of significant digits • Results: • Sex: males provide higher precision (scandinavian artifact) • Foreigners have lower roundings • Interview mode: CAPI leads to highest precision, longer interview duration produced more precision • Job effects: some professions are more precise than others • Panel participation does not have a monotone effect on rounding behavior. Swiss Statistical Meeting, Zürich

20. Literature: Rounding as resp. behav. Kroh (2004): • analyses interview effects on rounding with self-reported body weight • Data: body weight of GSOEP 2002 • Method: Binary Probit on the event of rounded weight statement • Results: • Sex: Women provide rounded weights more often • Lower educated interviewees and singles provide rounded weights more frequently • Overweighted people tend to stronger roundings! Swiss Statistical Meeting, Zürich

21. Our Data The Swiss Household Panel

22. The Swiss Houeshold Panel (SHP) • SHP is an annually collected comprehensive survey • Comprises information on: • housing, living standard, income and ist components • socio-demographics, education, employment, • politics, values, and leisure. • Three separate questionnaires: • grid • personal • household • Personal questionnaire has to be answered by every household-member who reached the age of 14 • SHP is completely surveyed by CATI (Computer Assisted Telephone Interviews) • Sample size: 7,799 persons (1999) to 5,220 (2003), (refresh: 2004) Swiss Statistical Meeting, Zürich

23. SHP Interviewer Survey • Additionally, in second wave (2000): survey of the interviewers with 24 questions on: • Socio-demographics • Interviewer experience and occupation • Opinions towards the survey • From 53 interviewers worked for SHP in 2000: • 45 participated • 41 filled in questionnaire completely • No information on interviewers in 1999, and 2001-2003 • Therefore, missing interviewer information on • 1,211 out of 7,799 cases in 1999 • approx. 700 cases in 2001, 2002 and 2003 Swiss Statistical Meeting, Zürich

24. Own analysis

25. Research goals revisited • Q1.) Find an appropriate rounding intensity measure • Q2.) Occurrence of rounding and correlation of rounding with similar respondent behavior • Q3.) Is there heterogeneity in degree of rounding, and how can it be explained? • Characteristics of respondent (Respondent Effects) • Person of the interviewer (Interviewer Effects) • Interview type and interview situation (Situation Effects) • Q4.) Is the degree of rounding driven by the value of concerned variable? • Q5.) Is there a panel duration effect? Swiss Statistical Meeting, Zürich

26. Rounding Decision Model • Hypothesis: • The respondent is free to decide about his rounding intensity (RI) • … which is determined by the costs and benefits of precision: • i.e. cognitive burden, disclosure of privacy • The respondent chooses the RI which maximizes his utility: • If the cost and benefit components are attributed to the characteristics of the respondent, his interviewer and the interactions thereof, the latent rounding intensity (RI*) is: • With: αt =baseline cost-surplus in answering the question at time t, Rit are the characteristics of the respondent i, Ij are the characteristics of the interviewer j, (R*I) are the interaction of both and εit is white noise Swiss Statistical Meeting, Zürich

27. Rounding measures Which measure reflects the latent rounding intensity? • NRD: Number of rounded digits (discrete absolute measure) • NSD: Number of significant digits (discrete absolute measure) • RQ: Rounding–Quotient = rounding digit / number of digits (discrete relative measure) • RSM: Rounding strain measure = NRD-(NSD-1) • Relative rounding error (%)(continous relative measure) Swiss Statistical Meeting, Zürich

28. Empirical strategy • Regression of rounding measure on possible determinants: • Respondent characteristics: sex, age, education, employment status, satisfaction, health status, language, experience, nationality • Interviewer characteristics and interview experience • Interviewer-Respondent interactions • Interview situation effects: panel duration • The value of rounded variable, log amount-splines, higher polynomials of variables value • Using: • Ordered Probit modelwith a set of fully interacted covariates (RHS Var * NoD-dummies) • Dependent variable: • Number of Rounded Digits for the first income statement in the SHP questionnaire Swiss Statistical Meeting, Zürich

29. Correlation Rounding <-> Nonresponse • large autocorrelations of rounding measures • small positive correlation of rounding with Item-Nonresponse Swiss Statistical Meeting, Zürich

30. Respondent Effects … on Rounding Intensity (NRD): Swiss Statistical Meeting, Zürich

31. Interviewer Effects Weak but significant effects, since SHP is conducted via CATI (telephone interviews) No significant Interviewer-Respondent Interaction / Social Distance effects! Swiss Statistical Meeting, Zürich

32. NoD or Income Effect? • Model is augmented with log-income splines for 2,3,5, and 6 digits (4 digits as reference) • (robustness check: estimation of 5th order income polynomial) • We find different slopes of the income effect by NoD • with a negative effect for 6-digit incomes • no log-linear income effect or • additional NoD-Effect Swiss Statistical Meeting, Zürich

33. Conclusion • Rounding in income data of the SHP is a rule, rather than an exception • Rounding intensity differs over respondents • There are robust patterns of influences on rounding behavior by respondents characteristics, interviewers characteristics, but non for interviewer-respondents interactions • Rounding intensity is also driven by the amount of considered variable, but its magnitude seems to be relatively decreasing Swiss Statistical Meeting, Zürich

34. The End Thank you for your attention ! Paper will soon be available at: http://www.wwz.unibas.ch/stat/team/serfling Swiss Statistical Meeting, Zürich