Background & Design EMA Component Gallup ATUS Results for Location Results for Activities

1. “Validating the American Time Use Survey (ATUS): Does anybody really remember what they were doing yesterday?” • Background & Design • EMA Component • Gallup ATUS • Results for Location • Results for Activities • Results for Feelings Alan B. Krueger Princeton University

2. American Time Use Survey • ATUS is a BLS survey conducted by the Census Bureau. • ATUS began in 2003 • The sample consists of sampled individuals age 15 and older from households that have completed the final (8th) interview of the Current Population Survey • One indiv idual per selected household in the CPS final rotation group is chosen to participate in the ATUS • This person is interviewed once about his or her time use on the previous day. • The ATUS collects data on how people spend their time; used to measure quality of life, work hours, home production, care giving, etc.

3. Study Design: Two Components • Real Time Data (EMA). Designed by Arthur Stone - 168 workers carried a PDA that asked questions about their activities in real time for 3 consecutive days (Thursday, Friday and Saturday). - Convenience sample from Syracuse, NY and Stony Brook, NY. (Tried to conduct random sample but failed.) Data collected April –August, 2008 - Paid $120 for participation - Awake day divided into 6 evenly spaced intervals, and PDA beeps randomly within intervals. Approximately 168 x 6 beeps x 3 days ≈ 3,000 moments. - Called Ecological Momentary Assessment (EMA). • Telephone Recall Survey. The same workers were given a survey that closely follows the ATUS on each day following EMA. This survey component was conducted by Gallup for Princeton and I call it P-ATUS. - 478 interviews.  Compare P-ATUS to EMA in aggregate and overlapping moments.

4. EMA: Designed by Arthur Stone w/Alan Krueger ►Training and practice take place on Wednesday ► Enter in sleep times ► Beeps for 2 minutes ► 5 minutes to respond after beep. Device turns off after then. ► Recorded time is when first tap occurs. Palm Zire 31

5. NB: Can only check one.

6. Done Done Done

8. Additional EMA Questions • Duration of activity (in intervals) ‘till beep • Feelings: happy, angry, tired, stressed, interested, pain  on 0 (not at all)-6 (very much) scale • Are you at work?  Workplace engagement • Confidence, control, competence • With supervisor? • Take medication since last prompt? • Caffeine, cigarettes, alcohol in last 30 mins?

9. Actiheart Device The Actiheart is an innovative, remarkably compact device that records physical activity and heart rate with a high level of accuracy. Levels of caloric expenditure can be determined using the information acquired by the device.Actiheart is not a peak detector, like all other heart rate detectors. It actually digitizes the ECG signal and calculates the heart rate from the true R-to-R interval. What this means is that Actiheart loses less than 1% of the data. Other detectors may lose up to 30% of the data.

10. Physical Measurements • Actiheart Device: physical activity, caloric expenditure, and heart rate in continuous time • Cortisol Measurements -- 6 times a day, at end of EMA questions • Will use: (1) to validate self reports; (2) interest in activities and health pathways

11. Princeton ATUS: Start with ATUS • Gallup modified ATUS Blaise program for Krueger already. Called PATS. Now call it P-ATUS • Collect all activities engaged in during the day with ATUS instrument • Princeton Module: Randomly select 3 awake episodes and ask about feelings after complete time diary • Interviewed on Friday, Saturday and Sunday about preceding days; on some occasions interviewed about two days previously (8.2%) or three days previously (0.6%) • Coded main and secondary activities • Used Census/BLS materials to train interviewers

12. Key Questions: • What did you do (next)? • How long did you do [ACTIVITY]? • Who was in the room/who accompanied you? • Where were you? • How did you feel at this time?

14. Sample Characteristics • ATUS (June-Aug, 2003-07) • 50% Female • Mean Age = 41 • Median Inc. $50-75k • 83% White • 12% Black • 13% Hispanic • 29% Some College • 33% College+ • 9% Education Sector • 10% Healthcare Sector EMA/P-ATUS • 83% Female • Mean Age=43 • Median Inc. $50-75k • 83% White • 9% Black • 4% Hispanic • 41% Some College • 53% College+ • 61% Education Sector • 19% Healthcare Sector

15. Time Use in P-ATUS Looks ReasonableLocation: ATUS (2003-07) vs. P-ATUS r = 0.995 Note: ATUS is April-Aug.

16. Is the Sample More Accurate at Reporting than Other Samples? • Can look at consistency of reporting on Day 1 vs. Day 3 for same subjects (or Day 1 vs. Day 2 if Day 3 unavailable). • Correlation of education 1-2 days apart is 0.81 • Correlation of log household income 1-2 days apart is 0.93  The sample does not seem unusually accurate despite higher education

17. Structure of the Data EMA P-ATUS Match Rate – Percent of moments when Activity or Location is same in EMA and P-ATUS. Can also expand window to +/- 15 mins.

18. Comparison of Location in EMA and P-ATUS 79% match within 15 mins. Sample consists of 2,396 overlapping moments from EMA and P-ATUS.

19. Location in EMA and P-ATUS During Overlapping Moments 2.7 point difference r=0.996 Sample is 2,396 moments.

20. Next Turn to Activities • Activities are trickier because people can do multiple activities at once. • If any of the activities on EMA match any of the activities in P-ATUS in overlapping moments, we call it an exact match.

21. Multi-Tasking: Number of Activities Done at a Time EMA Data None of the above ► 26% of the time people are multitasking. ► On average, 1.32 activities are done at any one moment (treating “none of the above” as one activity).

22. Much Less Reporting of Multi-Tasking in P-ATUS Data ► For 97.2% of time, people indicated one activity. ► For 2.6% of time, people indicated two activities. ► For 0.1% of time, people indicated three activities. Similar to ATUS 2006 (according to Jay Stewart) NB: Non-sleeping episodes.

23. Average Number of Simultaneous Activities EMA Data NB: Because unit is activity, this chart over samples multi-tasked moments.

24. What Goes on during Work Activities? EMA Data • 73% of time only checked work • 9% of time on phone • 6% of time eating/drinking • 5% of time socializing • 4% of time caring for someone • 1% of time class related NB: Not conditional on multi-tasking. Requires activity is work.

25. Comparison of Activities in EMA and P-ATUS 71% match within 15 mins. Sample consists of 2,360 overlapping moments from EMA and P-ATUS.

26. Subject-Level Analysis – Percent EMA-PATUS Matching Activity Within 15 Minutes Mean is 70%, Median is 73%, Q1 is 63%, Q3 is 82%. N=165 subjects

27. Activity in EMA and P-ATUS During Overlapping Moments Difference is 4.6 points r=0.990 Note: If multiple activities, used EMA activity that matched P-ATUS; otherwise selected work, eating, sleep; otherwise randomly selected.

28. Reliability of Individual-Level Work Data y1 is EMA measure of percent of time spent working, and and y2 is P-ATUS measure. (Note: P-ATUS is full awake day and EMA is sampled.) ei is measurement error i is average over 3 days, i=1,…,165 yi1= a + b yi2 + ei b = cov(y1,y2)/var(y1) Note: plim of b is attenuation bias y1= 7.42 + 0.69 y2 R-square=0.32 (2.55) (0.08)

29. NB: Drops on Saturday because work is low.

30. Activity Match Rate By Phone Interview Lag: 1 or 2/3 Days Number of Days between EMA and P-ATUS Note: Sample size is 2,131 for 1 day lag and 221 for 2/3 day lag.

31. Linear Probability Models Dependent Variable Equals 1 if Exact Match on Activity, 0 if Mismatch Female

32. To do: Turn into a 15-minute data set and regress heart beat on activity dummies, location, companions, time of day etc., and person fixed effects.

33. Stressed Pain

34. Average of Subjects’ Ratings Note: Order of emotions was randomized in P-ATUS. Sample is 165 individuals. Except for happy, all differences are significant at .005 level in paired t-test.

35. Correlations between Affect EMA and P-ATUS at Person Level, Averaging Over Overlapping Days Happy 0.75 Sad 0.81 Tired 0.79 Stressed 0.78 Interested 0.74 Pain 0.86 n=165 Note: If one adjusts the data for sampling variance in the person averages, r rises to 0.92 for happy and 0.94 for pain.

36. Conclusions/Remaining Challenges Data indicate a reasonable match rate between real-time reporting and P-ATUS – better for location than activity. Discrepancies might be due to mistakes in EMA or P-ATUS. Two day recall does as well as one day recall Bias. Work hours are overstated in P-ATUS. John Robinson found that time diaries show lower work hours than CPS weekly hours recall question. This study reinforces – diary overstates actual work time.  My interpretation: Less precise/longer recall questions about work blur work and nonwork activities. In ATUS people tend to report nonwork activities as work if they occur at the workplace. Nonresponse in EMA (skipped beeps) can be studied. ATUS has a relatively low response rate for a government survey – but EMA is even worse, and not an option. The module on feelings does a good job characterizing individuals’ days compared with real-time reporting Can validate further with physiological data.