Time-based Features for Improved Eating Activity Detection: MS Defense Study

1. Department of Computer and Electrical Engineering A Study of Time-based Features and Regularity of Manipulation to Improve the Detection of Eating Activity Periods During Free-Living MS Defense Exam Jose Luis Reyes Dr. Adam Hoover (chair) Dr. Eric Muth Dr. Richard Groff April 24, 2014

2. Outline • Motivation and Background • Design and Methods • Results • Conclusion

3. Obesity • Common • 34% of U.S. population are obese [Centers for Disease Control and Prevention] • Serious • 5th leading risk for global deaths [WHO, 2014] • Heart disease, stroke, type 2 diabetes, and certain types of cancer [Centers for Disease Control and Prevention] • Costly • In 2008, annual medical cost was $147 billion in the U.S. [Centers for Disease Control and Prevention] • In 2008, medical cost was $1,429 higher than of those of normal weight. [Centers for Disease Control and Prevention]

4. Obesity treatments • Dietary changes • Exercise and activity • Behavior changes • Weight-loss medication • Weight-loss surgery • Limit energy intake (EI)* • Balancing EI and EE (energy expenditure)

5. Monitoring EI • Most widely used tools • Food diary • 24-hour recall • Food frequency questionnaire • Technology-based tools • Camera [Martin et al., 2009] • Wearable sensors [Amft et al., 2008]

6. Bite Counter • Watch-like device • Wrist motion tracking • Accelerometer and gyroscope

7. Previous work • Goal: Detection of eating activity periods • Based on accelerometer (AccX, AccY, AccZ) and gyroscope (Yaw, Pitch, Roll) readings • Data segmentation • Classification of eating activity (EA) and non-eating activity (non-EA) periods based on features • Overall accuracy obtained was 81%

8. Novelty • Previous work considered only sensor-based features • We consider the time component • Time since last eating activity • Cumulative eating time • Periodicity of manipulation over time • Regularity of manipulation

9. Design and methods • Overview of algorithm • Data collection • New features • Regularity of manipulation • Time since last EA • Cumulative eating time • Evaluation metrics

10. Overview of algorithm (Dong et al., 2013) • Data smoothing • - Gaussian kernel

11. Overview of algorithm • Sum of acceleration,

12. Overview of algorithm • Data segmentation • Peak detection • Sum of acceleration • Hysteresis threshold

13. Overview of algorithm • Features • Manipulation • Linear acceleration • Wrist roll motion • Regularity of roll

14. Overview of algorithm • Naive Bayes Classifier • Assign most probable class, ci in C • Given features f1,f2, …, fN • Feature probability

15. Data collection • Collected using iPhone 4 • Programmable , large amount of memory, accelerometer and gyroscope • Recorded at 15Hz • 2 sets of data • Set 1: 20 recordings • Set 2: 23 recordings • A total of 449 hours of data • Data training • 5 minute non-EA segments • Full segments for EA

16. Current work • Motivation: improve previous accuracy of 81% • Introduction of 3 new features: • Regularity of manipulation • Time since last EA • Cumulative eating time

17. Features • Feature 1, regularity of manipulation • Regularity of peaks around 4000-5000 (deg/s)/G • Peaks every 10 – 30 seconds? EA manipulation segment Non-EA manipulation segment

18. Regularity of manipulation • Smooth manipulation data (N = 225, R = 37.6) • Compute FFT • Compute: • Units: (deg/s3)/G

19. Regularity of manipulation 29>> • Calculate for each segment in data • Distribution statistics can be used for Bayes classifier Distributions (set 1)

20. Regularity of manipulation Distributions (set 2) 34>>

21. Features • Feature 2, time since last eating activity • Time component • After a person eats, very unlikely to eat again immediately • Probability starts increasing as time passes

22. Time since last EA • Let tlast = end time of last segments classified as EA • Let t = middle of time of unknown segment currently being classified • Then,

23. Time since last EA • Bayes classifier requires probability distributions for both EA and non-EA • It is possible to calculate time between meals • Nonsensical for opposite class • Time since last non-EA? • 1 – p(f|EA)

24. Time since last EA • Compute cumulative distribution function (CDF) of time since last EA. • p(f|EA) = CDF, p(f|nonEA) = 1 - CDF CDF for time since last EA (set 2)

25. Features • Feature 3, cumulative eating time • Time component • People spend a certain amount of time eating and drinking in a day(Around 1.1 hrs. according to Dept. of Labor Statistics )

26. Cumulative eating time • At time t, cumulative eating time: • Distribution of times involving non events are nonsensical • Compute CDF for each recording and average in each data set

27. Cumulative eating time CDF for cumulative eating time (set 2)

28. Cumulative eating time • p(f|EA) = • σ2cdf, μcdf from average CDF • p(f|nonEA) = 1 – p(f|EA)

29. Evaluation metrics • Overall accuracy • EA accuracy • Non-EA accuracy

30. Results • Previous work • Statistics • Accuracy

31. Results • Regularity of manipulation • Statistics • Accuracy

32. Regularity of manipulation (Results) • Standard deviation relatively large for EA distribution (<<18) • Set 1’s EA distribution non Gaussian • FFT not completely discriminating between EAs and non-EAs

33. Regularity of manipulation (Results) Smoothed manipulation segment from EA distribution (right tail) Smoothed manipulation segment from non-EA distribution (left tail)

34. Regularity of manipulation (Results) Smoothed manipulation segment from EA distribution (middle) Smoothed manipulation segment from non-EA distribution (middle) <<20

35. Regularity of manipulation (Results) Original data for segment in middle of EA distribution Original data for segment in middle of non-EA distribution

36. Results • Time since last EA • Statistics • Accuracy Set 1 Set 2

37. Time since last EA (Results) Original 4 features Original 4 features + time since last EA

38. Time since last EA (Results) • FPs are strong inhibitors for immediately subsequent data Original Including time since last EA

39. Results • Cumulative eating time • Statistics • Accuracy Set 1 Set 2

40. Cumulative eating time (Results) Original 4 features Original 4 features + cumulative eating time

41. Cumulative eating time (Results) • FPs are strong inhibitors for immediately subsequent data Original Including cumulative eating time

42. Conclusion • FFT not discriminating between EAs and non-EAs completely • Time-based features act as clocks • Future work • Explore regularity of manipulation using non-sinusoidal transform • Explore off-line analysis using time-based features so the optimal daily solution can be found (HMMs)

43. Thank you! Questions?