Our Menu For Today

1. Permutation Entropy of Activity Data Acquired From Lifestyle Monitoring DevicesRachel HeathSykTek,Valentine, NSW 2280Australia

2. Our Menu For Today • Entrée: How do we know whether our data are nonlinear? Does greater complexity accompany better health? • Main course: What is permutation entropy? Does increased permutation entropy imply better adjustment? Lifestyle devices monitor our activity, sleep and health. What can permutation entropy tell us about our activity? • Conclusion: Lifestyle monitoring devices can detect changes in behavior using complexity measures such as Permutation Entropy

3. What is Complexity? • Complexity is related to the entropy, or disorder, of a sequence of observations. • Mostly, nonlinear systems exhibit more complex behaviour than do linear systems. • The more noisy a system, the larger its measured complexity. • Complex nonlinear systems are difficult to predict unless they are completely deterministic and the starting values are known precisely. • Just because a system exhibits complex behaviour, it need not be nonlinear, e.g., filtered noise, symbolic systems.

4. How Can We Test for Nonlinearity in Data? • We compute a complexity index for the experimental series and also for surrogate series that have the same linear properties as the experimental series (mean, variance, autocorrelation). • Nonlinear determinism is suggested when the complexity index for the experimental series is reliably different from that computed for the surrogate series.

5. Permutation Entropy • A complexity measure computed using ordinal features of the data. • A moving data window can be used to accommodate temporal variations in the data. • Permutation Entropy is robust to data transformations that preserve order. • Permutation Entropy is less affected by measurement noise than some other complexity measures. • Permutation Entropy ranges between 0 and 1, the larger the value the greater the complexity. • Permutation Entropy can be more sensitive to change than other complexity measures such as Sample Entropy.

6. Permutation Entropy Algorithm Keller, K., & Sinn, M. (2005). Ordinal analysis of time series. Physica A, 356, 114-120. • Select permutation length d (usually 5  7) • Set the window length to d! and time series lag = L (e.g. first zero-crossing delay from time series ACF) • Within each successive data window, compute the frequency distribution for the observed permutations of successive d observations • Permutation Entropy for window w is the normalised Shannon information measure on this permutation probability distribution for all permutations with nonzero frequency. Sum over all permutations of d objects

7. Example calculation with d = 3, L = 1 Daily Max Temperature in Milwaukee, 29 June to 10 July 2014 order freq p -plog(p) 81, 83, 76, 77, 63, 64, 74, 80, 84, 83, 77, 75 (2,3,1) 81, 83, 76, 77, 63, 64, 74, 80, 84, 83, 77, 75 (3,1,2) 81, 83, 76, 77, 63, 64, 74, 80, 84, 83, 77, 75 (2,3,1) 2 .2 0.140 81, 83, 76, 77, 63, 64, 74, 80, 84, 83, 77, 75 (3,1,2) 2 .2 0.140 81, 83, 76, 77, 63, 64, 74, 80, 84, 83, 77, 75 (1,2,3) 81, 83, 76, 77, 63, 64, 74, 80, 84, 83, 77, 75 (1,2,3) 81, 83, 76, 77, 63, 64, 74, 80, 84, 83, 77, 75 (1,2,3) 3 .3 0.156 81, 83, 76, 77, 63, 64, 74, 80, 84, 83, 77, 75(1,3,2) 1 .1 0.100 81, 83, 76, 77, 63, 64, 74, 80, 84, 83, 77, 75(3,2,1) 81, 83, 76, 77, 63, 64, 74, 80, 84, 83, 77, 75 (3,2,1) 2 .2 0.140 Sum(-plog(p)) = 0.676 Permutation Entropy = Sum(-plog(p)) /log(3!) = 0.676/0.778 = 0.87 Somewhat complex for overseas visitors, perhaps!

8. Detecting Nonlinearity in Activity Data Using Permutation Entropy • Compute at least 30 surrogate time series obtained by shuffling the original series but retaining linear information, mean, variance and autocorrelation function. • Compute Permutation Entropy for each surrogate series using the same parameters as used for the original series. • For each data window, compare the observed Permutation Entropy with the distribution of Permutation Entropy computed using the surrogate series. • Claim statistical significance of nonlinearity if the observed Permutation Entropy exceeds the confidence bounds of the surrogate distribution.

9. Detecting Change in Complexity Using Permutation Entropy • The Permutation Entropy value can be accumulated over time to detect a significant change in complexity. • A significant decrease in complexity may suggest deterioration in a medical condition, whereas a significant increase in complexity hints at improvement. • Many medical applications of Permutation Entropy are based on the idea that disease can be detected by a decrease in the estimated complexity of physiological data. • Perhaps good health can be best considered in terms of an optimal value of physiological and/or behavioural complexity.

10. Permutation Entropy Predicts the Onset of an Epileptic Seizure Li, Ouyang & Richards, Epilepsy Research, 2007 Permutation Entropy anticipates seizure earlier than Sample Entropy

11. Permutation Entropy Detects Relapse in a Patient with Mood Disorder Using Activity Data(Using unpublished data kindly provided by Prof G. Murray, 2011) Hospitalisation occurred about day 105 but Permutation Entropy detected a complexity decrease about two weeks earlier possibly eliminating the need for hospitalisation.

12. Lifestyle Devices that Measure Activity and Sleep • Inexpensive, wearable, provide basic data for health monitoring, webpage summaries available for users • Can synchronise data with desktop computers and mobile devices • Common devices include BodyMedia FIT (upper arm) and FitBit Flex (wrist) • A clinical version of BodyMedia FIT provides detailed data analyses • Possible cloud group monitoring by doctors and coaches

13. How I Recorded Activity Time Series from www.bodymedia.com • Access activity graph using a mobile device, e.g. iPhone • Take an image on the iPhone and send it via email • Use ImageJ software to convert activity image to numerical time series The clinical version of BodyMedia does this automatically for many more $$$$

14. Permutation Entropy Analysis of Activity Data BodyMedia Fit measured activity over three sessions for two subjects, 34 days for Subject 1 and 29 days for Subject 2 • Sum 6 successive activity values  total activity every 10 mins • Compute first differences to remove trend. • Set permutation length, d = 5 and lag L = 1 • For nonlinearity check, use TISEAN surrogates routine to compute 30 phase-randomised surrogate series for each data set • Compute standardised scores for each window w using • Check for significant change in Permutation Entropy if z(w) >2 or z(w) < -2

15. Intermittent activity series, complexity reduction at end of day 7

16. Intermittent activity data, no change in complexity but declining trend

17. Intermittent activity data, several decreases in complexity

18. Noisy activity data, decrease in complexity around day 4

19. Slightly intermittent activity data, no change in complexity

20. Noisy activity data, no change in complexity

21. Conclusions • Permutation Entropy provides an easily computed complexity index for nonlinear systems • Permutation Entropy handles nonstationarity quite well and can be used to monitor changes in complexity over time in behavioural time series • Permutation Entropy provides a standardised index to detect change and compare individuals • The technique can be implemented within popular lifestyle monitoring devices to enhance their value as diagnostic devices for behavioural change • Further work has used Multifractal Analysis to analyse these and other activity time series

22. Dedication I dedicate this work to the memory of Susan Heath (1950 – 2014). Susan supported my many attempts to understand her illness and to ameliorate her suffering, while maintaining her positive attitude and affection.