Personalised Ambient Monitoring: aiding those with Bipolar Disorder

1. Personalised Ambient Monitoring: aiding those with Bipolar Disorder Sally Brailsford John Crowe Christopher James Evan Magill

2. ThePAM project Enabling health, independence and wellbeing for psychiatric patients through Personalised Ambient Monitoring

3. A sandpit project • Funded by the Engineering and Physical Sciences Research Council • Sandpit theme: “Bringing Care to the Patient”

4. The PAM team • Sally Brailsford, Southampton • John Crowe, Nottingham • ChristopherJames, Southampton(PI) • Evan Magill, Stirling • plus 4 PhD students • Syed Mohiuddin, Pawel Prociow, James Amor and Jesse Blum Behavioural Analysis Sensors PAM Ambient Monitoring Operational Research

5. PAM external steering group • Ms Sylvia Wyatt, Future Healthcare Network, NHS Scotland • Dr Amy Drahota, Research Fellow, University of Portsmouth • Mr Peter Jones, Community mental health nurse, Lancashire Care NHS Trust • Dr Paul Courtney, Consultant Psychiatrist, Hampshire Partnership Trust • Mr Richard Barritt, Chief Executive, Solent MIND • Mr James Stubbs, Service User Representative

6. The aims of PAM • To build a system of unobtrusive sensors, linked (through a standard mobile phone) to a remote computer system, which automatically monitors the activity patterns of people with mental health problems • To determine whether it is possible to use such a system to obtain ‘activity signatures’ in a manner which is acceptable to the patient and can provide useful information about the trajectory of their health status • And if this is so, to determine how this information can best be used to maintain health and aid independence

7. Bipolar Disorder • Severely disabling mental illness which affects functionality, relationships, employment and quality of life; affects 2% of the UK population (MHF, 2006) • Bipolar disorder is the 6th most common disabling illness worldwide (WHO, 2004) • In 2002, the estimated annual cost to the UK NHS of managing bipolar disorder was £199M, of which £70M was spent on hospital admissions (Gupta and Guest, 2002) • Many pharmacological treatments are available but these can have unpleasant side-effects and adherence is often poor, leading to hospital admission

8. How does BD alter lifestyle?

9. Managing Bipolar Disorder • Most patients want to manage their own condition, using medication only when necessary • Motivated patients of above-average intelligence, interested in self care and independence • Early warning signs or prodromes can be detected while patient is still “self-aware” and can take action (seek medical help, start medication etc) to avoid hospital admission • Paper-based “mood diaries” shown to be effective in trials

10. Problems with paper-based systems • Do not provide a sense of control over daily life • Patients complain about vigilance and energy required • Problems with accuracy, completeness and honesty of patient-reported data • Patients may forget to document important details • Comorbidity and drug response go unmeasured • No reduction in depressive relapses (Perry et al, 1999)

11. The aim of PAM • To use a system of electronic sensors to provide an automated equivalent of a mood diary, which alerts the patient to a change in activity pattern which could signal the onset of a bipolar episode • Patient would be sent an SMS alerting them to a possible change, which they could then act on (if they chose) • PAM is mainly aimed at people who live alone • Aim is to identify a baseline “activity signature” and then identify significant deviations from this

12. Device Nodes • Worn • Mobile Phone • Questionnaire • Gateway Application • GPS Transceiver • Wearable Accelerometer • Wearable Microphone • Wearable Light Sensor • Environmental • Microphone • Light Sensor • Passive Infrared Sensors • Micro-switches • Bed Sensor • Camera • Infrared Receiver For Remote Control • PC

13. Wearable sensor set • User input: • General health questionnaires • Mood self-assessment GSM location GPS module XYZ accelerometer • Wearable Node • Acceleration • General light level • Artificial light level • Ambient sound properties Internal accelerometer Bluetooth Encounters* • Bluetooth • 3G / GPRS • User input • Internal

14. Environmental sensor set • Bluetooth • WiFi • 433 MHz RF • Environmental processing unit • Processing • Storage • Backup • Upload Wide-angle Camera PIR sensors • Home appliances monitoring • Microwave • Refrigerator • Oven • Environmental Node • Monitoring of: • Remote control activity Main and cupboard doors. • General light level • Artificial light level • Ambient sound. Bed occupancy sensor

15. Example data – wearable light levels Working Bus awaiting Bus awaiting Commuting Walking Home Artificial light General light

16. Threads of research activity • The four centres collaborated across the project but we gravitated towards independent themes (as required by the four PhD students) • accelerometry & behaviour analysis • outside the home • BD modelling • rule-based sensor network

17. accelerometry & behaviour analysis accelerometry & behaviour analysis

18. accelerometry & behaviour analysis Accelerometry & Behavioural analysis • Determine what a person is doing (sleeping, eating, restlessly pacing around, etc) by feature extraction algorithms on sensor data (e.g. the “neuroscale” algorithm) • Develop an “activity signature” for that individual, describing their normal activity pattern when well • Develop a set of decision rules which determine whether an individual’s current activity is “normal” – for them – or may indicate the potential onset of a prodrome

19. walking at a lecture activity over time clustered activity accelerometry & behaviour analysis Tri-axial accelerometry

20. outside the home outside the home

21. outside the home Outside the homeExample: tracking movement & position Off-the-shelf GPS module BT enabled accelerometer logfile.txt 13 Feb 2010 13:06:41; G; 5256.0723; -112.181; 0.0; 13 Feb 2010 13:06:42; A; 0.044; -0.888; 0.484; 13 Feb 2010 13:06:44; A; 0.036; -0.892; 0.492; 13 Feb 2010 13:06:45; A; 0.036; -0.892; 0.496;

22. outside the home Positional data and pre-processing

23. outside the home Identifying meaningful locations

24. outside the home Activity data – Bluetooth • Participants on average encountered more than 1000 unique Bluetooth devices of which: • 80% were one-off encounters • 15% were “occasional” (1-10) encounters • 4% were “frequent” (10-40) encounters • 1% were “regular” (40 or more) encounters • This data can be used to monitor social interactions and enhance location information

25. BD modelling BD Modelling

26. BD modelling Operational Research modelling of PAM • Aim is to develop a “natural history” model for BD and use it to test the sensitivity and specificity of the PAM algorithms for detecting change in a patient’s health status, in the context of:- • A random (personalised) selection of sensors • Unknown reliability of the chosen sensors and the computer network system • Occasional failure (or deliberate removal) of a sensor • Variety in patient behaviour, in all states of health

27. BD modelling Challenges for modelling BD • No OR modelling approach of BD in the literature, although some Markov models for depression (Patten et al, 2005) • No universally accepted staging models for BD found in the medical literature • Symptoms vary among patients ; and patients may exhibit mixed behaviour (manic and depressed) • Lack of easily measurable criteria • Took advice from clinical psychiatrist on our Steering Group

28. Initial conceptual model of BD BD modelling “Normal” Depressed Manic

29. …  = 0  = 1 BD modelling Final state transition model • The parameter  represents mental health state: totally depressed ( = 0), “normal” or healthy ( ≈ 0.5) and totally manic ( = 1) • Each day, with a certain probability, the person may either stay in the same state, or progress to an adjacent state, in steps of 0.01

30. BD modelling An illustrative sample path for λ

31. Hours of sleepPhone calls Normal 64 Depressed 101 Manic 212 BD modelling

32. BD modelling PAM-detected physical activity levels during various mood states

33. rule-based sensor network rule-based sensor networks

34. rule-based sensor network Programming sensor networks (PROSEN) • distribute rules to rule engines embedded in smart sensors • flexible programming • support for run-time updating of rules • aids personalisation and changing mental states • initial work in a wind farm setting ….

35. rule-based sensor network PROSEN & REED • REED (Rule Execution and Event Distribution): • supports the distribution of rules and trigger events • employs a rule-based paradigm : • allows sensor networks to be programmed at run time • allows allow sensor network behaviour to be changed at run time • allows subscribe-notify service to be constructed • potential for processing, filtering and collating data

36. rule-based sensor network Communications paradigm • Low-level decision and event driven • Interact by sending/receiving decisions and events • Low-level decision: • <trigger event, condition, action> • Executed if the condition is true • manipulate/store local data • generate events • may generate low-level decisions • Event received from: • components in PN • Neighbour PN • Policy server Test of a local state

37. Sensor controller Middleware Interface event condition action … event condition action Operation System Interface Operation System Processing Storage Communications Low-level AI (“novelty” filter) Sensor diagnostics rule-based sensor network REED Middleware architecture … Decision Space management Function call Event Decision Decision Space Initial default decisions <“power up”, true, “sending HELLO event”> <“temp sensor reading update”, “temp < -20”, “send ‘temp too low’ event to Policy server”> Decision Event Event Decision

38. rule-based sensor network Mobile phone-centric sensor-based care system 39

39. Backend – Gateway Connection rule-based sensor network

40. Network Interface rule-based sensor network

41. Mobile Phone Based Body Area Network rule-based sensor network

42. PAM Sensor Reading (PSR) rule-based sensor network <Readings> <Readingset Message_type="gps" Entity="egps" Entity_instance=“aaa_extgps" Frequency="1" Unitoftime="4s" Id="1251993327994" /> <Sr Ref="1251993327994">50.936348, -1.393458, 0.0, 4.0</Sr> <Readingset Message_type="wl" Entity="w" Entity_instance=“aaa_wearable" Frequency="1" Unitoftime="s" Id="1251993354943" /> <Readingset Message_type="wa" Entity="w" Entity_instance=“aaa_wearable" Frequency="1" Unitoftime="s" Id="1251993354952" /> <Sr Ref="1251993354952">-0.5083, 1.7986, 0.0782</Sr> <Sr Ref="1251993354952">-0.1173, 1.1339, 0.2346</Sr> <Sr Ref="1251993354952">-0.0782, 0.8993, 0.1173</Sr> <Sr Ref="1251993354943">2.0, 0.0</Sr> ...

43. MOBILE RULE-BASED APPLICATIONS rule-based sensor network • Custom Symbian S60 Java ME applications installed on the mobile phone interfaced with m-Prolog. (Also Sony-Ericsson) • PAM-Gateway • Control data capture from wearable units (such as GPS, accelerometer, ambient light and sound levels) • PAM-Transfer • Perform automatic mobile to PC data transmission • PAM-Q • Dynamically adjustable questionnaires

44. RELIABILITY AND ACCEPTABILITY ISSUES rule-based sensor network • Mobile phone battery life • On-body gateway disconnection • On-body device form factor issues • Environmental sensor reliability issues • Rule coherence

45. POWER ISSUES Internal GPS: 5 hours @ 0.68 w rule-based sensor network BT, but no storage: 7.5 hours @ 0.48 w BT, and storage: 5.5 hours @ 0.67 w No BT & no user applications: 9 hours @ 0.41 w

46. Rule Coherence when rules are: changing over time possibly unique for particular individuals originating from different stakeholders how can we ensure the integrity of the rules in particular the lack of conflicts between rules rule-based sensor network 47

47. Example: “traditional” feature interaction Alice cannot call Charlie Originating Call Screening (OCS) If Alice calls Bob Bob’s Call Forwarding transfers call to Charlie Charlie Alice OCS CFx Bob rule-based sensor network X 48

48. classes of feature interactions MAI: Two (or more) features control the same device (Multiple Action Interaction) STI: One event goes to different services which perform different conflicting actions (Shared Trigger Interaction) off F D hot D F on F hot F rule-based sensor network heater Power Saving Env cntrl temp air con wind cntrl 49

49. classes of feature interactions SAI: A service performs an action on a device which triggers another feature. The chain might involve any number of links (Sequential Action Interaction, Loops) MTI: The existence of one feature prevents the another one from operating. (Missed Trigger Interaction) F close D F ! D F off F cold rule-based sensor network Env Cntrl blinds move alarm temp heat cntrl Power Saving 50

50. Conflict Analysis rule-based sensor network Missed Trigger Interaction occurs when the Context Triggering rules delay the activation of a home gateway. • Offline and online analysis looking for conflicts between device rules • Like FI for call control • Searching for 5 types of conflict: • STI, SAI, LI, MAI, MTI • 12 case studies were developed to explore the conflicts 51 51