Applied Pervasive Computing

1. Applied Pervasive Computing Presentation 2: Pervasive Computing Introduced

2. Agenda • Mission: • Establish basic terminology / background knowledge • Deeper insight at later courses (advanced pervasive computing) • Historical Background / UbiComp • Mark Weiser / Ubiquitous Computing • Pervaisve Computing Today • Main areas • Related terms • Mobile Computing • Devices and infrastructure • Sentient Computing • RFID, Bluetooth, Barcodes • HCI / AI • Developing • Where to get more info

3. Historical Background • 70’s and 80’s: • focus on PC and GUI development • Xerox PARC research into alternatives • (Xerox PARC invented the PC and GUI/Windows) • 80’s Mark Weiser introduces new paradigm • Ubiquitous Computing (Weiser) • The Invisible Computer (Norman) • Next Generation replaces ”Office Metaphor” / PC • Implosion: computers getting smaller • Explosion: interactive rooms, augmented reality • “computere i alting” or ”ting der tænker” • Today: also known as pervasive computing Mark Weiser (1952-1999) The origins of ubiquitous computing research at PARC in the late 1980s http://www.research.ibm.com/journal/sj/384/weiser.html

4. Mark Weiser Definition • Mark Weiser envisioned: Ubiquitous computing is just now beginning. First were mainframes, each shared by lots of people. Now we are in the personal computing era, person and machine staring uneasily at each other across the desktop. Next comes ubiquitous computing, or the age of calm technology, when technology recedes into the background of our lives. Mark Weiser is the father of ubiquitous computing (1991). [Mark Weiser, "The Computer for the Twenty-First Century”, Scientific American, pp. 94-10, Sept. 1991] What Ubiquitous Computing Isn't Ubiquitous computing is roughly the opposite of virtual reality. Where virtual reality puts people inside a computer-generated world, ubiquitous computing forces the computer to live out here in the world with people. Virtual reality is primarily a horse power problem; ubiquitous computing is a very difficult integration of human factors, computer science, engineering, and social sciences. Ubiquitous: Being or seeming to be everywhere at the same time; omnipresent (DK: allestedsnærværende)

5. UbiComp at Xerox PARC • At Xerox PARC – UbiComp was Implemented Liveboard Infra Red Transciever Pads Tabs

6. UbiComp at Xerox PARC • Tabs, Pads and Boards for interaction • More natural interaction within context of work, rather than PC • Wireless infrastructure, PAN / LAN / WAN • Enabled data access “anywhere, anytime” • Context awareness (Olivetti Research: Active Badge) • Auto redirect phone calls and find people, • Auto “push” relevant data and application within new context • Interaction Substrates (alternative interaction devices) • Alternatives to mouse and keyboard: speech, hand typewriting

7. Many Others • iLand • RoomWare, DynaWall, CommChair, • InteracTable, more.. • Proces Data (Denmark) 1990’s • The Intelligent Building • All connected by a network (fieldbus) • Access from all PC’s, and small touch-screens • Windows, Radiator Heater, Curtains, Lights • When too hot – all Windows opens, and the thermostats turn down the Radiators, when too much light, dim the lights – and use curtains (Calm technology), when no people in building – turn off the lights • Pig-Feeder 1990’s • Using RFID tags in pigs ears, the feeding station will recognize the pig, and calculate whether it has already received its ration • Data is collected and used for optimization calculations • Also: if pig is selected for separation – it is automatically separated

8. Pervasive Computing • Term more frequently used today than Ubiquitous • IBM - Pervasive computing defined : • “Pervasive computing describes access to information using new communications or networking technology. The technology implies computing power, freed from the desktop, extended to wireless handheld devices, automobile telematics systems, home appliances, and commercial tools-of-the-trade.” • Important aspects: • Mobile Computing / Embedded / Distributed Computing • Sentient Computing / Sensor Networks / Context Awareness • Implicit HCI / AI / Agent Technologies / Calm technology What Pervasive Technologies do you know? Have you worked with these technolgoies? Are you involved in any Pervasive Projects? Pervasive: pervasive (spreading or spread throughout), pervade (spread or diffuse through ), DK: gennemsyrende

9. Related Terms • Tangible Computing • Interacting with the digital world through real world physical objects – e.g. paper, moving objects and more • http://sigchi.org/chi97/proceedings/paper/hi.htm • Embodied Virtuality • Same as Ubiquitous / Pervasive computing • Opposed to Virtual Reality • Augmented Reality • Enhancing the ”real world” • See-through displays with overlay graphics • Projecting digital images on surfaces • Digital Pen / Digital Paper

10. Pervasive Computing Topics • Location awareness • Context awareness • Activity recognition • Smart objects (Internet-of-Things) • Automated capture and access • Natural interfaces / continuous interaction / everyday computing / invisible computing / calm technology • Smart spaces

11. Enabling technologies • Mobile • Distributed • Sentient (context-aware) • Embedded

12. Mobile Computing • Key element of Pervasive Computing • Using Tablet PC’s, PDA’s, Cell phones or Information Appliances (Norman) to access “Overall Information Space” anytime / anywhere • Relies on Distributed Computing for infrastructure • PAN / LAN / WAN technologies • IrDa, ZigBee, WiBree, Bluetooth, WiFi, GSM/GPRS/UMTS/Tetra • Middleware: CORBA, Web services, Java RMI, .NET Remoting • Problems to consider: • Range, Battery life, Bandwidth, Security, • Processing power, COTS availability, Cost • Symbiotic Web-of-Technologies • HealthCare@Mobile • Taxmaster4: PDA or Cell Phone

13. Sentient Computing • Related concepts: • Sensor Networks & Context/Location Awareness • Using sensors to make devices aware of the current context • Hospital bed “senses” which patient is in the bed • And informs the central computer • Mobile device can “sense” a patient • Enabling Technologies • Sensors collecting data, temperature, blood pressure, TM4: CAN/Pulse, accelerometers, • RFID / NFC / SemaCodes / Barcodes for identification, • Bluetooth, WiFi, (UWB) for local positioning, • GPS, GSM triangulation, for global positioning • JCAF: Java Context Awareness Framework

14. Context Awareness Equipping the embedded devices with intelligence Devices understand what is expected of them and reacts Context Awareness (other half from Sentient) When starting car – TM4 mobile program starts (active context awareness) When driving: TM4 embedded device records driving (passive context awareness – recorded for later use) When nurse arrives at patients home – SIH mobile application is automatically started, and the right information is sent to mobile device Norman: devices too complex – use information appliances

15. Mobile Device Development • WAP: WML / HTML / XHTML • Most cell phones support WML today (and most smartphones many HTML) • Problem: no port access (no Bluetooth interface, no sensors) • Java: J2ME MIDP CLDC (v. 1.0 & 2.0) • Most cell phones support v. 1.0, many 2.0 and Bluetooth/WS • Symbian C++ • Many based on Symbian, only fraction of J2ME supported • Differences in C++ API • Windows Phone 7 • (Windows CE) / PDA’s / MDA’s • .NET Framework, (C#, CLS), eMbedded C++ / VB • Android • Open Handset Alliance, open source • Java, easy to program • iOS (iPhone) • Limited to Apple devices • ObjectiveC

16. Developing Pervasive Applications • Communication infrastructure • E.g. Bluetooth for PAN, WiFi for LAN, GPRS for WAN • ZigBee, Z-Wave, 6LoWPAN • Web services, ICE, CORBA • Mobile Device • Mobile phone / Smartphone / PDA / Dedicated • J2ME MIDP, JSR 82 (Bluetooth), JSR 175 (Web services), Netbeans with Mobile Extension • .NET CF C#, VB, C++ • Local Server • WiFi / Bluetooth based / ZigBee • Sensors • Bluetooth equipped sensors, NFC, RFID, Internet of Things • Central Server • Using a Web Server: Apache, Apache Tomcat, IIS, with database

17. More Info • Conferences: • International Conference on Ubiquitous Computing (Ubicomp) • http://www.ubicomp.org/ • International Conference on Pervasive Computing (Pervasive) • http://www.dgp.toronto.edu/conferences/pervasive2007/index.phtml • International Conference on Pervasive Computing (Pervasive) • http://www.percom.org/ • Pervasive Healthcare • Magazines: • Pervasive Computing (IEEE) • http://www.computer.org/portal/site/pervasive/ • Personal and Ubiquitous Computing (Springer) • http://www.springerlink.com/content/1617-4917/

18. Videos • Continua Alliance: • http://www.continuaalliance.org/about-the-alliance/continua-vision-video.html • MS Fremtidsvideo: • http://www.youtube.com/watch?v=V35Kv6-ZNGA&feature=related • Intel Health guide video: • http://www.youtube.com/watch?v=6u-bhsXd0OA • RFID: • http://www.youtube.com/watch?v=ZsxIVXvlEtc&feature=related • Shimmer modulet: • http://www.youtube.com/watch?v=Earu8WP-bM4&feature=channel • Smart Cane: • http://www.youtube.com/watch?v=PXkpXph5FZE&feature=related

19. IHA Pervasive Computing Projects • Pervasive Healthcare • OpenCare Project / SIH • Firefighter (with Systematic) • Pervasive Home • Minimum Configuration Home Automation (MC HA) • Pervasive Agriculture • IIOSS (Intelligent Identification and Observation of Sows and Slaughter pigs) • Pervasive Automation • The Taxmaster 4 project series

20. SIH / OpenCare Project • Started September 2007. • Last three semesters: more than 30 students involved • 10 IHA Professors • More than 30 projects already completed • Only a few incorporated

21. Age Tsunami Approaching

22. Pervasive Healthcare • Emerging Discipline • Elements from • Bio Medical Engineering • Medical Informatics • Telemedicine • Pervasive Computing • iHospital • ElderCare • Assisted Living • Acute Medicine

23. Assistive Living & Home Care • Focus area at IHA • Keep Elderly at home longer • Avoid Transport to Hospital / GP • Early Warning: • Blood Pressure • ECG / HRV • Fall Detection • Fluid Balance • Others

24. Current Research Projects • Infrastructure & Zero-Configuration Framework • Automatic Medicine Dispenser • Heart Rate Variability & ECG • Fluid Balance • Personal Communication Device and Fall Detection Efficient and Open Infrastructure

25. What is Infrastructure? • Getting sensor data to the right recipients • Secure and reliable communications, • Resource redundancy • Providing UI and persistence features • Hiding away the complexities of a distributed system • Fast development and deployment of 3rd party sensors

26. Why is it Important? • If infrastructure is not designed carefully you will get problems with: • Security • Distribution of data • Bandwidth (both PAN, LAN, WAN) • Operating time of Power Cells (Batteries) • Extendibility of your software • Openness and Flexibility

27. The Need for an Open and Easily Extendible Infrastructure • If each sensor vendor provided his own infrastructure, then each vendor would have to integrate with hundreds of Hospital EPR and GP systems – Extremely costly and inefficient • Healthcare professional will not be able to combine equipment from different vendors, as the economic overhead of the infrastructure equipment would be too great • Most existing vendors protect their gateway and infrastructure, instead of opening up (vendor lock-in) • Our idea: open standards, open source code, component oriented, and multiple middleware platform support

28. Technologies Used • We have chosen the .NET platform • Very efficient and easy • Deployment on Mono/LINUX and Windows platforms • Effective for pervasive platforms • Multiple programming languages • Excellent integration with legacy C and C++ • Excellent wireless capabilities • Excellent GUI capabilities / high usability

29. Technologies Used • Supports Bluetooth • Planning on several others • Open platform: write your own • Supports ADSL, UMTS, GPRS • Combine for redundancy • Will support Web services (WCF), CORBA/IIOP • Component based - low coupling – replace components • Supports plug-in of GUI / SUI elements and more • Change the whole GUI, integrate into own system

30. Technologies Used • Windows Touch screen based PC chosen • Considered using a mobile platform but: • Mobile platforms have huge battery issues • Limited wireless capabilities • Very limited GUI/SUI capabilities • Limited usability • Low processing power • Windows PC platform does not have these problems: • Touch-screen UI is extremely user friendly interface for healthcare professionals and relatives • Only problem: it is not mobile, what happens when the user gets mobile?

31. Technologies Used • A Symbiotic Web-of-Technologies • Personal Communication Device supports mobile users • Periodically activates the BT stack to check • Synchronizes drivers with stationary PC • Performs wireless sensor handoff when out of range

32. Challenges • Security is still a challenge • Will other research teams join in on our open source idea and participate in developing it further? • Will commercial vendors join and open up their systems- or integrate with this? • Is this too complex to unify?

33. IIOSS Project • Started September 2008. • Ended October 2010. • 13 students worked on projects • 3 student programmers • 6 IHA professors

34. VISION • 2013 EU Requirement • Help needed: • Identification • Pin-pointing • Additional: • Behavioral tracking • Weight measurement • Disease and wound identification

35. TECHNOLOGIES BEING TESTED • RFID (active and passive) • Camera / Signal processing • Laser / projector • PDA, cell phone, • Speech, Gesture, Laser • Infrastructure: LAN, WiFi, Bluetooth, GPRS

36. What do you THINK? • What is the potential here? • What are the challenges? • Are you up for it?