Life <--> Logging

1. Life <--> Logging Melekam Tsegaye Shaun Bangay Alfredo Terzoli Research area: Wearable, Pervasive and Ubiquitous Computing http://www.cs.ru.ac.za/research/students/g98t4414/research 25/11/2004

2. Life logging • Recording/capturing everything you see and do. • Past, present and future events • Why ? • for the sake of improving your memory • Your memory defines who you are! • Amnesia?

3. Forms of life logs • Manual paper diaries • Electronic diaries on PDAs • Audio/Video recordings • Emergence of the Weblog - blog

4. Current Trends • Weblogs are mostly text and image based • e.g. Blogger.com (blog via email) • Next natural transition is audio blogging • Individuals make an audio recording, publish to a web server e.g. audioblog.com • Hardware needed to start life logging • Camera, microphone, local storage (Digicam/Cell phone) • Web server

5. audioblog.com • Effortless audio logging • recording audio to a remote server from • your web browser • cell phone • email • audioblog.com recently introduced a video blogging extension

7. Life-log applications • Nokia lifeblog • Software for organizing video clips recorded using a cell phone camera

9. Life-log applications(2) • Microsoft sense cam • Passive Capture, improved participation in events • ACM workshop on Continuous archival and retrieval of personal experiences  New York, USA October  2004 • Correlations and relationships from sensors used to manage captured images • light intensity sensor • x/y tilt (tilt sensor ) • motion sensor to avoid taking blurry pictures (accelerometer) • temperature • infra red sensor • clock • location • MMC card storage • 2xAA cell batteries, 12 hours • Size of a pager • Useful: quick replay of a day’s events

10. Our work • wPim - Wearable audio enhanced personal information management system • Record audio wherever you are • Organize people and events (past, future and present) • Retrieve using time of capture and textual descriptors

11. Current Work • Once you capture your memories how do you retrieve segments of interest for later use ? • using time, location ? • We want to analyze data produced by multiple sensors • at time of av capture and extract useful information • to be used to auto tag complex streams such as video and audio • Assembling a large number of sensors and conducting experiments can be difficult • Some sensors are not available in a convenient form • We waste too much time dealing with hardware issues and produce poor software solutions

12. Current Work(2) • The approach we take in our attempts to solve the problems mentioned earlier is • To define multiple virtual sensors that sense environmental and user states • make the virtual sensors generate data (essentially, simulate the process sensing data) • extract useful contextual information from the generated data • make use of the extracted information to build a contextual tag database • use the information in the tag database to automatically tag complex data streams such as video and audio streams • Not simulation only, we want to be able use a combination of virtual and real sensors to run our experiments • E.g. A wearable life logging facility that uses a combination of virtual and real sensors

13. Prototype sensors for: Video, Audio, Locationother sensors and I/O devices through a multi-channel (a2d) data capture device.

14. Problems • Social issues • Privacy • Laws • Hostility and fear from society • First person experience capture vs big brother scenario • Hardware limitations • Wireless network connectivity is not yet ubiquitous, bandwidth limitations • long lasting power sources • comfortable wearable hardware

15. Questions ?