1 / 42

experience sampling method: preliminary findings

experience sampling method: preliminary findings. sunny consolvo & miriam walker intel research seattle sunny@intel-research.net user-centered evaluations of ubicomp workshop 4 th int’l conference on ubiquitous computing september 29, 2002. before we get started….

kanoa
Download Presentation

experience sampling method: preliminary findings

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. experience sampling method: preliminary findings sunny consolvo & miriam walker intel research seattle sunny@intel-research.net user-centered evaluations of ubicomp workshop 4th int’l conference on ubiquitous computing september 29, 2002

  2. before we get started… • this work ispreliminary • wrapped up data collection this month • expect future publication experience sampling method -- ubicomp 2002

  3. outline of talk • intro to experience sampling method (esm) • case study: the personal server • issues • wrap-up experience sampling method -- ubicomp 2002

  4. esm: a brief introduction • also called: • beeper studies • time sampling • field study technique from psychology • used to understand areas such as… • mood • time use • social interactions • we’re exploring its uses for ubicomp experience sampling method -- ubicomp 2002

  5. esm: a brief introduction • fill out questionnaire when alerted • researcher is not present • alerted several times per day • business as usual until alerted • duration is often 1-3 weeks experience sampling method -- ubicomp 2002

  6. running esm: tools • need to choose tools for: • alerting participant • random alerts • timed alerts • event-based/user-driven • delivering questionnaire • capturing participant responses experience sampling method -- ubicomp 2002

  7. choosing tools: alerting • tools for alerting the participant include: • cell phones • pagers • watches • PDAs • custom devices • some desirable abilities: • specify time period during which alerts may go off • specify # of alerts per time period & overall experience sampling method -- ubicomp 2002

  8. choosing tools: delivering • tools for delivering questionnaires include: • paper booklets • PDAs • phones (traditional or mobile) • live researcher • recorded message • audio recordings (e.g., cassette player) experience sampling method -- ubicomp 2002

  9. choosing tools: capturing • tools for capturing responses include: • paper booklets • PDAs • phones (traditional or mobile) • live researcher • recording service (e.g., Tellme) • audio recordings (e.g., cassette player) • cameras experience sampling method -- ubicomp 2002

  10. esm: some advantages • reduces effects of intrusive observers • larger sample size & time interval • reduces memory biases by asking about current activity, feelings, etc. • it’s all about now experience sampling method -- ubicomp 2002

  11. esm: more advantages • less time consuming than observations, yet… • study activities as they occurin the field • study activities and emotions often not accessible to observers • not as disruptive to others experience sampling method -- ubicomp 2002

  12. esm: some disadvantages • questionnaires must be brief • participant must carry tools at all times • power consumption and/or service availability • certain events may only show up in very large samples sizes/durations • more intrusive than interviews & surveys experience sampling method -- ubicomp 2002

  13. outline of talk • intro to experience sampling method (esm) • case study: personal server • issues • wrap-up experience sampling method -- ubicomp 2002

  14. case study: personal server • personal server team was interested in: • how people use information • what they want • when they want it • where they want it • used esm to help collect this data experience sampling method -- ubicomp 2002

  15. pilot study #1 • 4 participants • lab researchers & administrative staff • palm m100s for • alerting • delivering • capturing • original boston college ESP software • for running ESM on palms • duration: 1½ - 4 days experience sampling method -- ubicomp 2002

  16. pilot study #1: problems • palm dropped  batteries fell out  data lost • only method of alert was beep (m100) • participants wouldn’t carry it in several situations • once volume of beep was set, it couldn’t be changed • only response option: multi-choice, single answer • difficult for participants to answer some questions • e.g., “where are you?” • some participants wanted a bag to carry the palm • participants wanted to know about missed questionnaires experience sampling method -- ubicomp 2002

  17. pilot study #2 • 4 participants • lab researchers, administrative staff, family • modified iESP software • palm m500s for alerting and delivering • for capturing: • 2 participants carried fold-out keyboards (for palm) • 2 participants carried paper booklets • all 4 carried digital cameras • for describing where & what they were doing • duration: 1½ - 4 days experience sampling method -- ubicomp 2002

  18. pilot study #2: justifications • upgrading to m500s: • ability to alert via beep and/or vibrate • almost impossible for batteries to fall out • iESP software added functionality, including: • freeform text entry • multi-choice, multi-answer • new formats for multi-choice, single answer • slider (e.g., “on a scale of 1 to 5…”) • ability to change volume, vibrate, etc. during study • ability to check how many questionnaires have been completed vs. asked during study experience sampling method -- ubicomp 2002

  19. our implementation: the study • 31 participants • recruited from local community • pre/post-study interviews + 1 week with equipment • 10 questionnaires/day • 70 questionnaires total • incentive (in USD) • $50 for participating and returning equipment • $1 per completed questionnaire • $120 total possible experience sampling method -- ubicomp 2002

  20. the study (continued) • palm m500 to alert, deliver, & capture • modified iESP software • more about capturing: • also used camera • free-form data entry: • paper booklet for all 31 participants • graffiti / soft keyboard for all 31 participants • full-sized fold-out keyboard for 9 of 31 participants experience sampling method -- ubicomp 2002

  21. what the participants carried experience sampling method -- ubicomp 2002

  22. type of data collected • responses to following types of questions: • yes / no • multiple choice, single answer • multiple choice, multiple answers • freeform text entry • looks a lot like survey data except • multiple responses from same participant in variety of circumstances over time experience sampling method -- ubicomp 2002

  23. other data collected • qualitative data from interviews • participants’ photos • some photos were of interest to the participant • other photos documented where s/he was or what s/he was doing experience sampling method -- ubicomp 2002

  24. questions on the palm experience sampling method -- ubicomp 2002

  25. more questions experience sampling method -- ubicomp 2002

  26. participant photos: in transit experience sampling method -- ubicomp 2002

  27. participant photos: at home experience sampling method -- ubicomp 2002

  28. participant photos: at work experience sampling method -- ubicomp 2002

  29. participant photos: retail experience sampling method -- ubicomp 2002

  30. participant photos: pets experience sampling method -- ubicomp 2002

  31. outline of talk • intro to experience sampling method (esm) • case study: personal server • issues • wrap-up experience sampling method -- ubicomp 2002

  32. our tool decisions… • chose palm because: • no service problems—cost or coverage • power consumption not an issue for duration • know when participant fills out questionnaire • probabilities and contingencies can be used • if a, then b • if b, then not c • d 50% of time • don’t have to transcribe data experience sampling method -- ubicomp 2002

  33. tool decisions • used camera to see if it would help participants describe… • where they were • what they were doing • photos used to jog memories during post-study interviews experience sampling method -- ubicomp 2002

  34. according to the participants… • 1/31 would not participate again • 2/31 would only do it for more money • 28/31 would participate again • “it was cool” • “I felt like a spy” • “friends were jealous of the [fold-out] keyboard” • a lot to carry • mixed feelings about carry bag • several claimed they would be purchasing palms • only one participant wanted voice recording option • when asked, most did not respond favorably experience sampling method -- ubicomp 2002

  35. from our perspective… • woo-hoo! we got all of the equipment back • only 1/31 palms was broken • of the 31 participants and 70 questionnaires: • 28 participants completed 50+ questionnaires • 13 of whom completed 60+ questionnaires • 2 participants completed 25- questionnaires • 1 participant performed a hard reset and returned the palm with no data or software experience sampling method -- ubicomp 2002

  36. outline of talk • intro to experience sampling method (esm) • case study: personal server • issues • wrap-up experience sampling method -- ubicomp 2002

  37. wrap up: we’ll use it again • pleased with results so far • intend to use esm again experience sampling method -- ubicomp 2002

  38. wrap-up: pda software • original ESP software available at: • http://www2.bc.edu/~barretli/esp/ • iESP software (includes upgrades): • launching on intel research seattle web site in Oct or Nov 2002 • if you want to be notified… • business card with “iESP notification” written on it • legibly fill out notification sign-up sheet experience sampling method -- ubicomp 2002

  39. acknowledgements • drs. lisa & daniel barrett • justin silves • gaetano borriello • ken anderson • rahul shah • trevor paring • roy want • john light • our pilot testers • and our colleagues at intel research seattle experience sampling method -- ubicomp 2002

  40. questions? experience sampling method -- ubicomp 2002

  41. examples of study questions • where is your cell phone? in my… • hand • pocket/belt clip • bag • car • home • office • somewhere else and with me • somewhere else and not with me experience sampling method -- ubicomp 2002

  42. examples of study questions • in the last 30 minutes, which of the following would you have liked to have access to but didn’t have available? • web pages • movie times • restaurant reviews • personal calendar • telephone number(s) • computer files • music files • shopping list • “to do” list experience sampling method -- ubicomp 2002

More Related