Using Handhelds to Enhance Classrooms and to Help the Handicapped

1. Human Computer Interaction Institute School of Computer Science Carnegie Mellon University Using Handhelds to Enhance Classrooms and to Help the Handicapped Brad A. MyersCarnegie Mellon University bam@cs.cmu.edu http://www.cs.cmu.edu/~pebbles SUNY Technology Conference 2002

2. Overview in Previous Session • Pebbles project: • How handhelds can be usedat the same time as other devices • Many domains, applications and research topics

3. Overview of This Talk • Pebbles applications for classrooms • Slideshow Commander for the presenter • Or RemoteCommander or Shortcutter • Using handhelds for real-time “Concept Tests” • Pebbles applications for the Handicapped • Handhelds as an assistive technology for people with Muscular Dystrophy and related disorders

4. Presenter Use Slideshow Commander for the Presenter

5. SlideShow Commander • For PowerPoint • PC Only: Windows 95, 98, NT, 2000, ME, XP • Use PC to give the presentation • Full features of PowerPoint • Use hand-held as “remote control” for PC • Can wander away from the keyboard • Two-way communication • View, Navigate, Highlight, Time

6. Studies of Presentations • Summer study of 2 HCI Master’s students • Contextual Inquiry of 9 presentations • Audience: 10 to hundreds • 7 used PowerPoint, 4 used NetMeeting • Found 220 “breakdowns” • Most were minor problems • Averaged 8.7 lost minutes per talk (14.5%) • Designed SlideShow Commander to eliminate some of these problems

7. View on Hand-Held: slide • See thumb-nail of current slide • Black and white or color

8. Scribble on Current Slide • Can handwrite and draw on thumbnail picture • Like with overheads • Or check off items as you go

9. New! Tap On-screen Buttons • Mode on handheld switches between scribble and tap • Invoke sounds, animations, etc.

10. New! Preview forward or backwards • iPaq version only • See what is coming next on the handheld without showing students • Or look back at a previous slide • Jump to that slide without showing ones in between

11. View on Hand-Held: notes • See the notes of the current slide

12. View on Hand-Held: titles • See the list of titles

13. View on handheld: Time • Palm version only • Multi-function timer • View large • And in corner

14. Control Demonstrations on PocketPC • Task list of PC applications • Tap brings one to front • PowerPoint continuesbehind – easy to resume

15. Control Demonstrations on Palm • Control PC applications and external devices • Uses Shortcutter • Create panels of controls for any app • Also can control lights and projector • Easy demo and resume show

16. Commercialized • SlideShow Commandercommercialized by: • Available from Office Depot,CompUSA, etc. • Palm and PocketPC • (Some demonstrated features not available in commercial version) Synergy Solutions, Inc. http://www.synsolutions.com/ http://www.slideshowcommander.com/

17. But what if not using PowerPoint? • Can use RemoteCommander to control any PC application • Can use Shortcutter to send specific keystrokes • One professor uses Shortcutter to control Adobe Acrobat • Still only PC

18. Student Use of Handhelds Using handheld computers in classrooms For testing Improve large lectures Provide immediate feedback to instructor

19. Hardware • HP donated 110 Jornada 680 and 100 720 computers • Windows CE • Lucent donated Wavelan wireless cards

20. Context • Collaborator: Prof. David Yaron of Chemistry • Chemistry 09-106: Modern Chemistry II • Spring, 2000 • About 90 students • Mostly freshmen • Loaned one Jornada and Wavelan card to each student for the whole semester • So would get used to using it • Offset technical difficulties with benefits to student

21. Wireless Andrew • CMU has almost complete coverage of 802.11b support • Wavelan (now ORiNOCO) • Donations from Lucent, etc. • Funding from state of Pennsylvania, etc. • Can use computers wirelessly in all classrooms and offices • Most lawns, dorm rooms, etc. • Windows, Mac, Linux, Windows CE support

22. Testing • Large lectures are a poor way to educate • Work on engaging students • So they pay attention and think about material • “Concept Tests” • Short tests given during lectures • Students discuss, and answer questions again • Evidence that significantly contributes to learning • Mazur, E. (1997). Peer Instruction:A User's Manual, Prentice Hall.

23. Concept Tests • Students have to think about answers • Instructor can determine who is participating • Can determine what students understand • Computerize: • Use technology to provide capabilities not otherwise available • Rather than just a new way to do old things

24. Implementation • Use web-based tests for students • Web-based instructor’s display • Hand-helds and instructor’s computer use standard IE browser • Server provides database of tests • Currently only multiple-choice • Server also stores students’ answers • Tests represented as XML

25. Pebbles Tests • Create through html pages • Arbitrary html in questions and answers • Pictures, formatting, links • Use FrontPage, etc. to author • Embed tags to show question and answers

26. Concept Test • Students allowed to answer multiple times • Server stores all answers • Prof. Yaron displayedquestions and answers onthe board and didn’tauthor in html • Used for lectures and alsoduring in-class chemistrydemonstrations andexperiments

27. Instructor’s View • In class, instructor sees a chart of answers • Optionally refreshes every 3 seconds • May be projected forwhole class tosee • Top displaysinstructions forstudents

28.  For Instructor • Faculty can get data from prior tests • Enables grading,trend analysis • By student name,or anonymous

29. Results • Unfortunately, no data about effects on learning • Students thought concept tests were a valuable part of the class • Students significantly preferred using handhelds for concept tests.

30. Problems • Connection tonetwork worked? • How oftenneeded to reset?

31. Biggest drawback: Keyboard was: Screen size was: Problems, cont.

32. Future • Hope for further study of impact of handhelds • This software is available for you to download off the main Pebbles web page: http://www.pebbles.hcii.cmu.edu/classroom/ • webserversoftware • Written in Perl5

33. Handhelds for the Handicapped Handhelds as Assistive Technology As an alternative input device for a computer

34. History • Drew Rossman foundRemoteCommanderon the web • Found it helped hisdaughter who has a formof Muscular Dystrophy known asSpinal Muscular Atrophy, Type II. • Wrote articles and created webpage to help promote its use • We adapted the software to try toimprove performance

35. Muscular Dystrophy • About 250,000 people in the United States have Muscular Dystrophy (MD) • Use computers for recreation, education, etc. • Often home-bound, so computers may be even-more important • Increased susceptibility to infections • Web provides opportunities for employment • People with MD often have fine-motor control but lose gross motor control • Difficulty with mouse and keyboard, but stylus OK • May tire easily, so having choice of different input devices can be useful

36. Alternative Input Devices • Commercial alternative input devices can be expensive and difficult to use • Must move from keyboard device to trackball device • So use inexpensivePalm instead

37. Adaptations to Remote Commander • Made on-screen buttons for left, right mouse buttons and keyboard modifiers • Physical Palm buttons too difficult to press

38. Adaptations, cont. • Turn off key repeat • So not repeating by accident • Faster Acceleration • Smaller movements on Palm • More flexible tapping • To get mouse button down • Long time before power-off • Palm power button too hard to press

39. Adaptations, cont. • Various Keyboard Layouts • Bigger buttons • Alphabetic layout • Most common keys grouped together to minimize travel distance

40. New! Word Prediction • Predicts word from first letters • Predicts next word from previous word • Layout to minimize distance from keys • Most likely choice in center and highlighted • Available with soft keyboard and Graffiti

41. Case Studies • Observed four people with MD using Remote Commander • Measured typing and mousespeed with PC-based tests

42. Observations • Design changes: • Many of the design decisions already mentioned • Needed lighter & longer stylus • Needed better lighting • Cradle not sufficiently stable: needed long cord; • Positives: • Tiny keyboard requiring small movements • Keyboard and mouse in one place • Ability to control PC from across room • Ability to work with regular keyboard and mouse, for shared computers • Problems: • Battery life is a real barrier • Still much slower than regular keyboard

43. Future work • Handhelds as assistive technologies in other areas • Handhelds as interface to other devices • Control room lights, telephone, wheelchair, etc.? • Handhelds to help with other disabilities • For the visually impaired? • SlideShow Commander • Auditory interface to PC apps, and the environment? • Ideas welcome!

44. Supported by grants from: DARPA Microsoft Pittsburgh Digital Greenhouse NSF General Motors And equipment grants from: Hewlett Packard Lucent Technologies Palm Computing Symbol Technologies IBM SMART Technologies, Inc. Synergy Solutions, Inc. Handango Thanks to Our Sponsors!

45. Human Computer Interaction Institute School of Computer Science Carnegie Mellon University Thank you! Using Handhelds to Enhance Classrooms and to Help the Handicapped Brad A. MyersCarnegie Mellon University bam@cs.cmu.edu http://www.cs.cmu.edu/~pebbles SUNY Technology Conference 2002