1 / 26

Dialog Design - Gesture & Pen Interfaces, Mobile Devices

Dialog Design - Gesture & Pen Interfaces, Mobile Devices.

ginata
Download Presentation

Dialog Design - Gesture & Pen Interfaces, Mobile Devices

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dialog Design - Gesture & Pen Interfaces, Mobile Devices This material has been developed by Georgia Tech HCI faculty, and continues to evolve. Contributors include Gregory Abowd, Jim Foley, Elizabeth Mynatt, Jeff Pierce, Colin Potts, Chris Shaw, John Stasko, Bruce Walker, and Melody Moore Jackson. Comments directed to foley@cc.gatech.edu are encouraged. Permission is granted to use with acknowledgement for non-profit purposes. Last revision: October 2007. IAT 334

  2. Dialog Styles 1. Command languages 2. WIMP - Window, Icon, Menu, Pointer 3. Direct manipulation 4. Speech/natural language 5. Gesture & pen IAT 334

  3. Agenda • PDA overview • Pen input styles IAT 334

  4. How to use a PDA IAT 334

  5. Personal Digital Asst. (PDA) Apple iPhone Palm Treo Apple Newton (1993) Blackberry Curve IAT 334

  6. PDAs • Now ubiquitous • Small displays • Often touch and pen interfaces • Small thumb-based keyboards • Recent Improvements • Wi-Fi, GPS, more memory, better CPU, better OS, BlueTooth IAT 334

  7. Is it a PDA? Phone? GPS? Camera? Computer? • Line between devices is blurred today • Apple iPhone – phone, MP3 player, PDA, camera • Palm Treo 700w – phone, Windows computer, PDA, camera • Nokia N82 – Phone, GPS, 2 cameras, robot brain IAT 334

  8. Cally/Callo IAT 334

  9. No Shredder… IAT 334

  10. Input Options • Pen / Stylus is dominant form • Main techniques • Free-form ink • Soft keyboard • Numeric keyboard => text • Stroke recognition - strokes not in the shape of characters • Hand printing / writing recognition • Sometimes have or can connect keyboard IAT 334

  11. Free-form Ink • Ink is the data, take as is • Human is responsible forunderstanding andinterpretation • Like a sketch pad IAT 334

  12. Soft Keyboards • Common on PDAs and mobile devices IAT 334

  13. Soft Keyboard • Presents a small diagram of keyboard • You click on buttons/keys with pen or finger • QWERTY vs. alphabetical • Tradeoffs? • Alternatives? Apple iPhone soft keyboard IAT 334

  14. Numeric Keypad -T9 • Tegic Communications developed • You press out letters of your word, it matches the most likely word, then gives optional choices • Faster than multiple presses per key • Used in mobile phones • www.tegic.com/t9 IAT 334

  15. Stroke Recognition - Graffiti • Graffiti - Unistroke alphabet on Palm PDA • What are yourexperienceswith Graffiti? • Graffiti demo: • http://www.youtube.com/watch?v=R-hbjG2hzuk IAT 334

  16. Stroke Recognition - Cirrin • Developed by Jen Mankoff (GT -> Berkeley CS Faculty) • Word-level unistroke technique • UIST ‘98 paper • Use stylus to go from one letterto the next -> • Nokia N8 does similar with QWERTY layout IAT 334

  17. QuikWriting • Break the gesture into octant components: • Start pen in center, • drag in one of 8 directions • drag along edge • drag to center http://mrl.nyu.edu/~perlin/demos/quikwriting2_1.html IAT 334

  18. Hand Printing & Hand Writing Recognition • Recognizing letters and numbers and special symbols • Lots of commercial systems • English, kanji, etc. • Not perfect, but people aren’t either! • People - 96% handprinted single characters • Computer - >97% is really good • OCR (Optical Character Recognition) IAT 334

  19. Recognition Issues • Off-line vs. On-line • Off-line: After all writing is done, speed not an issue, only quality. • Work with either a bit map or vector sequence • On-line: Must respond in real-time - but have richer set of features - acceleration, velocity, pressure • Use best-guess pattern matching, including digram, trigram probabilities and word lists to remove ambiguity • 1 I l IAT 334

  20. More Issues • Boxed vs. Free-Form input • Sometimes encounter boxes on forms • Printed vs. Cursive • Cursive is much more difficult to impossible • Letters vs. Words • Cursive is easier to do in words vs individual letters, as words create more context IAT 334

  21. Pen Gesture Commands • Might mean delete • Insert • Paragraph Define a series of (hopefully) simple drawing gesturesthat mean different commands in a system IAT 334

  22. Pen Use Modes • Often, want a mix of free-form drawing and special commands • How does user switch modes? • Mode icon on screen • Button on pen • Button on device IAT 334

  23. Error Correction • Having to correct errors can slow input tremendously • Strategies • Erase and try again • When uncertain system shows list of best guesses • ... IAT 334

  24. A Different Application • Signature verification • But not with a mouse :) IAT 334

  25. Multi-touch interfaces • Apple iPhone Capacitive touchscreen: http://www.apple.com/iphone/features/index.html#touch Gestures: flick, tap, pinch, un-pinch http://www.apple.com/iphone/features/index.html#map IAT 334

  26. Pen Videos • Pick-and-Drop by Rekimotohttp://www.youtube.com/watch?v=rFw9aMubL-Y • I-Love-Sketch by Seok-HyungBaehttp://www.youtube.com/watch?v=5Hd2clwURlQ • Jabberstamp by Hayes Raffle http://www.youtube.com/watch?v=NIe-XDHcsOE • ShapeWriteriPhone App http://www.youtube.com/watch?v=TOg91yfvZpo • Marginalia : The Hybrid Textbook http://www.youtube.com/watch?v=zwYdmsdqqTk IAT 334

More Related