In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK - PowerPoint PPT Presentation

johana
in search of text writing methods for off the desktop computing atomik and shark l.
Skip this Video
Loading SlideShow in 5 Seconds..
In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK PowerPoint Presentation
Download Presentation
In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK

play fullscreen
1 / 38
Download Presentation
In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK
308 Views
Download Presentation

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK Shumin Zhai In collaboration with Barton Smith, Per-Ola Kristensson (LinkopingU), Alison Sue, Clemens Drews, Paul Lee (Stanford), Johnny Accot, Michael Hunter (BYU), Jingtao Wang (Berkeley) IBM Almaden Research Center San Jose, CA

  2. Computing off the desktop • Desktop computing “workstation” interface foundation • Large and personal display • Input device (mouse) • Typewriter keyboard • HCI Frontier – beyond the desktop • Interfaces without display-mouse-keyboard tripod • Numerous difficult challenges

  3. The text input challenge • Indispensable user task • Efficiency • Learning • Size / portability • Visual cognitive attention • “History” of writing technology

  4. Text Entry Methods • Reduced keyboard • T9, miniature keyboard • Hand writing • English, Unistroke, Graffiti • Speech • Human factors limitation • Stylus (graphical) keyboards

  5. The QWERTY Keyboard • Invented by Sholes, Glidden, and Soule in1868 ― minimizing mechanical jamming • QWERTYnomics (P. David vs. Liebowitz & Margolis) • Touch typing ― low visual attention demand • Happen to be good for two hands alternation ― Dvorak did not prevail

  6. Wj Key ii Key j Dij Fitts’ law For stylus keyboard — a = 0.08 sec, b = 0.127 sec/bit (Zhai, Su, Accot, CHI 2002)

  7. Letter Transition Frequency (Digraph) • Mayzner and Tresselt (1965) • British National Corpus (BNC) • 2 new modern corpora • News - NY Time, SJ Mercury, LA Times • Chat room logs

  8. 34.2 WPM Movement Efficiency Model of Stylus Keyboards (Soukoreff & MacKenzie,1995; Zhai, Sue & Accot 2002)

  9. Manual explorations OPTI, MacKenzie & Zhang (42.8 wpm) FITALY keyboard (41.2 wpm)

  10. Zhai, Hunter, Smith, UIST2000 Algorithmic design - dynamic simulation Hooke’s Keyboard (45.1 wpm)

  11. Fitts-digraph “energy” • “Random walk” Zhai, Hunter & Smith, HCI 2002 Metropolis Method • UI physics - Keyboard as a “molecule” • Annealing – varying T

  12. 46.6 wpm – 36% more efficient than QWERTY

  13. 30% smaller search area by Hick’s law analysis Smith & Zhai INTERACT2001 Alphabetical “tuning” for novice users Novice user taping speed (wpm)

  14. Word connectivity • Zipf’s law Pi ~ 1/ia • connectivity Index

  15. 18000 16000 14000 Word connectivity Human Movement Study: Fitts’ law MT = a + b Log2(Dsi/Wi + 1) 12000 10000 8000 6000 4000 2000 0 sp E T A H O N S R I D L U W M C G Y F B P K V J X Q Z English Letter Corpus(News, chat etc) “Fitts-digraph energy” Metropolis “random walk” optimization Alphabetical tuning Alphabetically Tuned and Optimized Mobile Interface Keyboard (ATOMIK)

  16. Limitations and hints from ATOMIK • Tapping one key at a time – tedious. The stylus can be more expressive and dexterous. • Does not utilize language redundancy/statistical intelligence. • People tend to remember the pattern of a whole word, not individual letters.

  17. “word” Zhai, Kristensson, CHI 2003 The new phase - SHARK The basic idea: gesturing the word pattern defined by the keyboard

  18. Shorthand Aided Rapid Keyboarding ― SHARK Sample “sokgraphs” (Shorthand On Keyboard)

  19. Principle 1 - efficiency “Writing” one word at a time (not letters)

  20. A form of shorthand

  21. Principle 2: Scale and location relaxation • Sokgraph patterns, not individual letters crossed, are recognized and entered • Lower visual attention demand from tapping

  22. Principle 3: Duality tapping/tracing to gesturing • (Novice) User’s choice • Tapping and tracing as a bridge to shorthand gesturing. • Same trajectory pattern.

  23. Principle 4: Zipf’s law and common word components • A small number of words make disproportional percent of text • Common components e.g. -tion, -ing • Benefits early

  24. Principle 5 – Skill transition • Consistent movement patterns between tapping/tracing and gesturing • Visually guided action to recall based action • Gradual shift: closed-loop to open-loop • Falling back and relearning

  25. Related Work • Artificial alphabets • Unistrokes (Goldberg & Richardson 1993) • Graffiti (Blickenstorfer 1995) • Quikwriting (Perlin 1998) • Cirrin (Mankoff & Abowd 1998) • Dasher (Ward, Blackwell, Mackay 2000) • Marking menus (Kurtenbach & Buxton 1993) • T-Cube (Venolia & Neiberg 1994)

  26. Shark Gesture Recognition • Gesture recognition • sampling • filtering • normalization • matching against prototypes • Many shape matching algorithms • complexity – scalability • accuracy • cognitive, perceptive, motoric factors • Currently elastic matching

  27. Elastic Matching (Tappert 1982) • Measuring curve to curve distance • Minimizing average distance by finding closest corresponding points • Dynamic programming

  28. Live demo

  29. Many issues • Most compelling • Can people learn, remember, produce recognizable SHARK gestures at all? • Are SHARK gesture too arbitrary? • Is SHARK really feasible?

  30. Zhai, Kristensson, CHI 2003 A “Feasibility” Experiment

  31. Results: number of words learned per session

  32. Study conclusions • SHARK gestures can be learned • About 15 words per hour • About 60 words learned in 4 hours – already very useful (40% BNC)

  33. More research questions • Robust sokgraph recognition algorithms are being developed • Intimate human-machine interaction • Visual attention • Learning, skill acquisition • How people perceive, remember, produce gestures (e.g. topological vs. proportional)? • Speed accuracy trade-off • How fast people can do gestures? • How “sloppy” people get? • What is “reasonable”? • How do user computer “negotiate”? • Information quantification and modeling • Theory!

  34. D D W W Snapshot of other research programs ― Laws of action • Law of Pointing (Fitts’ law) • t = f (D/W) (Fitts, 1954) • Pointing with amplitude and directional constraint (Accot & Zhai, CHI 2003) • Two types of speed-accuracy tradeoff (Zhai 2004) • Law of Crossing • More than dotting the i’s (Accot & Zhai, CHI’02) • Law of Steering • Beyond Fitts’ law (Drury 1975, Accot& Zhai CHI’97) • VR locomotion (Zhai, Waltjer, IEEE VR 2003 best paper) • More “laws” needed

  35. Snapshot of other research programs ― eye gaze sensing based interaction • Hand-Eye coordinated action ― MAGIC pointing (Zhai, Morimoto, Ihde CHI’99; Zhai CACM 2003) • EASE Chinese input (Wang, Zhai, Su, CHI’01)

  36. Thank you and questions

  37. xW W W Varying Key Sizes • Fitts’ law • log(D/W + 1) • Central location effect • Asymmetry • Packing • Varying control precision Combined time Time from left to right key Time from right to left key

  38. 35 30 25 WPM 20 15 10 test 0 test 2 test 4 test 6 test 8 test 10 test 1 test 3 test 5 test 7 test 9 Zhai, Sue, Accot, CHI 2002 Learning • ERI (Expanding rehearsal interval)