In search of text writing methods for off the desktop computing atomik and shark
1 / 38

In Search of Text Writing Methods for Off the Desktop Computing ? ATOMIK and SHARK - PowerPoint PPT Presentation

  • Updated On :

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK Shumin Zhai In collaboration with Barton Smith, Per-Ola Kristensson ( Linkoping U ), Alison Sue, Clemens Drews, Paul Lee ( Stanford ), Johnny Accot, Michael Hunter ( BYU ), Jingtao Wang ( Berkeley )

Related searches for In Search of Text Writing Methods for Off the Desktop Computing ? ATOMIK and SHARK

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'In Search of Text Writing Methods for Off the Desktop Computing ? ATOMIK and SHARK' - johana

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
In search of text writing methods for off the desktop computing atomik and shark l.jpg

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK

Shumin Zhai

In collaboration with

Barton Smith, Per-Ola Kristensson (LinkopingU), Alison Sue, Clemens Drews, Paul Lee (Stanford), Johnny Accot, Michael Hunter (BYU), Jingtao Wang (Berkeley)

IBM Almaden Research Center

San Jose, CA

Slide2 l.jpg

Computing Computing off the desktop

  • Desktop computing “workstation” interface foundation

    • Large and personal display

    • Input device (mouse)

    • Typewriter keyboard

  • HCI Frontier – beyond the desktop

    • Interfaces without display-mouse-keyboard tripod

    • Numerous difficult challenges

The text input challenge l.jpg
The text input challenge Computing

  • Indispensable user task

  • Efficiency

  • Learning

  • Size / portability

  • Visual cognitive attention

  • “History” of writing technology

Text entry methods l.jpg
Text Entry Methods Computing

  • Reduced keyboard

    • T9, miniature keyboard

  • Hand writing

    • English, Unistroke, Graffiti

  • Speech

    • Human factors limitation

  • Stylus (graphical) keyboards

The qwerty keyboard l.jpg
The QWERTY Computing Keyboard

  • Invented by Sholes, Glidden, and Soule in1868 ― minimizing mechanical jamming

  • QWERTYnomics (P. David vs. Liebowitz & Margolis)

  • Touch typing ― low visual attention demand

  • Happen to be good for two hands alternation ― Dvorak did not prevail

Fitts law l.jpg

W Computing j

Key ii

Key j


Fitts’ law

For stylus keyboard — a = 0.08 sec, b = 0.127 sec/bit (Zhai, Su, Accot, CHI 2002)

Letter transition frequency digraph l.jpg
Letter Transition Frequency (Digraph) Computing

  • Mayzner and Tresselt (1965)

  • British National Corpus (BNC)

  • 2 new modern corpora

    • News - NY Time, SJ Mercury, LA Times

    • Chat room logs

Movement efficiency model of stylus keyboards l.jpg

34.2 Computing WPM

Movement Efficiency Model of Stylus Keyboards

(Soukoreff & MacKenzie,1995; Zhai, Sue & Accot 2002)

Manual explorations l.jpg
Manual explorations Computing

OPTI, MacKenzie & Zhang

(42.8 wpm)

FITALY keyboard

(41.2 wpm)

Algorithmic design dynamic simulation l.jpg

Zhai, Hunter, Smith, UIST2000 Computing

Algorithmic design - dynamic simulation

Hooke’s Keyboard (45.1 wpm)

Metropolis method l.jpg

  • “Random walk”

Zhai, Hunter & Smith, HCI 2002

Metropolis Method

  • UI physics - Keyboard as a “molecule”

  • Annealing – varying T

Alphabetical tuning for novice users l.jpg

30% smaller search area by Hick’s law analysis Computing

Smith & Zhai INTERACT2001

Alphabetical “tuning” for novice users

Novice user taping speed (wpm)

Word connectivity l.jpg
Word connectivity Computing

  • Zipf’s law Pi ~ 1/ia

  • connectivity Index

Slide15 l.jpg

18000 Computing



Word connectivity

Human Movement Study: Fitts’ law

MT = a + b Log2(Dsi/Wi + 1)



































English Letter Corpus(News, chat etc)

“Fitts-digraph energy”

Metropolis “random walk” optimization

Alphabetical tuning

Alphabetically Tuned and Optimized Mobile Interface Keyboard


Limitations and hints from atomik l.jpg
Limitations and hints from ATOMIK Computing

  • Tapping one key at a time – tedious. The stylus can be more expressive and dexterous.

  • Does not utilize language redundancy/statistical intelligence.

  • People tend to remember the pattern of a whole word, not individual letters.

The new phase shark l.jpg

“word” Computing

Zhai, Kristensson, CHI 2003

The new phase - SHARK

The basic idea: gesturing the word pattern defined by the keyboard

Sample sokgraphs s horthand o n k eyboard l.jpg

Shorthand Aided Rapid Computing Keyboarding ― SHARK

Sample “sokgraphs” (Shorthand On Keyboard)

Principle 1 efficiency l.jpg
Principle 1 - efficiency Computing

“Writing” one word at a time (not letters)

A form of shorthand l.jpg
A form of Computing shorthand

Principle 2 scale and location relaxation l.jpg
Principle 2: Scale and location relaxation Computing

  • Sokgraph patterns, not individual letters crossed, are recognized and entered

  • Lower visual attention demand from tapping

Principle 3 duality tapping tracing to gesturing l.jpg
Principle 3: Duality tapping/tracing to gesturing Computing

  • (Novice) User’s choice

  • Tapping and tracing as a bridge to shorthand gesturing.

  • Same trajectory pattern.

Principle 4 zipf s law and common word components l.jpg
Principle 4: Zipf’s law and Computing common word components

  • A small number of words make disproportional percent of text

  • Common components e.g. -tion, -ing

  • Benefits early

Principle 5 skill transition l.jpg
Principle 5 – Skill transition Computing

  • Consistent movement patterns between tapping/tracing and gesturing

  • Visually guided action to recall based action

  • Gradual shift: closed-loop to open-loop

  • Falling back and relearning

Related work l.jpg
Related Work Computing

  • Artificial alphabets

    • Unistrokes (Goldberg & Richardson 1993)

    • Graffiti (Blickenstorfer 1995)

  • Quikwriting (Perlin 1998)

  • Cirrin (Mankoff & Abowd 1998)

  • Dasher (Ward, Blackwell, Mackay 2000)

  • Marking menus (Kurtenbach & Buxton 1993)

  • T-Cube (Venolia & Neiberg 1994)

Shark gesture recognition l.jpg
Shark Gesture Recognition Computing

  • Gesture recognition

    • sampling

    • filtering

    • normalization

    • matching against prototypes

  • Many shape matching algorithms

    • complexity – scalability

    • accuracy

    • cognitive, perceptive, motoric factors

  • Currently elastic matching

Elastic matching tappert 1982 l.jpg
Elastic Matching (Tappert 1982) Computing

  • Measuring curve to curve distance

  • Minimizing average distance by finding closest corresponding points

  • Dynamic programming

Live demo l.jpg
Live demo Computing

Many issues l.jpg
Many issues Computing

  • Most compelling

    • Can people learn, remember, produce recognizable SHARK gestures at all?

    • Are SHARK gesture too arbitrary?

    • Is SHARK really feasible?

A feasibility experiment l.jpg

Zhai, Kristensson, CHI 2003 Computing

A “Feasibility” Experiment

Results number of words learned per session l.jpg
Results: Computing number of words learned per session

Study conclusions l.jpg
Study conclusions Computing

  • SHARK gestures can be learned

  • About 15 words per hour

  • About 60 words learned in 4 hours – already very useful (40% BNC)

More research questions l.jpg
More research questions Computing

  • Robust sokgraph recognition algorithms are being developed

  • Intimate human-machine interaction

  • Visual attention

  • Learning, skill acquisition

  • How people perceive, remember, produce gestures (e.g. topological vs. proportional)?

  • Speed accuracy trade-off

    • How fast people can do gestures?

    • How “sloppy” people get?

    • What is “reasonable”?

    • How do user computer “negotiate”?

  • Information quantification and modeling

  • Theory!

Snapshot of other research programs laws of action l.jpg

D Computing




Snapshot of other research programs ― Laws of action

  • Law of Pointing (Fitts’ law)

    • t = f (D/W) (Fitts, 1954)

    • Pointing with amplitude and directional constraint (Accot & Zhai, CHI 2003)

    • Two types of speed-accuracy tradeoff (Zhai 2004)

  • Law of Crossing

    • More than dotting the i’s (Accot & Zhai, CHI’02)

  • Law of Steering

    • Beyond Fitts’ law (Drury 1975, Accot& Zhai CHI’97)

    • VR locomotion (Zhai, Waltjer, IEEE VR 2003 best paper)

  • More “laws” needed

Snapshot of other research programs eye gaze sensing based interaction l.jpg
Snapshot of other research programs Computing ― eye gaze sensing based interaction

  • Hand-Eye coordinated action ― MAGIC pointing (Zhai, Morimoto, Ihde CHI’99; Zhai CACM 2003)

  • EASE Chinese input (Wang, Zhai, Su, CHI’01)

Varying key sizes l.jpg

xW Computing



Varying Key Sizes

  • Fitts’ law

    • log(D/W + 1)

  • Central location effect

  • Asymmetry

  • Packing

  • Varying control precision

Combined time

Time from left to right key

Time from right to left key

Learning l.jpg

35 Computing







test 0

test 2

test 4

test 6

test 8

test 10

test 1

test 3

test 5

test 7

test 9

Zhai, Sue, Accot, CHI 2002


  • ERI (Expanding rehearsal interval)