In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK - PowerPoint PPT Presentation

In search of text writing methods for off the desktop computing atomik and shark l.jpg
Download
1 / 38

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK Shumin Zhai In collaboration with Barton Smith, Per-Ola Kristensson ( Linkoping U ), Alison Sue, Clemens Drews, Paul Lee ( Stanford ), Johnny Accot, Michael Hunter ( BYU ), Jingtao Wang ( Berkeley )

Related searches for In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


In search of text writing methods for off the desktop computing atomik and shark l.jpg

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK

Shumin Zhai

In collaboration with

Barton Smith, Per-Ola Kristensson (LinkopingU), Alison Sue, Clemens Drews, Paul Lee (Stanford), Johnny Accot, Michael Hunter (BYU), Jingtao Wang (Berkeley)

IBM Almaden Research Center

San Jose, CA


Slide2 l.jpg

Computing off the desktop

  • Desktop computing “workstation” interface foundation

    • Large and personal display

    • Input device (mouse)

    • Typewriter keyboard

  • HCI Frontier – beyond the desktop

    • Interfaces without display-mouse-keyboard tripod

    • Numerous difficult challenges


The text input challenge l.jpg

The text input challenge

  • Indispensable user task

  • Efficiency

  • Learning

  • Size / portability

  • Visual cognitive attention

  • “History” of writing technology


Text entry methods l.jpg

Text Entry Methods

  • Reduced keyboard

    • T9, miniature keyboard

  • Hand writing

    • English, Unistroke, Graffiti

  • Speech

    • Human factors limitation

  • Stylus (graphical) keyboards


The qwerty keyboard l.jpg

The QWERTY Keyboard

  • Invented by Sholes, Glidden, and Soule in1868 ― minimizing mechanical jamming

  • QWERTYnomics (P. David vs. Liebowitz & Margolis)

  • Touch typing ― low visual attention demand

  • Happen to be good for two hands alternation ― Dvorak did not prevail


Fitts law l.jpg

Wj

Key ii

Key j

Dij

Fitts’ law

For stylus keyboard — a = 0.08 sec, b = 0.127 sec/bit (Zhai, Su, Accot, CHI 2002)


Letter transition frequency digraph l.jpg

Letter Transition Frequency (Digraph)

  • Mayzner and Tresselt (1965)

  • British National Corpus (BNC)

  • 2 new modern corpora

    • News - NY Time, SJ Mercury, LA Times

    • Chat room logs


Movement efficiency model of stylus keyboards l.jpg

34.2 WPM

Movement Efficiency Model of Stylus Keyboards

(Soukoreff & MacKenzie,1995; Zhai, Sue & Accot 2002)


Manual explorations l.jpg

Manual explorations

OPTI, MacKenzie & Zhang

(42.8 wpm)

FITALY keyboard

(41.2 wpm)


Algorithmic design dynamic simulation l.jpg

Zhai, Hunter, Smith, UIST2000

Algorithmic design - dynamic simulation

Hooke’s Keyboard (45.1 wpm)


Metropolis method l.jpg

  • Fitts-digraph “energy”

  • “Random walk”

Zhai, Hunter & Smith, HCI 2002

Metropolis Method

  • UI physics - Keyboard as a “molecule”

  • Annealing – varying T


Slide12 l.jpg

46.6 wpm – 36% more efficient than QWERTY


Alphabetical tuning for novice users l.jpg

30% smaller search area by Hick’s law analysis

Smith & Zhai INTERACT2001

Alphabetical “tuning” for novice users

Novice user taping speed (wpm)


Word connectivity l.jpg

Word connectivity

  • Zipf’s law Pi ~ 1/ia

  • connectivity Index


Slide15 l.jpg

18000

16000

14000

Word connectivity

Human Movement Study: Fitts’ law

MT = a + b Log2(Dsi/Wi + 1)

12000

10000

8000

6000

4000

2000

0

sp

E

T

A

H

O

N

S

R

I

D

L

U

W

M

C

G

Y

F

B

P

K

V

J

X

Q

Z

English Letter Corpus(News, chat etc)

“Fitts-digraph energy”

Metropolis “random walk” optimization

Alphabetical tuning

Alphabetically Tuned and Optimized Mobile Interface Keyboard

(ATOMIK)


Limitations and hints from atomik l.jpg

Limitations and hints from ATOMIK

  • Tapping one key at a time – tedious. The stylus can be more expressive and dexterous.

  • Does not utilize language redundancy/statistical intelligence.

  • People tend to remember the pattern of a whole word, not individual letters.


The new phase shark l.jpg

“word”

Zhai, Kristensson, CHI 2003

The new phase - SHARK

The basic idea: gesturing the word pattern defined by the keyboard


Sample sokgraphs s horthand o n k eyboard l.jpg

Shorthand Aided Rapid Keyboarding ― SHARK

Sample “sokgraphs” (Shorthand On Keyboard)


Principle 1 efficiency l.jpg

Principle 1 - efficiency

“Writing” one word at a time (not letters)


A form of shorthand l.jpg

A form of shorthand


Principle 2 scale and location relaxation l.jpg

Principle 2: Scale and location relaxation

  • Sokgraph patterns, not individual letters crossed, are recognized and entered

  • Lower visual attention demand from tapping


Principle 3 duality tapping tracing to gesturing l.jpg

Principle 3: Duality tapping/tracing to gesturing

  • (Novice) User’s choice

  • Tapping and tracing as a bridge to shorthand gesturing.

  • Same trajectory pattern.


Principle 4 zipf s law and common word components l.jpg

Principle 4: Zipf’s law and common word components

  • A small number of words make disproportional percent of text

  • Common components e.g. -tion, -ing

  • Benefits early


Principle 5 skill transition l.jpg

Principle 5 – Skill transition

  • Consistent movement patterns between tapping/tracing and gesturing

  • Visually guided action to recall based action

  • Gradual shift: closed-loop to open-loop

  • Falling back and relearning


Related work l.jpg

Related Work

  • Artificial alphabets

    • Unistrokes (Goldberg & Richardson 1993)

    • Graffiti (Blickenstorfer 1995)

  • Quikwriting (Perlin 1998)

  • Cirrin (Mankoff & Abowd 1998)

  • Dasher (Ward, Blackwell, Mackay 2000)

  • Marking menus (Kurtenbach & Buxton 1993)

  • T-Cube (Venolia & Neiberg 1994)


Shark gesture recognition l.jpg

Shark Gesture Recognition

  • Gesture recognition

    • sampling

    • filtering

    • normalization

    • matching against prototypes

  • Many shape matching algorithms

    • complexity – scalability

    • accuracy

    • cognitive, perceptive, motoric factors

  • Currently elastic matching


Elastic matching tappert 1982 l.jpg

Elastic Matching (Tappert 1982)

  • Measuring curve to curve distance

  • Minimizing average distance by finding closest corresponding points

  • Dynamic programming


Live demo l.jpg

Live demo


Many issues l.jpg

Many issues

  • Most compelling

    • Can people learn, remember, produce recognizable SHARK gestures at all?

    • Are SHARK gesture too arbitrary?

    • Is SHARK really feasible?


A feasibility experiment l.jpg

Zhai, Kristensson, CHI 2003

A “Feasibility” Experiment


Results number of words learned per session l.jpg

Results: number of words learned per session


Study conclusions l.jpg

Study conclusions

  • SHARK gestures can be learned

  • About 15 words per hour

  • About 60 words learned in 4 hours – already very useful (40% BNC)


More research questions l.jpg

More research questions

  • Robust sokgraph recognition algorithms are being developed

  • Intimate human-machine interaction

  • Visual attention

  • Learning, skill acquisition

  • How people perceive, remember, produce gestures (e.g. topological vs. proportional)?

  • Speed accuracy trade-off

    • How fast people can do gestures?

    • How “sloppy” people get?

    • What is “reasonable”?

    • How do user computer “negotiate”?

  • Information quantification and modeling

  • Theory!


Snapshot of other research programs laws of action l.jpg

D

D

W

W

Snapshot of other research programs ― Laws of action

  • Law of Pointing (Fitts’ law)

    • t = f (D/W) (Fitts, 1954)

    • Pointing with amplitude and directional constraint (Accot & Zhai, CHI 2003)

    • Two types of speed-accuracy tradeoff (Zhai 2004)

  • Law of Crossing

    • More than dotting the i’s (Accot & Zhai, CHI’02)

  • Law of Steering

    • Beyond Fitts’ law (Drury 1975, Accot& Zhai CHI’97)

    • VR locomotion (Zhai, Waltjer, IEEE VR 2003 best paper)

  • More “laws” needed


Snapshot of other research programs eye gaze sensing based interaction l.jpg

Snapshot of other research programs ― eye gaze sensing based interaction

  • Hand-Eye coordinated action ― MAGIC pointing (Zhai, Morimoto, Ihde CHI’99; Zhai CACM 2003)

  • EASE Chinese input (Wang, Zhai, Su, CHI’01)


Thank you and questions l.jpg

Thank you and questions


Varying key sizes l.jpg

xW

W

W

Varying Key Sizes

  • Fitts’ law

    • log(D/W + 1)

  • Central location effect

  • Asymmetry

  • Packing

  • Varying control precision

Combined time

Time from left to right key

Time from right to left key


Learning l.jpg

35

30

25

WPM

20

15

10

test 0

test 2

test 4

test 6

test 8

test 10

test 1

test 3

test 5

test 7

test 9

Zhai, Sue, Accot, CHI 2002

Learning

  • ERI (Expanding rehearsal interval)


  • Login