In search of text writing methods for off the desktop computing atomik and shark
Download
1 / 38

Microsoft Powerpoint - PowerPoint PPT Presentation


  • 285 Views
  • Updated On :

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK Shumin Zhai In collaboration with Barton Smith, Per-Ola Kristensson ( Linkoping U ), Alison Sue, Clemens Drews, Paul Lee ( Stanford ), Johnny Accot, Michael Hunter ( BYU ), Jingtao Wang ( Berkeley )

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Microsoft Powerpoint' - johana


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
In search of text writing methods for off the desktop computing atomik and shark l.jpg

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK

Shumin Zhai

In collaboration with

Barton Smith, Per-Ola Kristensson (LinkopingU), Alison Sue, Clemens Drews, Paul Lee (Stanford), Johnny Accot, Michael Hunter (BYU), Jingtao Wang (Berkeley)

IBM Almaden Research Center

San Jose, CA


Slide2 l.jpg

Computing Computing off the desktop

  • Desktop computing “workstation” interface foundation

    • Large and personal display

    • Input device (mouse)

    • Typewriter keyboard

  • HCI Frontier – beyond the desktop

    • Interfaces without display-mouse-keyboard tripod

    • Numerous difficult challenges


The text input challenge l.jpg
The text input challenge Computing

  • Indispensable user task

  • Efficiency

  • Learning

  • Size / portability

  • Visual cognitive attention

  • “History” of writing technology


Text entry methods l.jpg
Text Entry Methods Computing

  • Reduced keyboard

    • T9, miniature keyboard

  • Hand writing

    • English, Unistroke, Graffiti

  • Speech

    • Human factors limitation

  • Stylus (graphical) keyboards


The qwerty keyboard l.jpg
The QWERTY Computing Keyboard

  • Invented by Sholes, Glidden, and Soule in1868 ― minimizing mechanical jamming

  • QWERTYnomics (P. David vs. Liebowitz & Margolis)

  • Touch typing ― low visual attention demand

  • Happen to be good for two hands alternation ― Dvorak did not prevail


Fitts law l.jpg

W Computing j

Key ii

Key j

Dij

Fitts’ law

For stylus keyboard — a = 0.08 sec, b = 0.127 sec/bit (Zhai, Su, Accot, CHI 2002)


Letter transition frequency digraph l.jpg
Letter Transition Frequency (Digraph) Computing

  • Mayzner and Tresselt (1965)

  • British National Corpus (BNC)

  • 2 new modern corpora

    • News - NY Time, SJ Mercury, LA Times

    • Chat room logs


Movement efficiency model of stylus keyboards l.jpg

34.2 Computing WPM

Movement Efficiency Model of Stylus Keyboards

(Soukoreff & MacKenzie,1995; Zhai, Sue & Accot 2002)


Manual explorations l.jpg
Manual explorations Computing

OPTI, MacKenzie & Zhang

(42.8 wpm)

FITALY keyboard

(41.2 wpm)


Algorithmic design dynamic simulation l.jpg

Zhai, Hunter, Smith, UIST2000 Computing

Algorithmic design - dynamic simulation

Hooke’s Keyboard (45.1 wpm)


Metropolis method l.jpg

  • “Random walk”

Zhai, Hunter & Smith, HCI 2002

Metropolis Method

  • UI physics - Keyboard as a “molecule”

  • Annealing – varying T



Alphabetical tuning for novice users l.jpg

30% smaller search area by Hick’s law analysis Computing

Smith & Zhai INTERACT2001

Alphabetical “tuning” for novice users

Novice user taping speed (wpm)


Word connectivity l.jpg
Word connectivity Computing

  • Zipf’s law Pi ~ 1/ia

  • connectivity Index


Slide15 l.jpg

18000 Computing

16000

14000

Word connectivity

Human Movement Study: Fitts’ law

MT = a + b Log2(Dsi/Wi + 1)

12000

10000

8000

6000

4000

2000

0

sp

E

T

A

H

O

N

S

R

I

D

L

U

W

M

C

G

Y

F

B

P

K

V

J

X

Q

Z

English Letter Corpus(News, chat etc)

“Fitts-digraph energy”

Metropolis “random walk” optimization

Alphabetical tuning

Alphabetically Tuned and Optimized Mobile Interface Keyboard

(ATOMIK)


Limitations and hints from atomik l.jpg
Limitations and hints from ATOMIK Computing

  • Tapping one key at a time – tedious. The stylus can be more expressive and dexterous.

  • Does not utilize language redundancy/statistical intelligence.

  • People tend to remember the pattern of a whole word, not individual letters.


The new phase shark l.jpg

“word” Computing

Zhai, Kristensson, CHI 2003

The new phase - SHARK

The basic idea: gesturing the word pattern defined by the keyboard


Sample sokgraphs s horthand o n k eyboard l.jpg

Shorthand Aided Rapid Computing Keyboarding ― SHARK

Sample “sokgraphs” (Shorthand On Keyboard)


Principle 1 efficiency l.jpg
Principle 1 - efficiency Computing

“Writing” one word at a time (not letters)


A form of shorthand l.jpg
A form of Computing shorthand


Principle 2 scale and location relaxation l.jpg
Principle 2: Scale and location relaxation Computing

  • Sokgraph patterns, not individual letters crossed, are recognized and entered

  • Lower visual attention demand from tapping


Principle 3 duality tapping tracing to gesturing l.jpg
Principle 3: Duality tapping/tracing to gesturing Computing

  • (Novice) User’s choice

  • Tapping and tracing as a bridge to shorthand gesturing.

  • Same trajectory pattern.


Principle 4 zipf s law and common word components l.jpg
Principle 4: Zipf’s law and Computing common word components

  • A small number of words make disproportional percent of text

  • Common components e.g. -tion, -ing

  • Benefits early


Principle 5 skill transition l.jpg
Principle 5 – Skill transition Computing

  • Consistent movement patterns between tapping/tracing and gesturing

  • Visually guided action to recall based action

  • Gradual shift: closed-loop to open-loop

  • Falling back and relearning


Related work l.jpg
Related Work Computing

  • Artificial alphabets

    • Unistrokes (Goldberg & Richardson 1993)

    • Graffiti (Blickenstorfer 1995)

  • Quikwriting (Perlin 1998)

  • Cirrin (Mankoff & Abowd 1998)

  • Dasher (Ward, Blackwell, Mackay 2000)

  • Marking menus (Kurtenbach & Buxton 1993)

  • T-Cube (Venolia & Neiberg 1994)


Shark gesture recognition l.jpg
Shark Gesture Recognition Computing

  • Gesture recognition

    • sampling

    • filtering

    • normalization

    • matching against prototypes

  • Many shape matching algorithms

    • complexity – scalability

    • accuracy

    • cognitive, perceptive, motoric factors

  • Currently elastic matching


Elastic matching tappert 1982 l.jpg
Elastic Matching (Tappert 1982) Computing

  • Measuring curve to curve distance

  • Minimizing average distance by finding closest corresponding points

  • Dynamic programming


Live demo l.jpg
Live demo Computing


Many issues l.jpg
Many issues Computing

  • Most compelling

    • Can people learn, remember, produce recognizable SHARK gestures at all?

    • Are SHARK gesture too arbitrary?

    • Is SHARK really feasible?


A feasibility experiment l.jpg

Zhai, Kristensson, CHI 2003 Computing

A “Feasibility” Experiment


Results number of words learned per session l.jpg
Results: Computing number of words learned per session


Study conclusions l.jpg
Study conclusions Computing

  • SHARK gestures can be learned

  • About 15 words per hour

  • About 60 words learned in 4 hours – already very useful (40% BNC)


More research questions l.jpg
More research questions Computing

  • Robust sokgraph recognition algorithms are being developed

  • Intimate human-machine interaction

  • Visual attention

  • Learning, skill acquisition

  • How people perceive, remember, produce gestures (e.g. topological vs. proportional)?

  • Speed accuracy trade-off

    • How fast people can do gestures?

    • How “sloppy” people get?

    • What is “reasonable”?

    • How do user computer “negotiate”?

  • Information quantification and modeling

  • Theory!


Snapshot of other research programs laws of action l.jpg

D Computing

D

W

W

Snapshot of other research programs ― Laws of action

  • Law of Pointing (Fitts’ law)

    • t = f (D/W) (Fitts, 1954)

    • Pointing with amplitude and directional constraint (Accot & Zhai, CHI 2003)

    • Two types of speed-accuracy tradeoff (Zhai 2004)

  • Law of Crossing

    • More than dotting the i’s (Accot & Zhai, CHI’02)

  • Law of Steering

    • Beyond Fitts’ law (Drury 1975, Accot& Zhai CHI’97)

    • VR locomotion (Zhai, Waltjer, IEEE VR 2003 best paper)

  • More “laws” needed


Snapshot of other research programs eye gaze sensing based interaction l.jpg
Snapshot of other research programs Computing ― eye gaze sensing based interaction

  • Hand-Eye coordinated action ― MAGIC pointing (Zhai, Morimoto, Ihde CHI’99; Zhai CACM 2003)

  • EASE Chinese input (Wang, Zhai, Su, CHI’01)



Varying key sizes l.jpg

xW Computing

W

W

Varying Key Sizes

  • Fitts’ law

    • log(D/W + 1)

  • Central location effect

  • Asymmetry

  • Packing

  • Varying control precision

Combined time

Time from left to right key

Time from right to left key


Learning l.jpg

35 Computing

30

25

WPM

20

15

10

test 0

test 2

test 4

test 6

test 8

test 10

test 1

test 3

test 5

test 7

test 9

Zhai, Sue, Accot, CHI 2002

Learning

  • ERI (Expanding rehearsal interval)


ad