in search of text writing methods for off the desktop computing atomik and shark
Download
Skip this Video
Download Presentation
In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK

Loading in 2 Seconds...

play fullscreen
1 / 38

Microsoft Powerpoint - PowerPoint PPT Presentation


  • 285 Views
  • Uploaded on

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK Shumin Zhai In collaboration with Barton Smith, Per-Ola Kristensson ( Linkoping U ), Alison Sue, Clemens Drews, Paul Lee ( Stanford ), Johnny Accot, Michael Hunter ( BYU ), Jingtao Wang ( Berkeley )

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Microsoft Powerpoint' - johana


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
in search of text writing methods for off the desktop computing atomik and shark

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK

Shumin Zhai

In collaboration with

Barton Smith, Per-Ola Kristensson (LinkopingU), Alison Sue, Clemens Drews, Paul Lee (Stanford), Johnny Accot, Michael Hunter (BYU), Jingtao Wang (Berkeley)

IBM Almaden Research Center

San Jose, CA

slide2

Computing off the desktop

  • Desktop computing “workstation” interface foundation
    • Large and personal display
    • Input device (mouse)
    • Typewriter keyboard
  • HCI Frontier – beyond the desktop
    • Interfaces without display-mouse-keyboard tripod
    • Numerous difficult challenges
the text input challenge
The text input challenge
  • Indispensable user task
  • Efficiency
  • Learning
  • Size / portability
  • Visual cognitive attention
  • “History” of writing technology
text entry methods
Text Entry Methods
  • Reduced keyboard
    • T9, miniature keyboard
  • Hand writing
    • English, Unistroke, Graffiti
  • Speech
    • Human factors limitation
  • Stylus (graphical) keyboards
the qwerty keyboard
The QWERTY Keyboard
  • Invented by Sholes, Glidden, and Soule in1868 ― minimizing mechanical jamming
  • QWERTYnomics (P. David vs. Liebowitz & Margolis)
  • Touch typing ― low visual attention demand
  • Happen to be good for two hands alternation ― Dvorak did not prevail
fitts law

Wj

Key ii

Key j

Dij

Fitts’ law

For stylus keyboard — a = 0.08 sec, b = 0.127 sec/bit (Zhai, Su, Accot, CHI 2002)

letter transition frequency digraph
Letter Transition Frequency (Digraph)
  • Mayzner and Tresselt (1965)
  • British National Corpus (BNC)
  • 2 new modern corpora
    • News - NY Time, SJ Mercury, LA Times
    • Chat room logs
movement efficiency model of stylus keyboards

34.2 WPM

Movement Efficiency Model of Stylus Keyboards

(Soukoreff & MacKenzie,1995; Zhai, Sue & Accot 2002)

manual explorations
Manual explorations

OPTI, MacKenzie & Zhang

(42.8 wpm)

FITALY keyboard

(41.2 wpm)

metropolis method

Fitts-digraph “energy”

  • “Random walk”

Zhai, Hunter & Smith, HCI 2002

Metropolis Method
  • UI physics - Keyboard as a “molecule”
  • Annealing – varying T
word connectivity
Word connectivity
  • Zipf’s law Pi ~ 1/ia
  • connectivity Index
slide15

18000

16000

14000

Word connectivity

Human Movement Study: Fitts’ law

MT = a + b Log2(Dsi/Wi + 1)

12000

10000

8000

6000

4000

2000

0

sp

E

T

A

H

O

N

S

R

I

D

L

U

W

M

C

G

Y

F

B

P

K

V

J

X

Q

Z

English Letter Corpus(News, chat etc)

“Fitts-digraph energy”

Metropolis “random walk” optimization

Alphabetical tuning

Alphabetically Tuned and Optimized Mobile Interface Keyboard

(ATOMIK)

limitations and hints from atomik
Limitations and hints from ATOMIK
  • Tapping one key at a time – tedious. The stylus can be more expressive and dexterous.
  • Does not utilize language redundancy/statistical intelligence.
  • People tend to remember the pattern of a whole word, not individual letters.
the new phase shark

“word”

Zhai, Kristensson, CHI 2003

The new phase - SHARK

The basic idea: gesturing the word pattern defined by the keyboard

principle 1 efficiency
Principle 1 - efficiency

“Writing” one word at a time (not letters)

principle 2 scale and location relaxation
Principle 2: Scale and location relaxation
  • Sokgraph patterns, not individual letters crossed, are recognized and entered
  • Lower visual attention demand from tapping
principle 3 duality tapping tracing to gesturing
Principle 3: Duality tapping/tracing to gesturing
  • (Novice) User’s choice
  • Tapping and tracing as a bridge to shorthand gesturing.
  • Same trajectory pattern.
principle 4 zipf s law and common word components
Principle 4: Zipf’s law and common word components
  • A small number of words make disproportional percent of text
  • Common components e.g. -tion, -ing
  • Benefits early
principle 5 skill transition
Principle 5 – Skill transition
  • Consistent movement patterns between tapping/tracing and gesturing
  • Visually guided action to recall based action
  • Gradual shift: closed-loop to open-loop
  • Falling back and relearning
related work
Related Work
  • Artificial alphabets
    • Unistrokes (Goldberg & Richardson 1993)
    • Graffiti (Blickenstorfer 1995)
  • Quikwriting (Perlin 1998)
  • Cirrin (Mankoff & Abowd 1998)
  • Dasher (Ward, Blackwell, Mackay 2000)
  • Marking menus (Kurtenbach & Buxton 1993)
  • T-Cube (Venolia & Neiberg 1994)
shark gesture recognition
Shark Gesture Recognition
  • Gesture recognition
    • sampling
    • filtering
    • normalization
    • matching against prototypes
  • Many shape matching algorithms
    • complexity – scalability
    • accuracy
    • cognitive, perceptive, motoric factors
  • Currently elastic matching
elastic matching tappert 1982
Elastic Matching (Tappert 1982)
  • Measuring curve to curve distance
  • Minimizing average distance by finding closest corresponding points
  • Dynamic programming
many issues
Many issues
  • Most compelling
    • Can people learn, remember, produce recognizable SHARK gestures at all?
    • Are SHARK gesture too arbitrary?
    • Is SHARK really feasible?
study conclusions
Study conclusions
  • SHARK gestures can be learned
  • About 15 words per hour
  • About 60 words learned in 4 hours – already very useful (40% BNC)
more research questions
More research questions
  • Robust sokgraph recognition algorithms are being developed
  • Intimate human-machine interaction
  • Visual attention
  • Learning, skill acquisition
  • How people perceive, remember, produce gestures (e.g. topological vs. proportional)?
  • Speed accuracy trade-off
    • How fast people can do gestures?
    • How “sloppy” people get?
    • What is “reasonable”?
    • How do user computer “negotiate”?
  • Information quantification and modeling
  • Theory!
snapshot of other research programs laws of action

D

D

W

W

Snapshot of other research programs ― Laws of action
  • Law of Pointing (Fitts’ law)
    • t = f (D/W) (Fitts, 1954)
    • Pointing with amplitude and directional constraint (Accot & Zhai, CHI 2003)
    • Two types of speed-accuracy tradeoff (Zhai 2004)
  • Law of Crossing
    • More than dotting the i’s (Accot & Zhai, CHI’02)
  • Law of Steering
    • Beyond Fitts’ law (Drury 1975, Accot& Zhai CHI’97)
    • VR locomotion (Zhai, Waltjer, IEEE VR 2003 best paper)
  • More “laws” needed
snapshot of other research programs eye gaze sensing based interaction
Snapshot of other research programs ― eye gaze sensing based interaction
  • Hand-Eye coordinated action ― MAGIC pointing (Zhai, Morimoto, Ihde CHI’99; Zhai CACM 2003)
  • EASE Chinese input (Wang, Zhai, Su, CHI’01)
varying key sizes

xW

W

W

Varying Key Sizes
  • Fitts’ law
    • log(D/W + 1)
  • Central location effect
  • Asymmetry
  • Packing
  • Varying control precision

Combined time

Time from left to right key

Time from right to left key

learning

35

30

25

WPM

20

15

10

test 0

test 2

test 4

test 6

test 8

test 10

test 1

test 3

test 5

test 7

test 9

Zhai, Sue, Accot, CHI 2002

Learning
  • ERI (Expanding rehearsal interval)
ad