in search of text writing methods for off the desktop computing atomik and shark l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK PowerPoint Presentation
Download Presentation
In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK

Loading in 2 Seconds...

play fullscreen
1 / 38

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK - PowerPoint PPT Presentation


  • 300 Views
  • Uploaded on

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK Shumin Zhai In collaboration with Barton Smith, Per-Ola Kristensson ( Linkoping U ), Alison Sue, Clemens Drews, Paul Lee ( Stanford ), Johnny Accot, Michael Hunter ( BYU ), Jingtao Wang ( Berkeley )

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
in search of text writing methods for off the desktop computing atomik and shark

In Search of Text Writing Methods for Off the Desktop Computing ― ATOMIK and SHARK

Shumin Zhai

In collaboration with

Barton Smith, Per-Ola Kristensson (LinkopingU), Alison Sue, Clemens Drews, Paul Lee (Stanford), Johnny Accot, Michael Hunter (BYU), Jingtao Wang (Berkeley)

IBM Almaden Research Center

San Jose, CA

slide2

Computing off the desktop

  • Desktop computing “workstation” interface foundation
    • Large and personal display
    • Input device (mouse)
    • Typewriter keyboard
  • HCI Frontier – beyond the desktop
    • Interfaces without display-mouse-keyboard tripod
    • Numerous difficult challenges
the text input challenge
The text input challenge
  • Indispensable user task
  • Efficiency
  • Learning
  • Size / portability
  • Visual cognitive attention
  • “History” of writing technology
text entry methods
Text Entry Methods
  • Reduced keyboard
    • T9, miniature keyboard
  • Hand writing
    • English, Unistroke, Graffiti
  • Speech
    • Human factors limitation
  • Stylus (graphical) keyboards
the qwerty keyboard
The QWERTY Keyboard
  • Invented by Sholes, Glidden, and Soule in1868 ― minimizing mechanical jamming
  • QWERTYnomics (P. David vs. Liebowitz & Margolis)
  • Touch typing ― low visual attention demand
  • Happen to be good for two hands alternation ― Dvorak did not prevail
fitts law

Wj

Key ii

Key j

Dij

Fitts’ law

For stylus keyboard — a = 0.08 sec, b = 0.127 sec/bit (Zhai, Su, Accot, CHI 2002)

letter transition frequency digraph
Letter Transition Frequency (Digraph)
  • Mayzner and Tresselt (1965)
  • British National Corpus (BNC)
  • 2 new modern corpora
    • News - NY Time, SJ Mercury, LA Times
    • Chat room logs
movement efficiency model of stylus keyboards

34.2 WPM

Movement Efficiency Model of Stylus Keyboards

(Soukoreff & MacKenzie,1995; Zhai, Sue & Accot 2002)

manual explorations
Manual explorations

OPTI, MacKenzie & Zhang

(42.8 wpm)

FITALY keyboard

(41.2 wpm)

metropolis method

Fitts-digraph “energy”

  • “Random walk”

Zhai, Hunter & Smith, HCI 2002

Metropolis Method
  • UI physics - Keyboard as a “molecule”
  • Annealing – varying T
word connectivity
Word connectivity
  • Zipf’s law Pi ~ 1/ia
  • connectivity Index
slide15

18000

16000

14000

Word connectivity

Human Movement Study: Fitts’ law

MT = a + b Log2(Dsi/Wi + 1)

12000

10000

8000

6000

4000

2000

0

sp

E

T

A

H

O

N

S

R

I

D

L

U

W

M

C

G

Y

F

B

P

K

V

J

X

Q

Z

English Letter Corpus(News, chat etc)

“Fitts-digraph energy”

Metropolis “random walk” optimization

Alphabetical tuning

Alphabetically Tuned and Optimized Mobile Interface Keyboard

(ATOMIK)

limitations and hints from atomik
Limitations and hints from ATOMIK
  • Tapping one key at a time – tedious. The stylus can be more expressive and dexterous.
  • Does not utilize language redundancy/statistical intelligence.
  • People tend to remember the pattern of a whole word, not individual letters.
the new phase shark

“word”

Zhai, Kristensson, CHI 2003

The new phase - SHARK

The basic idea: gesturing the word pattern defined by the keyboard

principle 1 efficiency
Principle 1 - efficiency

“Writing” one word at a time (not letters)

principle 2 scale and location relaxation
Principle 2: Scale and location relaxation
  • Sokgraph patterns, not individual letters crossed, are recognized and entered
  • Lower visual attention demand from tapping
principle 3 duality tapping tracing to gesturing
Principle 3: Duality tapping/tracing to gesturing
  • (Novice) User’s choice
  • Tapping and tracing as a bridge to shorthand gesturing.
  • Same trajectory pattern.
principle 4 zipf s law and common word components
Principle 4: Zipf’s law and common word components
  • A small number of words make disproportional percent of text
  • Common components e.g. -tion, -ing
  • Benefits early
principle 5 skill transition
Principle 5 – Skill transition
  • Consistent movement patterns between tapping/tracing and gesturing
  • Visually guided action to recall based action
  • Gradual shift: closed-loop to open-loop
  • Falling back and relearning
related work
Related Work
  • Artificial alphabets
    • Unistrokes (Goldberg & Richardson 1993)
    • Graffiti (Blickenstorfer 1995)
  • Quikwriting (Perlin 1998)
  • Cirrin (Mankoff & Abowd 1998)
  • Dasher (Ward, Blackwell, Mackay 2000)
  • Marking menus (Kurtenbach & Buxton 1993)
  • T-Cube (Venolia & Neiberg 1994)
shark gesture recognition
Shark Gesture Recognition
  • Gesture recognition
    • sampling
    • filtering
    • normalization
    • matching against prototypes
  • Many shape matching algorithms
    • complexity – scalability
    • accuracy
    • cognitive, perceptive, motoric factors
  • Currently elastic matching
elastic matching tappert 1982
Elastic Matching (Tappert 1982)
  • Measuring curve to curve distance
  • Minimizing average distance by finding closest corresponding points
  • Dynamic programming
many issues
Many issues
  • Most compelling
    • Can people learn, remember, produce recognizable SHARK gestures at all?
    • Are SHARK gesture too arbitrary?
    • Is SHARK really feasible?
study conclusions
Study conclusions
  • SHARK gestures can be learned
  • About 15 words per hour
  • About 60 words learned in 4 hours – already very useful (40% BNC)
more research questions
More research questions
  • Robust sokgraph recognition algorithms are being developed
  • Intimate human-machine interaction
  • Visual attention
  • Learning, skill acquisition
  • How people perceive, remember, produce gestures (e.g. topological vs. proportional)?
  • Speed accuracy trade-off
    • How fast people can do gestures?
    • How “sloppy” people get?
    • What is “reasonable”?
    • How do user computer “negotiate”?
  • Information quantification and modeling
  • Theory!
snapshot of other research programs laws of action

D

D

W

W

Snapshot of other research programs ― Laws of action
  • Law of Pointing (Fitts’ law)
    • t = f (D/W) (Fitts, 1954)
    • Pointing with amplitude and directional constraint (Accot & Zhai, CHI 2003)
    • Two types of speed-accuracy tradeoff (Zhai 2004)
  • Law of Crossing
    • More than dotting the i’s (Accot & Zhai, CHI’02)
  • Law of Steering
    • Beyond Fitts’ law (Drury 1975, Accot& Zhai CHI’97)
    • VR locomotion (Zhai, Waltjer, IEEE VR 2003 best paper)
  • More “laws” needed
snapshot of other research programs eye gaze sensing based interaction
Snapshot of other research programs ― eye gaze sensing based interaction
  • Hand-Eye coordinated action ― MAGIC pointing (Zhai, Morimoto, Ihde CHI’99; Zhai CACM 2003)
  • EASE Chinese input (Wang, Zhai, Su, CHI’01)
varying key sizes

xW

W

W

Varying Key Sizes
  • Fitts’ law
    • log(D/W + 1)
  • Central location effect
  • Asymmetry
  • Packing
  • Varying control precision

Combined time

Time from left to right key

Time from right to left key

learning

35

30

25

WPM

20

15

10

test 0

test 2

test 4

test 6

test 8

test 10

test 1

test 3

test 5

test 7

test 9

Zhai, Sue, Accot, CHI 2002

Learning
  • ERI (Expanding rehearsal interval)