Interaction gestural input popup widgets interaction techniques and hardware input devices
This presentation is the property of its rightful owner.
Sponsored Links
1 / 97

Interaction: gestural input, popup widgets, interaction techniques, and hardware input devices PowerPoint PPT Presentation


  • 61 Views
  • Uploaded on
  • Presentation posted in: General

Interaction: gestural input, popup widgets, interaction techniques, and hardware input devices. Prof. Michael McGuffin. Gestural Input. Gestures can be executed rapidly, and can be used as symbols to activate commands or select objects Sketches

Download Presentation

Interaction: gestural input, popup widgets, interaction techniques, and hardware input devices

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Interaction gestural input popup widgets interaction techniques and hardware input devices

Interaction: gestural input, popup widgets, interaction techniques, and hardware input devices

Prof. Michael McGuffin


Gestural input

Gestural Input

  • Gestures can be executed rapidly, and can be used as symbols to activate commands or select objects

  • Sketches

    • can be used to quickly enter text or diagrams, without requiring the user to switch from the pointing device to something else

    • Have an informal, loose, implicit structure

    • Can carry much more (implicit) information than typed text

  • Gestures as characters to recognize

    • Graffiti, Unistroke


Gestural input1

Gestural Input

Kurtenbach and Buxton (video)

  • Example gestures: circle objects, drag to move, or drag + draw a "C"  to copy

  • Such a gesture selects the noun (objects) and verb (operation) in a single action, which may be more natural, fluid, or faster than doing selection + Ctrl-C/Ctrl-X + Ctrl-V


Gestural input2

Gestural Input

"Teddy" (Igarashi et al. 1999).

See Java applet at Takeo Igarashi’s website.


Phun 2008

Phun (≈2008)

  • A free game/physical simulator that uses gestural input

  • http://www.acc.umu.se/~emilk/


Interaction gestural input popup widgets interaction techniques and hardware input devices

Phun


Interaction gestural input popup widgets interaction techniques and hardware input devices

  • Alvarado and Davis 2001

  • http://www.youtube.com/watch?v=NZNTgglPbUA


Interaction gestural input popup widgets interaction techniques and hardware input devices

  • LaViola 2007 http://doi.acm.org/10.1145/1281500.1281558


Observation

Observation

  • Most user interfaces that use gestural input require the user to first learn the gestures before they can be used. This can be a significant obstacle for novice users.

  • In the following slides, we will see a technique called Marking Menus that eliminates this problem.


Modes and some examples of interaction techniques using gestural input or other input

Modes, and some examples of interaction techniques using gestural input or other input


Interaction techniques

Interaction Techniques

  • “are ways to use input devices to enter information”

  • "are made up of single input-device actions.”

    (Foley, et al. “Computer Graphics: Principles and Practice”)


Interaction gestural input popup widgets interaction techniques and hardware input devices

Verbs(actions,commands,tools,operations)

Nouns(objects,locations)


Interaction gestural input popup widgets interaction techniques and hardware input devices

Verbs in a pulldown menu

Nouns(objects,locations)


Interaction gestural input popup widgets interaction techniques and hardware input devices

Verbs ina popup menu

Nouns(objects,locations)


Summary

Summary

  • Modes can lead to mode errors, where the user thinks they are in one mode but are really in another

  • Visual feedback (example: changing cursor shape) to indicate the current mode is good, but often not enough to avoid mode errors

  • Popup menus help…

    • Avoid mode errors(via transient modes, and kinaesthetic feedback where the user feels pressure through their finger from holding down a button)

    • Increase screen space devoted to content, because the menu is only displayed when desired

    • Decrease the distance travelled by the cursor, because there is no back-and-forth motion between the workspace and peripheral interface elements

    • Create a tighter integration, or fusion, of the selection of noun and verb, leading to better mental chunking (Buxton 1986)


Given that popup menus have all these advantages can we design other widgets that are even better

Given that popup menus have all these advantages, can we design other widgets that are even better?


Radial menu or pie menu

Radial Menu (or “Pie Menu”)


Demo radial menu in simplepaint

Demo: Radial Menu in SimplePaint


An interface that uses effectively radial menus

An interface that uses, effectively, radial menus

Yatani et al., CHI 2008


Radial menus versus linear menus

Radial menusversus linear menus


Radial menus versus linear menus1

Radial menusversus linear menus


Radial menus versus linear menus2

Radial menusversus linear menus


Radial menus versus linear menus3

Radial menusversus linear menus


Radial menus versus linear menus4

Radial menusversus linear menus


Radial menus versus linear menus5

Radial menusversus linear menus

Directions are easier to rememberand reproduce than distances


Hierarchical radial menu

Hierarchical Radial Menu


Mouse gestures for firefox

"Mouse Gestures" for Firefox


Marking menu

Marking Menu

  • “Scale invariant recognition”: The recognition of the gestures (marks) does not depend on the length of the segments; only the angles matter. Hence, the marks can be drawn small and quickly, or ballistically.

  • A user who knows what mark to draw doesn’t even need to see the menu popup.


The set of gestures is discoverable and self revealing contrary to other gestural interfaces

The set of gestures is discoverable and “self-revealing”, contrary to other gestural interfaces


Improved graphical presentation

Improved graphical presentation


Marking menus

Marking Menus

  • Video/Demonstration


Marking menus1

Marking Menus

Video/Demonstration


Marking menus2

Marking Menus

Video/Demonstration


Transition from novice to expert

Transition from novice to expert

Marking Menus:Transition is gradual and natural !

Traditional menus:Pointing versus shortcuts


Summary1

Summary

Marking Menus

  • Can be operated faster than linear menus(ballistic, directional, scale-invariant marks)

  • Have a self-revealing gesture set

  • Enable a smooth transitionfrom novice use to expert use

  • Can be used to select both a noun and verb

  • Are limited to 8 items per submenu,and limited to a depth of about 3 levels


Interaction gestural input popup widgets interaction techniques and hardware input devices

Marking Menus

  • Allow for faster selection than with linear menus(directional, ballistic movements)

  • Can be used without looking at the screen(“eyes-free operation”)

  • Have a discoverable set of gestures

  • Allow for a gradual and natural transition from novice to expert

  • Can be used to select a noun and verb

  • Are limited to approximately 8 items per submenu, and a depth of about 3 levels


Summary2

Summary

  • Temporary modes, held by holding down a button or key with kinesthetic feedback …

    • Help avoid mode errors

    • Example: a key that must be held down to maintain a temporary mode change, such as the Shift key on keyboards

    • Example: popup widgets

      • Help combine noun+verb selection

      • Only require screen space when in use

  • One popup widget with many advantages: Marking Menus

    • Allow for ballistic gestures

    • Allows users to gradually transition from novice to expert


Other examples of modes tools etc

Other examplesof modes, tools, etc.

  • video: "Selection and Positioning tasks", Bill Buxton 1983


Some observations

Some observations

  • Techniques shown in the video:

    • 1. Drag-and-drop

    • 2. Modal toolbar

    • 3. « Moving menu » (a kind of popup menu)

    • 4. « Moving menu » with memory

  • Questions: of the 4 techniques, which ones …

    • Are not modal (or only have a temporary mode with kinesthetic feedback), and so help to avoid mode errors ? (Answer: 1 and 3)

    • Avoid back-and-forth movements between the workspace and the toolbar, and do not have screenspace consumed by a toolbar ? (Answer: 3 and 4)

    • Have visible affordances (toolbar), therefore showing the possible operations to a novice user ? (Answer: 1 and 2)

    • Allow the same shape to be created many times repeatedly, very quickly ?(Answer: 2 and 4)

    • Ensure that each shape is always selected with the same dragging gesture, allowing the user to learn the gestures by heart and then execute them rapidly ? (Answer: 1 and especially 3.)


Can we extend popup menus widgets for other uses

Can we extend popup menus/widgetsfor other uses?


Other popup menus and widgets

Other popup menus and widgets

  • Hotbox (Kurtenbach et al., 1999),Control Menus (Pook et al., 2000),Flow Menus (Guimbretière et Winograd, 2000),FaST Sliders (McGuffin et al., 2002),Tracking Menus (Fitzmaurice et al., 2003),Trailing Widget (Forlines et al., 2006)Hover Widgets (Grossman et al., 2006),PieCursor (Fitzmaurice et al., 2008)

  • These widgets and interaction techniques are designed for:

    • A large number of commands

    • Controlling continuous variables

    • Input of text and numbers with gestures

    • Use with a stylus (pen), for example, on a “tablet PC”


Interaction gestural input popup widgets interaction techniques and hardware input devices

FlowMenus (Guimbretière et al., 2000)


Interaction gestural input popup widgets interaction techniques and hardware input devices

2D manipulation with FlowMenus(Guimbretière et al., 2000)


The hotbox in maya a 2d menu

The Hotbox in Maya: a 2D menu

[Kurtenbach et al., 1999]


The hotbox in maya a 2d menu1

The Hotbox in Maya: a 2D menu

[Kurtenbach et al., 1999]


The hotbox in maya a 2d menu2

The Hotbox in Maya: a 2D menu

[Kurtenbach et al., 1999]


Interaction gestural input popup widgets interaction techniques and hardware input devices

Demo of a more recent kind of Hotbox (McGuffin and Jurisica 2009) used for manipulating a network visualization


Interaction gestural input popup widgets interaction techniques and hardware input devices

Tracking Menu (copy of video athttp://www.dgp.toronto.edu/~gf/videos/tm.aviat George Fitzmaurice’s website)


Interaction gestural input popup widgets interaction techniques and hardware input devices

Hover Widgets (copy of video at http://www.dgp.toronto.edu/~ravin/videos/chi2006_hoverwidgets.mov at Ravin Balakrishnan’s website)


Toolglass bimanual input

Toolglass: bimanual input

Click-through: Simultaneousselection of verb and noun!


Bimanual 3d navigation

Bimanual 3D Navigation

Erick Velazquez-Godinez and Michael McGuffin (unpublished, 2010)http://www.youtube.com/watch?v=ggoLLnxHwhAHaving the 3D position of each hand is enough to control the 6 degrees-of-freedom of a 3D camera: translate hands together to translate the camera, rotate hands around each other to rotate the camera, move hands toward or away from each other to zoom.


Tracking 2 or more fingers with a nintendo wiimote

Tracking 2 or more fingers with a Nintendo Wiimote

  • See online videos by Johnny Chung Lee http://johnnylee.net/projects/wii/


Other uses of two hands

Other uses of two hands?


Two handed bimanual input

Two-Handed (Bimanual) Input

Potential uses:

  • Dominant hand (DH) on mouse,non-dominant hand (NDH) on keyboard

  • Two mice, two cursors, symmetrical:

    • Rapid clicking by alternating between hands?

    • Simultaneous rotation+scaling+positioning in 2Dor rotation+zooming+panning in 2D

  • Two mice, asymmetrical:

    • NDH for camera, DH for selection/manipulation

    • NDH for tool palette, DH for clicking-through (Toolglass)


Kinematic chain model yves guiard 1987

Kinematic Chain Model(Yves Guiard 1987)

  • NDH (non-dominant hand) performs slower and less precise movements than the DH (dominant hand)

  • The NDH moves before the DH

  • The NDH establishes a frame of reference within which the DH performs work


Input devices

Input Devices


Keyboards

Keyboards


Interaction gestural input popup widgets interaction techniques and hardware input devices

http://world.std.com/~jdostale/kbd/SpaceCadet1.jpeg


Interaction gestural input popup widgets interaction techniques and hardware input devices

Foldable keyboard for Palm Pilot

Flexible keyboard; 24 $ at http://www.thinkgeek.com/computing/input/5a7f/?cpg=ab(price from 2008)


Projected keyboard

Projected Keyboard

http://en.wikipedia.org/wiki/Projection_keyboard

160$ at http://www.thinkgeek.com/computing/input/8193/(price of 2008)


Optimus

Optimus

http://computer.howstuffworks.com/keyboard.htm/printable


Optimus maximus

Optimus Maximus

Each of the 113 keys contain a 48x48 pixel screen.

1864 $ US ! (price of 2008)

http://www.artlebedev.com/everything/optimus/


Optimus maximus1

Optimus Maximus

Configured for English


Optimus maximus2

Optimus Maximus

Configured for Russian


Optimus maximus3

Optimus Maximus

Configured for the game "Half-life"


Optimus maximus4

Optimus Maximus

Configured for "Photoshop"


Bottons on a tape recorder

Bottons on a tape recorder

The status quo:

Uniform buttons

Buttons with different forms: allow use without looking (“eyes-free operation”)


Piano keyboard

modern design

naïve design

Piano Keyboard


Apple iphone keyboard

Apple iPhone Keyboard


Scholes keyboard versus dvorak keyboard

Scholes Keyboard versus Dvorak Keyboard

Still not in widespread use

Notice that all vowels are under one hand in the midlde row.

Designed in 1873 !

Still the standard :-(


Comparison of qwerty and dvorak

Comparison of QWERTY and Dvorak

http://infohost.nmt.edu/~shipman/ergo/parkinson.html


Half keyboard from matias corp

“Half Keyboard” from Matias Corp.

http://www.halfkeyboard.com/


Septambic keyer or chorded keyboard

« Septambic keyer »or « chorded keyboard »

4 x 24 – 1 = 63 possible "chords"


Pointing devices

Pointing Devices


The first mouse

The first mouse

  • 1968

  • Douglas Engelbart

  • Stanford Research Institute

  • Two rollers for x and y


40 years later

40+ years later

Rotation sensing

Rockin’ Mouse

Haptic feedback(or “force feedback”)

Touchpad


Other kinds of 2d pointing devices

Other kinds of 2D pointing devices

  • Digitizing tablet / graphics tablet with a stylus (pen) and/or puck (mouse)

  • Touchscreen

  • Light pen

  • Touchpad

  • Joystick

    • isometric (rigid, senses pressure, for example: Trackpoint)

    • elastic (like isotonic, but returns to center when released)

    • isotonic (can be moved freely)

  • Trackball

  • Eye tracker


Interaction gestural input popup widgets interaction techniques and hardware input devices

TrackPoint (isometric joystick)

Touchpad


Logitech trackball

Logitech Trackball


Graphics tablets

Graphics tablets

Wacom Bamboo

5.8x3.7 inches; 80$

Wacom Intuos3

12x19 inches; 750$

Wacom Cintiq 21UX

with integrated screen

21.3 inches (17x12.75); 2500$

(prices of 2008)


Input devices for tablets

Input devices for tablets

  • Styluses can have

  • a pressure sensor on their tip

  • a button on the tip to click

  • a button on the side ("barrel button")

  • a scroll wheel that can be rotated


Intupaint vandoren et al 2008 and fluidpaint vandoren et al 2009

IntuPaint (Vandoren et al. 2008)and FluidPaint (Vandoren et al. 2009)


Properties of pointing devices

Properties of Pointing Devices

Absolute vs relative sensing

Which is more general ?

Direct vs indirect

Which is more “intuitive” ?

Which is more predictable ?

Which is more relaxing ?

Which leaves visual feedback more visible ?

Which do artists prefer ?

Discrete vs continuous

Which is more general ?

Position control vs velocity control

Which is more general ?

Which typically requires less movement/smaller footprint ?

Velocity control:


Velocity control pointing devices for 3d

Velocity control pointing devices, for 3D

Spaceball

Spaceball 5000

Spacemouse

Magellan

SpaceNavigator (60$)

3dconnexion.com

(in 2008)

Other products from 3dconnexion.com

(in 2008)


Taxonomy of input devices buxton

Taxonomy of input devices (Buxton)

M: Mechanical intermediary

T: Touch


3 state model buxton 1990

3-State Model (Buxton, 1990)

State 0: no (x,y) coordinates

States 1 and 2: (x,y) position is known

  • Examples:

  • Graphics tablet: states 0, 1, 2

  • Mouse: states 1, 2

  • Touchscreen: states 0, 1


Touchmouse hinckley and sinclair 1999

TouchMouse(Hinckley and Sinclair 1999)

States 0, 1, 2


Touchmouse hinckley and sinclair 19991

TouchMouse(Hinckley and Sinclair 1999)


Pop through buttons zeleznik et al 2001 http doi acm org 10 1145 502348 502384

“Pop Through” Buttons(Zeleznik et al. 2001, http://doi.acm.org/10.1145/502348.502384)


Presence rekimoto et al 2003

PreSence (Rekimoto et al. 2003)


Haptic pen

Haptic Pen

Lee et al., "Haptic pen: a tactile feedback stylus for touch screens", UIST 2004


Handheld projector and pen

Handheld Projector and Pen

Cao et Balakrishnan, "Interacting with Dynamically Defined Information Spaces using a Handheld Projector and a Pen", UIST 2006


Penlight

PenLight

Song et al., "PenLight: combining a mobile projector and a digital pen for dynamic visual overlay", CHI 2009


  • Login