Interaction gestural input popup widgets interaction techniques and hardware input devices
Download
1 / 97

Interaction: gestural input, popup widgets, interaction techniques, and hardware input devices - PowerPoint PPT Presentation


  • 126 Views
  • Uploaded on

Interaction: gestural input, popup widgets, interaction techniques, and hardware input devices. Prof. Michael McGuffin. Gestural Input. Gestures can be executed rapidly, and can be used as symbols to activate commands or select objects Sketches

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Interaction: gestural input, popup widgets, interaction techniques, and hardware input devices' - coty


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Interaction gestural input popup widgets interaction techniques and hardware input devices

Interaction: gestural input, popup widgets, interaction techniques, and hardware input devices

Prof. Michael McGuffin


Gestural input
Gestural Input techniques, and hardware input devices

  • Gestures can be executed rapidly, and can be used as symbols to activate commands or select objects

  • Sketches

    • can be used to quickly enter text or diagrams, without requiring the user to switch from the pointing device to something else

    • Have an informal, loose, implicit structure

    • Can carry much more (implicit) information than typed text

  • Gestures as characters to recognize

    • Graffiti, Unistroke


Gestural input1
Gestural Input techniques, and hardware input devices

Kurtenbach and Buxton (video)

  • Example gestures: circle objects, drag to move, or drag + draw a "C"  to copy

  • Such a gesture selects the noun (objects) and verb (operation) in a single action, which may be more natural, fluid, or faster than doing selection + Ctrl-C/Ctrl-X + Ctrl-V


Gestural input2
Gestural Input techniques, and hardware input devices

"Teddy" (Igarashi et al. 1999).

See Java applet at Takeo Igarashi’s website.


Phun 2008
Phun (≈2008) techniques, and hardware input devices

  • A free game/physical simulator that uses gestural input

  • http://www.acc.umu.se/~emilk/


Phun techniques, and hardware input devices




Observation
Observation techniques, and hardware input devices

  • Most user interfaces that use gestural input require the user to first learn the gestures before they can be used. This can be a significant obstacle for novice users.

  • In the following slides, we will see a technique called Marking Menus that eliminates this problem.



Interaction techniques
Interaction Techniques gestural input or other input

  • “are ways to use input devices to enter information”

  • "are made up of single input-device actions.”

    (Foley, et al. “Computer Graphics: Principles and Practice”)


Verbs gestural input or other input(actions,commands,tools,operations)

Nouns(objects,locations)


Verbs in a pulldown menu gestural input or other input

Nouns(objects,locations)


Verbs in gestural input or other inputa popup menu

Nouns(objects,locations)


Summary
Summary gestural input or other input

  • Modes can lead to mode errors, where the user thinks they are in one mode but are really in another

  • Visual feedback (example: changing cursor shape) to indicate the current mode is good, but often not enough to avoid mode errors

  • Popup menus help…

    • Avoid mode errors(via transient modes, and kinaesthetic feedback where the user feels pressure through their finger from holding down a button)

    • Increase screen space devoted to content, because the menu is only displayed when desired

    • Decrease the distance travelled by the cursor, because there is no back-and-forth motion between the workspace and peripheral interface elements

    • Create a tighter integration, or fusion, of the selection of noun and verb, leading to better mental chunking (Buxton 1986)


Given that popup menus have all these advantages can we design other widgets that are even better
Given that popup menus have all these advantages, can we design other widgets that are even better?


Radial menu or pie menu
Radial Menu (or “Pie Menu”) design other widgets that are even better?


Demo radial menu in simplepaint
Demo: Radial Menu in SimplePaint design other widgets that are even better?


An interface that uses effectively radial menus
An interface that uses, effectively, radial menus design other widgets that are even better?

Yatani et al., CHI 2008


Radial menus versus linear menus
Radial menus design other widgets that are even better?versus linear menus


Radial menus versus linear menus1
Radial menus design other widgets that are even better?versus linear menus


Radial menus versus linear menus2
Radial menus design other widgets that are even better?versus linear menus


Radial menus versus linear menus3
Radial menus design other widgets that are even better?versus linear menus


Radial menus versus linear menus4
Radial menus design other widgets that are even better?versus linear menus


Radial menus versus linear menus5
Radial menus design other widgets that are even better?versus linear menus

Directions are easier to rememberand reproduce than distances


Hierarchical radial menu
Hierarchical Radial Menu design other widgets that are even better?


Mouse gestures for firefox
"Mouse Gestures" for Firefox design other widgets that are even better?


Marking menu
Marking Menu design other widgets that are even better?

  • “Scale invariant recognition”: The recognition of the gestures (marks) does not depend on the length of the segments; only the angles matter. Hence, the marks can be drawn small and quickly, or ballistically.

  • A user who knows what mark to draw doesn’t even need to see the menu popup.


The set of gestures is discoverable and self revealing contrary to other gestural interfaces
The set of gestures is discoverable and “self-revealing”, contrary to other gestural interfaces


Improved graphical presentation
Improved graphical presentation “self-revealing”, contrary to other gestural interfaces


Marking menus
Marking Menus “self-revealing”, contrary to other gestural interfaces

  • Video/Demonstration


Marking menus1
Marking Menus “self-revealing”, contrary to other gestural interfaces

Video/Demonstration


Marking menus2
Marking Menus “self-revealing”, contrary to other gestural interfaces

Video/Demonstration


Transition from novice to expert
Transition from novice to expert “self-revealing”, contrary to other gestural interfaces

Marking Menus:Transition is gradual and natural !

Traditional menus:Pointing versus shortcuts


Summary1
Summary “self-revealing”, contrary to other gestural interfaces

Marking Menus

  • Can be operated faster than linear menus(ballistic, directional, scale-invariant marks)

  • Have a self-revealing gesture set

  • Enable a smooth transitionfrom novice use to expert use

  • Can be used to select both a noun and verb

  • Are limited to 8 items per submenu,and limited to a depth of about 3 levels


Marking Menus “self-revealing”, contrary to other gestural interfaces

  • Allow for faster selection than with linear menus(directional, ballistic movements)

  • Can be used without looking at the screen(“eyes-free operation”)

  • Have a discoverable set of gestures

  • Allow for a gradual and natural transition from novice to expert

  • Can be used to select a noun and verb

  • Are limited to approximately 8 items per submenu, and a depth of about 3 levels


Summary2
Summary “self-revealing”, contrary to other gestural interfaces

  • Temporary modes, held by holding down a button or key with kinesthetic feedback …

    • Help avoid mode errors

    • Example: a key that must be held down to maintain a temporary mode change, such as the Shift key on keyboards

    • Example: popup widgets

      • Help combine noun+verb selection

      • Only require screen space when in use

  • One popup widget with many advantages: Marking Menus

    • Allow for ballistic gestures

    • Allows users to gradually transition from novice to expert


Other examples of modes tools etc
Other examples “self-revealing”, contrary to other gestural interfacesof modes, tools, etc.

  • video: "Selection and Positioning tasks", Bill Buxton 1983


Some observations
Some observations “self-revealing”, contrary to other gestural interfaces

  • Techniques shown in the video:

    • 1. Drag-and-drop

    • 2. Modal toolbar

    • 3. « Moving menu » (a kind of popup menu)

    • 4. « Moving menu » with memory

  • Questions: of the 4 techniques, which ones …

    • Are not modal (or only have a temporary mode with kinesthetic feedback), and so help to avoid mode errors ? (Answer: 1 and 3)

    • Avoid back-and-forth movements between the workspace and the toolbar, and do not have screenspace consumed by a toolbar ? (Answer: 3 and 4)

    • Have visible affordances (toolbar), therefore showing the possible operations to a novice user ? (Answer: 1 and 2)

    • Allow the same shape to be created many times repeatedly, very quickly ?(Answer: 2 and 4)

    • Ensure that each shape is always selected with the same dragging gesture, allowing the user to learn the gestures by heart and then execute them rapidly ? (Answer: 1 and especially 3.)


Can we extend popup menus widgets for other uses
Can we extend popup menus/widgets “self-revealing”, contrary to other gestural interfacesfor other uses?


Other popup menus and widgets
Other popup menus and widgets “self-revealing”, contrary to other gestural interfaces

  • Hotbox (Kurtenbach et al., 1999),Control Menus (Pook et al., 2000),Flow Menus (Guimbretière et Winograd, 2000),FaST Sliders (McGuffin et al., 2002),Tracking Menus (Fitzmaurice et al., 2003),Trailing Widget (Forlines et al., 2006)Hover Widgets (Grossman et al., 2006),PieCursor (Fitzmaurice et al., 2008)

  • These widgets and interaction techniques are designed for:

    • A large number of commands

    • Controlling continuous variables

    • Input of text and numbers with gestures

    • Use with a stylus (pen), for example, on a “tablet PC”


FlowMenus (Guimbretière et al., 2000) “self-revealing”, contrary to other gestural interfaces


2D manipulation with FlowMenus “self-revealing”, contrary to other gestural interfaces(Guimbretière et al., 2000)


The hotbox in maya a 2d menu
The Hotbox in Maya: a 2D menu “self-revealing”, contrary to other gestural interfaces

[Kurtenbach et al., 1999]


The hotbox in maya a 2d menu1
The Hotbox in Maya: a 2D menu “self-revealing”, contrary to other gestural interfaces

[Kurtenbach et al., 1999]


The hotbox in maya a 2d menu2
The Hotbox in Maya: a 2D menu “self-revealing”, contrary to other gestural interfaces

[Kurtenbach et al., 1999]


Demo of a more recent kind of Hotbox (McGuffin and Jurisica 2009) used for manipulating a network visualization


Tracking Menu ( 2009) used for manipulating a network visualizationcopy of video athttp://www.dgp.toronto.edu/~gf/videos/tm.aviat George Fitzmaurice’s website)


Hover Widgets ( 2009) used for manipulating a network visualizationcopy of video at http://www.dgp.toronto.edu/~ravin/videos/chi2006_hoverwidgets.mov at Ravin Balakrishnan’s website)


Toolglass bimanual input
Toolglass: bimanual input 2009) used for manipulating a network visualization

Click-through: Simultaneousselection of verb and noun!


Bimanual 3d navigation
Bimanual 3D Navigation 2009) used for manipulating a network visualization

Erick Velazquez-Godinez and Michael McGuffin (unpublished, 2010)http://www.youtube.com/watch?v=ggoLLnxHwhAHaving the 3D position of each hand is enough to control the 6 degrees-of-freedom of a 3D camera: translate hands together to translate the camera, rotate hands around each other to rotate the camera, move hands toward or away from each other to zoom.


Tracking 2 or more fingers with a nintendo wiimote
Tracking 2 or more fingers with a Nintendo Wiimote 2009) used for manipulating a network visualization

  • See online videos by Johnny Chung Lee http://johnnylee.net/projects/wii/


Other uses of two hands
Other uses of two hands? 2009) used for manipulating a network visualization


Two handed bimanual input
Two-Handed (Bimanual) Input 2009) used for manipulating a network visualization

Potential uses:

  • Dominant hand (DH) on mouse,non-dominant hand (NDH) on keyboard

  • Two mice, two cursors, symmetrical:

    • Rapid clicking by alternating between hands?

    • Simultaneous rotation+scaling+positioning in 2Dor rotation+zooming+panning in 2D

  • Two mice, asymmetrical:

    • NDH for camera, DH for selection/manipulation

    • NDH for tool palette, DH for clicking-through (Toolglass)


Kinematic chain model yves guiard 1987
Kinematic Chain Model 2009) used for manipulating a network visualization(Yves Guiard 1987)

  • NDH (non-dominant hand) performs slower and less precise movements than the DH (dominant hand)

  • The NDH moves before the DH

  • The NDH establishes a frame of reference within which the DH performs work


Input devices

Input Devices 2009) used for manipulating a network visualization


Keyboards
Keyboards 2009) used for manipulating a network visualization


http://world.std.com/~jdostale/kbd/SpaceCadet1.jpeg 2009) used for manipulating a network visualization


Foldable keyboard for Palm Pilot 2009) used for manipulating a network visualization

Flexible keyboard; 24 $ at http://www.thinkgeek.com/computing/input/5a7f/?cpg=ab(price from 2008)


Projected keyboard
Projected Keyboard 2009) used for manipulating a network visualization

http://en.wikipedia.org/wiki/Projection_keyboard

160$ at http://www.thinkgeek.com/computing/input/8193/(price of 2008)


Optimus
Optimus 2009) used for manipulating a network visualization

http://computer.howstuffworks.com/keyboard.htm/printable


Optimus maximus
Optimus Maximus 2009) used for manipulating a network visualization

Each of the 113 keys contain a 48x48 pixel screen.

1864 $ US ! (price of 2008)

http://www.artlebedev.com/everything/optimus/


Optimus maximus1
Optimus Maximus 2009) used for manipulating a network visualization

Configured for English


Optimus maximus2
Optimus Maximus 2009) used for manipulating a network visualization

Configured for Russian


Optimus maximus3
Optimus Maximus 2009) used for manipulating a network visualization

Configured for the game "Half-life"


Optimus maximus4
Optimus Maximus 2009) used for manipulating a network visualization

Configured for "Photoshop"


Bottons on a tape recorder
Bottons on a tape recorder 2009) used for manipulating a network visualization

The status quo:

Uniform buttons

Buttons with different forms: allow use without looking (“eyes-free operation”)


Piano keyboard

modern design 2009) used for manipulating a network visualization

naïve design

Piano Keyboard


Apple iphone keyboard
Apple iPhone Keyboard 2009) used for manipulating a network visualization


Scholes keyboard versus dvorak keyboard
Scholes Keyboard versus Dvorak Keyboard 2009) used for manipulating a network visualization

Still not in widespread use

Notice that all vowels are under one hand in the midlde row.

Designed in 1873 !

Still the standard :-(


Comparison of qwerty and dvorak
Comparison of QWERTY and Dvorak 2009) used for manipulating a network visualization

http://infohost.nmt.edu/~shipman/ergo/parkinson.html


Half keyboard from matias corp
“Half Keyboard” from Matias Corp. 2009) used for manipulating a network visualization

http://www.halfkeyboard.com/


Septambic keyer or chorded keyboard
« Septambic keyer » 2009) used for manipulating a network visualizationor « chorded keyboard »

4 x 24 – 1 = 63 possible "chords"


Pointing devices
Pointing Devices 2009) used for manipulating a network visualization


The first mouse
The first mouse 2009) used for manipulating a network visualization

  • 1968

  • Douglas Engelbart

  • Stanford Research Institute

  • Two rollers for x and y


40 years later
40+ years later 2009) used for manipulating a network visualization

Rotation sensing

Rockin’ Mouse

Haptic feedback(or “force feedback”)

Touchpad


Other kinds of 2d pointing devices
Other kinds of 2D pointing devices 2009) used for manipulating a network visualization

  • Digitizing tablet / graphics tablet with a stylus (pen) and/or puck (mouse)

  • Touchscreen

  • Light pen

  • Touchpad

  • Joystick

    • isometric (rigid, senses pressure, for example: Trackpoint)

    • elastic (like isotonic, but returns to center when released)

    • isotonic (can be moved freely)

  • Trackball

  • Eye tracker


TrackPoint (isometric joystick) 2009) used for manipulating a network visualization

Touchpad


Logitech trackball
Logitech Trackball 2009) used for manipulating a network visualization


Graphics tablets
Graphics tablets 2009) used for manipulating a network visualization

Wacom Bamboo

5.8x3.7 inches; 80$

Wacom Intuos3

12x19 inches; 750$

Wacom Cintiq 21UX

with integrated screen

21.3 inches (17x12.75); 2500$

(prices of 2008)


Input devices for tablets
Input devices for tablets 2009) used for manipulating a network visualization

  • Styluses can have

  • a pressure sensor on their tip

  • a button on the tip to click

  • a button on the side ("barrel button")

  • a scroll wheel that can be rotated


Intupaint vandoren et al 2008 and fluidpaint vandoren et al 2009
IntuPaint (Vandoren et al. 2008) 2009) used for manipulating a network visualizationand FluidPaint (Vandoren et al. 2009)


Properties of pointing devices
Properties of Pointing Devices 2009) used for manipulating a network visualization

Absolute vs relative sensing

Which is more general ?

Direct vs indirect

Which is more “intuitive” ?

Which is more predictable ?

Which is more relaxing ?

Which leaves visual feedback more visible ?

Which do artists prefer ?

Discrete vs continuous

Which is more general ?

Position control vs velocity control

Which is more general ?

Which typically requires less movement/smaller footprint ?

Velocity control:


Velocity control pointing devices for 3d
Velocity control pointing devices, for 3D 2009) used for manipulating a network visualization

Spaceball

Spaceball 5000

Spacemouse

Magellan

SpaceNavigator (60$)

3dconnexion.com

(in 2008)

Other products from 3dconnexion.com

(in 2008)


Taxonomy of input devices buxton
Taxonomy of input devices (Buxton) 2009) used for manipulating a network visualization

M: Mechanical intermediary

T: Touch


3 state model buxton 1990
3-State Model (Buxton, 1990) 2009) used for manipulating a network visualization

State 0: no (x,y) coordinates

States 1 and 2: (x,y) position is known

  • Examples:

  • Graphics tablet: states 0, 1, 2

  • Mouse: states 1, 2

  • Touchscreen: states 0, 1


Touchmouse hinckley and sinclair 1999
TouchMouse 2009) used for manipulating a network visualization(Hinckley and Sinclair 1999)

States 0, 1, 2


Touchmouse hinckley and sinclair 19991
TouchMouse 2009) used for manipulating a network visualization(Hinckley and Sinclair 1999)


Pop through buttons zeleznik et al 2001 http doi acm org 10 1145 502348 502384
“Pop Through” Buttons 2009) used for manipulating a network visualization(Zeleznik et al. 2001, http://doi.acm.org/10.1145/502348.502384)


Presence rekimoto et al 2003
PreSence (Rekimoto et al. 2003) 2009) used for manipulating a network visualization


Haptic pen
Haptic Pen 2009) used for manipulating a network visualization

Lee et al., "Haptic pen: a tactile feedback stylus for touch screens", UIST 2004


Handheld projector and pen
Handheld Projector and Pen 2009) used for manipulating a network visualization

Cao et Balakrishnan, "Interacting with Dynamically Defined Information Spaces using a Handheld Projector and a Pen", UIST 2006


Penlight
PenLight 2009) used for manipulating a network visualization

Song et al., "PenLight: combining a mobile projector and a digital pen for dynamic visual overlay", CHI 2009


ad