Interaction Devices

1. Interaction Devices Human Computer Interaction CIS 6930/4930

2. Interaction Performance • 60s vs. Today • Performance • Hz -> GHz • Memory • k -> GB • Storage • k -> TB

3. Interaction Performance • 60s vs. Today • Input • punch cards -> • Keyboards, Pens, tablets, mobile phones, mice, cameras, web cams • Output • 10 character/sec -> • Megapixel displays, HD capture and display, color laser, surround sound, force feedback, VR • Substantial bandwidth increase!

4. Interaction Future • Gestural input • Two-handed input • 3D/6D I/O • Glass (1:52) • Others: voice, wearable, whole body, eye trackers, data gloves, haptics, force feedback • Engineering research! • Entire companies created around one single technology • Magic Leap http://virginradiodubai.blob.core.windows.net/images/2014/05/googleglass_660.jpg

5. Interaction Current trends • Multimodal (using car navigation via buttons or voice) • Helps disabled (especially those with different levels of disability)

6. Keyboard and Keypads • QWERTY keyboards been around for a long time • (1870s – Christopher Sholes) • Cons: Not easy to learn • Pros: Familiarity • Stats: • Beginners: 1 keystroke per sec • Average office worker: 5 keystrokes (50 wpm) • Experts: 15 keystrokes per sec (150 wpm) • Is it possible to do better?

7. Keyboard and Keypads • How important is: • Accuracy • Training • Keyboard properties that matter • Size • Adjustability • Reduces RSI, better performance and comfort • Mobile phone keyboards, blackberry devices, etc.

8. Keyboard and Keypads • How important is: • Accuracy • Training • Keyboard properties that matter • Size • Adjustability • Reduces RSI, better performance and comfort • Mobile phone keyboards, blackberry devices, etc.

9. Keyboard Layouts • QWERTY • Frequently used pairs far apart • Fewer typewriter jams • Electronic approaches don’t jam.. why use it? • DVOARK (1920s) • 150 wpm->200 wpm • Reducing errors • Takes about one week to switch • Stops most from trying

10. Keyboard Layouts • ABCDE – style • Easier for non-typists • Studies show no improvement vs. QWERTY • Number pads • What’s in the top row? • Look at phones (slight faster), then look at calculators, keypads • Those for disabled • Split keyboards • KeyBowl’s orbiTouch • Eyetrackers, mice • Dasher - 2d motion with word prediction

11. Keys • Current keyboards have been extensively tested • Size • Shape • Required force • Spacing • Speed vs. error rates for majority of users • Distinctive click gives audio feedback • Why membrane keyboards are slow (Atari 400?) • Environment hazards might necessitate • Usually speed is not a factor

12. Keys Guidelines • Special keys should be denoted • State keys (such as caps, etc.) should have easily noted states • Special curves or dots for home keys for touch typists • Inverted T Cursor movement keys are important (though cross is easier for novices)

13. Keys Guidelines • Auto-repeat feature • Improves performance • But only if repeat is customizable (motor impaired, young, old) • Two thinking points: • Why are home keys fastest to type? • Why are certain keys larger? (Enter, Shift, Space bar) • Another example of Fitt’s Law

14. Keypads for small devices • PDAs, Cellphones, Game consoles • Fold out keyboards • Virtual keyboard • Cloth keyboards (ElekSen) • Most lack haptic feedback?

15. Keypads for small devices • Mobile phones • Combine static keys with dynamic soft keys • Multi-tap a key to get to a character • Study: Predictive techniques greatly improve performance • Ex. LetterWise = 20 wpm vs 15 wpm multitap • Draw keyboard on screen and tap w/ pen • Speed: 20 to 30 wpm (Sears ’93) • Swipe • Handwriting recognition (still hard) • Subset: Graffiti2 (uses unistrokes)

16. Pointing Devices • Direct manipulation needs some pointing device • Factors: • Size of device • Accuracy • Dimensionality • Interaction Tasks: • Select – menu selection, from a list • Position – 1D, 2D, 3D (ex. paint) • Orientation – Control orientation or provide direct 3D orientation input • Path – Multiple poses are recorded • ex. to draw a line • Quantify – control widgets that affect variables • Text – move text

17. Pointing Devices • Faster w/ less error than keyboard • Two types (Box 9.1) • Direct control – device is on the screen surface (touchscreen, stylus) • Indirect control – mouse, trackball, joystick, touchpad

18. Direct-control pointing • First device – lightpen • Point to a place on screen and press a button • Pros: • Easy to understand and use • Very fast for some operations (e.g. drawing) • Cons: • Hand gets tired fast! • Hand and pen blocks view of screen • Fragile

19. Direct-control pointing • Evolved into the touchscreen • Pros: Very robust, no moving parts • Cons: Depending on app, accuracy could be an issue • 1600x1600 res with acoustic wave • Must be careful about software design for selection (land-on strategy). • If you don’t show a cursor of where you are selecting, users get confused • User confidence is improved with a good lift-off strategy • Now combination • Nintendo DS • Samsung Note

20. Direct-control pointing • Primarily for novice users or large user base • Case study: Disney World • Need to consider those who are: disabled, illiterate, hard of hearing, errors in usage (two touch points), etc.

21. Indirect-Control Pointing • Pros: • Reduces hand-fatigue • Reduces obscuration problems • Cons: • Increases cognitive load • Spatial ability comes more into play

22. Indirect-Control Pointing - MOUSE • Pros: • Familiarity • Wide availability • Low cost • Easy to use • Accurate • Cons: • Time to grab mouse • Desk space • Encumbrance (wire), dirt • Long motions aren’t easy or obvious (pick up and replace) • Consider, weight, size, style, # of buttons, force feedback

23. Indirect-Control Pointing • Trackball • Pros: • Small physical footprint • Good for kiosks • Joystick • Easy to use, lots of buttons • Good for tracking (guide or follow an on screen object) • Does it map well to your app? • Touchpoint • Pressure-sensitive ‘nubbin’ on laptops • Keep fingers on the home position

24. Indirect-Control Pointing • Touchpad • Laptop mouse device • Lack of moving parts, and low profile • Accuracy potentially low for those with motor disabilities • Graphics Tablet • Comfort • Good for CAD, artists • Limited data entry

25. Comparing pointing devices • Direct pointing • Study: Faster but less accurate than indirect (Haller ’84) • Lots of studies confirm mouse is best for most tasks for speed and accuracy • Trackpoint < Trackballs & Touchpads < Mouse • Short distances – cursor keys are better (experts use keyboard for movement more) • Disabled prefer joysticks and trackballs • If force application is a problem, then touch sensitive is preferred • Vision impaired have problems with most pointing devices • Use multimodal approach or customizable cursors • Read Vanderheiden ’04 for a case study • Designers should smooth out trajectories • Large targets reduce time and frustration

26. Example • Five fastest places to click on for a right-handed user?

27. Example • What affects time?

28. Fitt’s Law Recreation • .5” • 1” 2”

29. FITTS’S LAW • Paul Fitts (1954) developed a model of human hand movement • Used to predict time to point at an object • What are the factors to determine the time to point to an object? • D – distance to target • W – size of target • Just from your own experience, is this function linear? • No, since if Target A is D distance and Target B is 2D distance, it doesn’t take twice as long • What about target size? Not linear there either • T = a + b log2(D/W + 1) • http://www.lynda.com/Web-User-Experience-tutorials/Understanding-Fittss-Law/103677/119792-4.html

30. FITTS’S LAW • T = a + b log2(D/W + 1) • T = mean time • a = time to start/stop in seconds (empirically measured per device) • b = inherent speed of the device (empirically measured per device) [time/bit or ms/bit] • Ex. a = 300 ms, b = 200 ms/bit, D = 14 cm, W = 2 cm • Ans: 300 + 200 log2(14/2 + 1) = 900 ms • Question: If I wanted to half the pointing time (on average), how much do I change the size? • Proven to provide good timings for most age groups • Newer versions taken into account • Direction (we are faster horizontally than vertically) • Device weight • Target shape • Arm position (resting or midair) • 2D and 3D (Zhai ’96)

31. Examples • T = a + b log2(D/W + 1)

32. Examples • T = a + b log2(D/W + 1)

33. Examples

34. FITTS’S LAW • T = a + b log2(D/W + 1) • T = mean time • a = time to start/stop in seconds (empirically measured per device) • b = inherent speed of the device (empirically measured per device) [time/bit or ms/bit] • First part is device characteristics • Second part is target difficulty

35. Very Successfully Studied • Applies to • Feet, eye gaze, head mounted sights • Many types of input devices • Physical environments (underwater!) • User populations (even mentally handicapped and drugged) • Drag & Drop and Point & Click • Limitations • Dimensionality • Software accelerated pointer motion • Training • Trajectory Tasks (Accot-Zhai Steering Law is a good predictor and joins Fitt’s Law) • Decision Making (Hick’s Law)

36. Very Successfully Studied • Results (what does it say about) • Buttons and widget size? • Edges? • Popup vs. pull-down menus • Pie vs. Linear menus • iPhone/web pages (real borders) vs. monitor+mouse (virtual borders) • Interesting readings: • http://particletree.com/features/visualizing-fittss-law/ • http://www.asktog.com/columns/022DesignedToGiveFitts.html • http://www.yorku.ca/mack/GI92.html • Using Fitt’s Law to slow people down

37. Precision Pointing Movement Time • Study: Sears and Shneiderman ’91 • Broke down task into gross and fine components for small targets • Precision Point Mean Time = a + b log2(D/W+1) + c log2(d/W) • c – speed for short distance movement • d – minor distance • Notice how the overall time changes with a smaller target. • Other factors • Age (Pg. 369) • Research: How can we design devices that produce smaller constants for the predictive equation • Two handed • Zooming

38. Affordance • Quality of an object, or an environment, that allows an individual to perform an action. • Gibson (’77) – perceived action possibilities • Norman – The Design of Everyday Things

39. Affordance Examples

40. Affordance Examples

41. Affordance Examples https://jbs2010.wordpress.com/2010/06/23/affordances-making-things-visible/

42. For your project • Look at the interface • What will people assume they can do with it? Write it down.

43. Tradeoff for new interfaces • Consider a military training simulator • How would you allow a user to user a gun in the simulator? Standard Device Low Affordance Low Cost High Reusability Engineered Device High Affordance High Cost Low Reusability

44. Novel Devices • Themes: • Make device more diverse • Users • Task • Improve match between task and device • Improve affordance • Refine input • Feedback strategies • Foot controls • Already used in music where hands might be busy • Cars • Foot mouse was twice as slow as hand mouse • Could specify ‘modes’ • Xk-a-75-r pedal switch

45. Novel Devices • Eye-tracking • Either worn by user or in the environment (e.g. Tobii) • Accuracy .4 degrees • Selections are by constant stare for 200-600 ms • How do you distinguish w/ a selection and a gaze? • video games, studying user behavior (1:41), design evaluation (pause 0:24) • Multiple degree of freedom devices • Logitech Spaceball and SpaceMouse • Ascension Bird • Polhemus Liberty and IsoTrack

46. Novel Devices • Boom Chameleon • Pros: Natural, good spatial understanding • Cons: limited applications, hard to interact (very passive). Not in production • Large simulators • DataGlove • Pinch glove • Gesture recognition • American Sign Language • Music • Pros: Natural • Cons: Size, hygiene, accuracy, durability

47. Novel Devices • Haptic Feedback • Why is resistance useful? • SensAble Technology’s Phantom, Novint Falcon • Cons: limited applications, computational complex (1 kHz update rate) • Sound and vibration can be a good approximation • Rumble pack • Two-Handed input • Different hands have different precision • Myron Kruger – novel user participation in art (Lots of exhibit art at siggraph)

48. Ubiquitous Computing and Tangible User Interfaces • Interacting with physical objects • https://www.youtube.com/watch?v=Rik8Z_TaxDw • Which sensors could you use? • Elderly, disabled • Research: Smart House http://www.linuxjournal.com/files/linuxjournal.com/linuxjournal/articles/030/3047/3047f2.png

49. Novel Devices • Paper/Whiteboards • Video capture of annotations • Record notes (special tracked pens Logitech digital pen) • Handheld Devices • Smartphones/PDA • Universal remote • Help disabled • Read LCD screens • Rooms in building • Maps • Interesting body-context-sensitive. • Ex. hold phone by ear = phone call answer.

50. Novel Devices • Miscellaneous • Shapetape – reports 3D shape. • Tracks limbs