1 / 46

Interaction Devices

Interaction Devices. Human Computer Interaction CIS 6930/4930 Section 4188/4186. Interaction Performance. 60s vs. Today Performance Hz -> GHz Memory k -> GB Storage k -> TB Input punch cards -> Keyboards, Pens, tablets, mobile phones, mice, digital cameras, web cams Output

erma
Download Presentation

Interaction Devices

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Interaction Devices Human Computer Interaction CIS 6930/4930 Section 4188/4186

  2. Interaction Performance • 60s vs. Today • Performance • Hz -> GHz • Memory • k -> GB • Storage • k -> TB • Input • punch cards -> • Keyboards, Pens, tablets, mobile phones, mice, digital cameras, web cams • Output • 10 character/sec • Megapixel displays, color laser, surround sound, force feedback, VR • Substantial bandwidth increase!

  3. Interaction Performance • Future? • Gestural input • Two-handed input • 3D I/O • Others: voice, wearable, whole body, eye trackers, data gloves, haptics, force feedback • Engineering research! • Entire companies created around one single technology • Current trend: • Multimodal (using car navigation via buttons or voice) • Helps disabled (esp. those w/ different levels of disability)

  4. Keyboard and Keypads • QWERTY keyboards been around for a long time • (1870s – Christopher Sholes) • Cons: Not easy to learn • Pros: Familiarity • Stats: • Beginners: 1 keystroke per sec • Average office worker: 5 keystrokes (50 wpm) • Experts: 15 keystrokes per sec (150 wpm) • Is it possible to do better? Suggestions?

  5. Keyboard and Keypads • Look at the piano for possible inspiration • Court reporter keyboards (one keypress = multiple letters or a word) • 300 wpm, requires extensive training and use • Keyboard properties that matter • Size • large - imposing for novices, appears more complex • mobile devices • Adjustable • Reduces RSI, better performance and comfort • Mobile phone keyboards, blackberry devices, etc.

  6. Keyboard Layouts • QWERTY • Frequently used pairs far apart • Fewer typewriter jams • Electronic approaches don’t jam.. why use it? • DVOARK (1920s) • 150 wpm->200 wpm • Reducing errors • Takes about one week to switch • Stops most from trying • ABCDE – style • Easier for non-typists • Studies show no improvement vs. QWERTY • Number pads • What’s in the top row? • Look at phones (slight faster), then look at calculators, keypads • Those for disabled • Split keyboards • KeyBowl’s orbiTouch (screenshot) • Eyetrackers, mice • Dasher - 2d motion with word prediction

  7. Keys • Current keyboards have been extensively tested • Size • Shape • Required force • Spacing • Speed vs. error rates for majority of users • Distinctive click gives audio feedback • Why membrane keyboards are slow (Atari 400?) • Environment hazards might necessitate • Usually speed is not a factor

  8. Keys Guidelines • Special keys should be denoted • State keys (such as caps, etc.) should have easily noted states • Special curves or dots for home keys for touch typists • Inverted T Cursor movement keys are important (though cross is easier for novices) • Auto-repeat feature • Improves performance, but only if repeat is customizable (motor impaired, young, old) • Two thinking points: • Why are home keys fastest to type? • Why are certain keys larger? (Enter, Shift, Space bar) • This is called Fitt’s Law

  9. Keypads for small devices • PDAs, Cellphones, Game consoles • Fold out keyboards • Virtual keyboard • Cloth keyboards (ElekSen) • Haptic feedback? • Mobile phones • Combine static keys with dynamic soft keys • Multi-tap a key to get to a character • Study: Predictive techniques greatly improve performance • Ex. LetterWise = 20 wpm vs 15 wpm multitap • Draw keyboard on screen and tap w/ pen • Speed: 20 to 30 wpm (Sears ’93) • Handwriting recognition (still hard) • Subset: Graffiti2 (uses unistrokes)

  10. Pointing Devices • Direct manipulation needs some pointing device • Factors: • Size of device • Accuracy • Dimensionality • Interaction Tasks: • Select – menu selection, from a list • Position – 1D, 2D, 3D (ex. paint) • Orientation – Control orientation or provide direct 3D orientation input • Path – Multiple poses are recorded • ex. to draw a line • Quantify – control widgets that affect variables • Text – move text • Faster w/ less error than keyboard • Two types (Box 9.1) • Direct control – device is on the screen surface (touchscreen, stylus) • Indirect control – mouse, trackball, joystick, touchpad

  11. Direct-control pointing • First device – lightpen • Point to a place on screen and press a button • Pros: • Easy to understand and use • Very fast for some operations (e.g. drawing) • Cons: • Hand gets tired fast! • Hand and pen blocks view of screen • Fragile • Evolved into the touchscreen • Pros: Very robust, no moving parts • Cons: Depending on app, accuracy could be an issue • 1600x1600 res with acoustic wave • Must be careful about software design for selection (land-on strategy). • If you don’t show a cursor of where you are selecting, users get confused • User confidence is improved with a good lift-off strategy

  12. Direct-control pointing • Primarily for novice users or large user base • Case study: Disney World • Need to consider those who are: disabled, illiterate, hard of hearing, errors in usage (two touch points), etc.

  13. Indirect-Control Pointing • Pros: • Reduces hand-fatigue • Reduces obscuration problems • Cons: • Increases cognitive load • Spatial ability comes more into play • Mouse • Pros: • Familiarity • Wide availability • Low cost • Easy to use • Accurate • Cons: • Time to grab mouse • Desk space • Encumbrance (wire), dirt • Long motions aren’t easy or obvious (pick up and replace) • Consider, weight, size, style, # of buttons, force feedback

  14. Indirect-Control Pointing • Trackball • Pros: • Small physical footprint • Good for kiosks • Joystick • Easy to use, lots of buttons • Good for tracking (guide or follow an on screen object) • Does it map well to your app? • Touchpoint • Pressure-sensitive ‘nubbin’ on laptops • Keep fingers on the home position

  15. Indirect-Control Pointing • Touchpad • Laptop mouse device • Lack of moving parts, and low profile • Accuracy, esp. those w/ motor disabilities • Graphics Tablet • Screen shot • comfort • good for cad, artists • Limited data entry

  16. Comparing pointing devices • Direct pointing • Study: Faster but less accurate than indirect (Haller ’84) • Lots of studies confirm mouse is best for most tasks for speed and accuracy • Trackpoint < Trackballs & Touchpads < Mouse • Short distances – cursor keys are better • Disabled prefer joysticks and trackballs • If force application is a problem, then touch sensitive is preferred • Vision impaired have problems with most pointing devices • Use multimodal approach or customizable cursors • Read Vanderheiden ’04 for a case study • Designers should smooth out trajectories • Large targets reduce time and frustration

  17. Example • Five fastest places to click on for a right-handed user?

  18. Example • What affects time?

  19. Fitts’s Law • Paul Fitts (1954) developed a model of human hand movement • Used to predict time to point at an object • What are the factors to determine the time to point to an object? • D – distance to target • W – size of target • Just from your own experience, is this function linear? • No, since if Target A is D distance and Target B is 2D distance, it doesn’t take twice as long • What about target size? Not linear there either • MT = a + b log2(D/W + 1) • a = time to start/stop in seconds (empirically measured per device) • b = inherent speed of the device (empirically measured per device) • Ex. a = 300 ms, b = 200 ms/bit, D = 14 cm, W = 2 cm • Ans: 300 + 200 log2(14/2 + 1) = 900 ms

  20. Fitts’s Law • MT = a + b log2(D/W + 1) • a = time to start/stop in seconds (empirically measured per device) • b = inherent speed of the device (empirically measured per device) • Ex. a = 300 ms, b = 200 ms/bit, D = 14 cm, W = 2 cm • Ans: 300 + 200 log2(14/2 + 1) = 900 ms • Question: If I wanted to half the pointing time (on average), how much do I change the size? • Proven to provide good timings for most age groups • Newer versions taken into account • Direction (we are faster horizontally than vertically) • Device weight • Target shape • Arm position (resting or midair) • 2D and 3D (Zhai ’96)

  21. Very Successfully Studied • Applies to • Feet, eye gaze, head mounted sights • Many types of input devices • Physical environments (underwater!) • User populations (even retarded and drugged) • Drag & Drop and Point & Click • Limitations • Software accelerated pointer motion • Training • Trajectory Tasks (Accot-Zhai Steering Law) • Decision Making (Hick’s Law) • Results (what does it say about) • Buttons and widget size? • Edges? • Popup vs. pull-down menus • Pie vs. Linear menus

  22. Precision Pointing Movement Time • Study: Sears and Shneiderman ’91 • Broke down task into gross and fine components for small targets • PPMT = a + b log2(D/W+1) + c log2(d/W) • c – speed for short distance movement • d – minor distance • Notice how the overall time changes with a smaller target. • Other factors • Age (Pg. 369) • Research: How can we design devices that produce smaller constants for the predictive equation • Two handed • Zooming

  23. Novel Devices • Themes: • Make device more diverse • Users • Task • Improve match between task and device • Improve affordance • Refine input • Feedback strategies • Foot controls • Already used in music where hands might be busy • Cars • Foot mouse was twice as slow as hand mouse • Could specify ‘modes’

  24. Novel Devices • Eye-tracking • Accuracy 1-2 degrees • selections are by constant stare for 200-600 ms • How do you distinguish w/ a selection and a gaze? • Combine w/ manual input • Multiple degree of freedom devices • Logitech Spaceball and SpaceMouse • Ascension Bird • Polhemus Liberty and IsoTrack

  25. Novel Devices • Boom Chameleon • Pros: Natural, good spatial understanding • Cons: limited applications, hard to interact (very passive) • DataGlove • Pinch glove • Gesture recognition • American Sign Language, musical director • Pros: Natural • Cons: Size, hygiene, accuracy, durability

  26. Novel Devices • Haptic Feedback • Why is resistance useful? • SensAble Technology’s Phantom • Cons: limited applications • Sound and vibration are easier and can be a good approximation • Rumble pack • Two-Handed input • Different hands have different precision • Non-dominant hand selects fill, the other selects objects • Ubiquitous Computing and Tangible User Interface • Active Badges allows you to move about the house w/ your profile • Which sensors could you use? • Elderly, disabled • Research: Smart House • Myron Kruger – novel user participation in art (Lots of exhibit art at siggraph)

  27. Novel Devices • Paper/Whiteboards • Video capture of annotations • Record notes (special tracked pens Logitech digital pen) • Handheld Devices • PDA • Universal remote • Help disabled • Read LCD screens • Rooms in building • Maps • Interesting body-context-sensitive. • Ex. hold PDA by ear = phone call answer.

  28. Novel Devices • Miscellaneous • Shapetape – reports 3D shape. • Tracks limbs • Engineer for specific app (like a gun trigger connected to serial port) • Pros: good affordance • Cons: Limited general use, time

  29. Speech and Auditory Interfaces • There’s the dream • Then there’s reality • Practical apps don’t really require freeform discussions with a computer • Goals: • Low cognitive load • Low error rates • Smaller goals: • Speech Store and Forward (voice mail) • Speech Generation • Currently not too bad, low cost, available

  30. Speech and Auditory Interfaces • Bandwidth is much lower than visual displays • Ephemeral nature of speech (tone, etc.) • Difficulty in parsing/searching (Box 9.2) • Types • Discrete-word recognition • Continuous speech • Voice information • Speech generation • Non-speech auditory • If you want to do research here, lots of research in the audio, audio psychology, and DSP field you should understand

  31. Discrete-Word Recognition • Individual words spoken by a specific person • Command and control • 90-98% for 100-10000 word vocabularies • Training • Speaker speaks the vocabulary • Speaker-independent • Still requires • Low noise operating environment • Microphones • Vocabulary choice • Clear voice (language disabled are hampered, stressed) • Reduce most questions to very distinct answers (yes/no)

  32. Discrete-Word Recognition • Helps: • Disabled • Elderly • Cognitive challenged • User is visually distracted • Mobility or space restrictions • Apps: • Telephone-based info • Study: much slower for cursor movement than mouse or keyboard (Christian ’00) • Study: choosing actions (such as drawing actions) improved performance by 21% (Pausch ’91) and word processing (Karl ’93) • However acoustic memory requires high cognitive load (> than hand/eye) • Toys are successful (dolls, robots). Accuracy isn’t as important • Feedback is difficult

  33. Continuous Speech Recognition • Dictation • Error rates and error repair are still poor • Higher cognitive load, could lower overall quality • Why is it hard? • Recognize boundaries (normal speech blurs them) • Context sensitivity • “How to wreck a nice beach” • Much training • Specialized vocabularies (like medical or legal) • Apps: • Dictate reports, notes, letters • Communication skills practice (virtual patient) • Automatic retrieval/transcription of audio content (like radio, CC) • Security/user ID

  34. Voice Information Systems • Use human voice as a source of info • Apps: • Tourist info • Museum audio tours • Voice menus (Interactive Voice Response IVR systems) • Use speech recognition to also cut through menus • If menus are too long, users get frustrated • Cheaper than hiring 24 hr/day reps • Voice mail systems • Interface isn’t the best • Get email in your car • Also helps with non-tech savvy like the elderly • Potentially aides with • Learning (engage more senses) • Cognitive load (hypothesize each sense has a limited ‘bandwidth’) • Think ER, or fighter jets

  35. Speech Generation • Play back speech (games) • Combine text (navigation systems) • Careful evaluation! • Speech isn’t always great • Door is ajar – now just a tone • Use flash • Supermarket scanners • Often times a simple tone is better • Why? Cognitive load • Thus cockpits and control rooms need speech • Competes w/ human-human communication

  36. Speech Generation • Ex: Text-to-Speech (TTS) • Latest TTS uses multiple syllabi to make generated speech sound better • Robotic speech could be desirable to get attention • All depends on app • Thus don’t assume one way is the best, you should user test • Apps: TTS for blind, JAWS • Web-based voice apps: VoiceXML and SALT (tagged web pages). • Good for disabled, and also for mobile devices • Use if • Message is short • Requires dynamic responses • Events in time • Good when visual displays aren’t that useful. When? • Bad lighting, vibrations (say liftoff)

  37. Non-speech Auditory Interface • Audio tones that provide information • Major Research Area • Sonification – converting information into audio • Audiolization • Auditory Interfaces • Browsers produced a click when you clicked on a link • Increases confidence • Can do tasks without visual cognitive load • Helps figure out when things are wrong • Greatly helps visually impaired

  38. Non-speech Auditory Interface • Terms: • Auditory icons – familiar sounds (record real world sound and play it in your app) • Earcons – new learned sounds (door ajar) • Role in video games is huge • Emotions, Tension, set mood • To create 3D sound • Need to do more than stereo • Take into account Head-related transfer function (HRTF) • Ear and head shape • New musical instruments • Theremin • New ways to arrange music

  39. Displays • Primary Source of feedback • Properties: • Physical Dimension • Resolution • Color Depth and correctness • Brightness, contrast, glare • Power • Refresh rate • Cost • Reliability • # of users

  40. Display Technology • Monochrome displays (single color) • Low cost • Greater intensity range (medical) • Color • Raster Scan CRT • LCD – thin, bright • Plasma – very bright, thin • LED – large public displays • Electronic Ink – new product w/ tiny capsules of negative black particles and positive white • Braille – refreshable cells with dots that rise up

  41. Large Displays • Wall displays • Informational • Control rooms, military, flight control rooms, emergency response • Provides • System overview • Increases situational awareness • Effective team review • Old: Array of CRTs • Interactive • Require new interaction methods (freehand sketch, PDAs) • Local and remote collaboration • Art, engineering

  42. Large Displays • Multiple Desktop Displays • Multiple CRTs or Flat panels for large desktops • Cheap • Familiar • Spatial divide up tasks • Comparison tasks are easier • Too much info? • HMD • Eventually -> Every surface a pixel

  43. Mobile device displays • Applications • Personal • Reprogrammable picture frames • Digital family portrait (GaTech) • Business • PDAs, cellphones • Medical • Monitor patients • Research: Modality Translation Services (Trace Center – University of Wisconsin) • As you move about it auto converts data, info, etc. for you

  44. Mobile device displays • Actions on mobile devices • Monitor information and alert (calendar) • Gather then spread out information (phone) • Participate in groups and relate to individual (networked devices) • Locate services and identify objects (GPS car system) • Capture and then share info (phone)

  45. Mobile device displays • Guidelines for design • Bergman ’00, Weiss, ’02 • Industry led research and design case studies (Lindholm ’03) • Typically short in time usage (except handheld games) • Optimize for repetitive tasks (rank functions by frequency) • Research: new ways to organize large amounts of info on a small screen • Study: Rapid Serial Visual Presentation (RSVP) presents text at a constant speed (33% improvement Oquist ’03) • Searching and web browsing still very poor performance • Promising: Hierarchical representation (show full document and allow user to select where to zoom into)

  46. Animation, Image, and Video • Content quality has also greatly increased • 3D rendering is near life-like • Digital Photography is common • Scanned documents • Video compression • Multimedia considerations for the disabled • Printers • 3D Printers create custom objects from 3D models

More Related