1 / 51

Button Click Modeling on Mobile Phone

Button Click Modeling on Mobile Phone. November 11, 2009. Haptics and Virtual Reality Laboratory (http://hvr.postech.ac.kr) POSTECH. Table. Introduction Surveyed Papers Summary on Click Modeling. Introduction - Backgrounds. Mobile Phones with Physical Buttons

juana
Download Presentation

Button Click Modeling on Mobile Phone

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Button Click Modeling on Mobile Phone November 11, 2009 Haptics and Virtual Reality Laboratory (http://hvr.postech.ac.kr) POSTECH

  2. Table • Introduction • Surveyed Papers • Summary on Click Modeling

  3. Introduction - Backgrounds • Mobile Phones with Physical Buttons • Pros - Reliable buttons with tactile feedbacks • Cons - Small size of visual screen, and complex buttons without flexibility • Mobile Phones with Touchscreen • Pros - Large size of visual screen, and buttons adaptive to any application • Cons - Unreliable buttons with/without tactile feedbacks <Palm Treo 750 (left) and SPH-W3300 (right)> <iPhone (left) and Omnia2 (right)>

  4. Introduction - Related Works (1) • Fukumoto Masaaki Group (NTT DoCoMo) • One paper in CHI’01 • The first paper that introduced tactile feedback on touchscreen • Tactile feedbacks with a simple pulse • Measured task performance compared to beep signal • Ivan Poupyrev Group (Sony) • 2 papers in UIST’02 and UIST’03 • Designed and invented piezoelectric-cantilever actuator • Tactile feedbacks with the pulse modulations • Measured similarity with their own GUI

  5. Introduction - Related Works (2) • Andrew Nashel Group (Univ. North Carolina) • One paper in CHI’03 • Tactile feedbacks on a number-pad • Analyzed the tactile feedback scenario for the mobile devices • Pauli Laitinen Group (Nokia) • Two papers in HAID’06, HAVE’06, and two in ICMI’08 • Designed and invented piezoelectric-plate actuator • Tactile feedbacks based on the kinetic energy level • Measured similarity to the real button metaphor

  6. Introduction - Related Works (3) • Stephen Brewster Group (Univ. Glasgow) • 2 papers in CHI 2007, 2008 • Tactile feedbacks without click modeling • Click scenario with assigned tactile signals • Measured workload with NASA TLX and performance • Takashi Maeno Group (Univ. Keio) • Two papers in World Haptics’09 and Virtual and Mixed Reality’09 • Tactile feedbacks generated from real-button characteristics • Measured subjective responses varied on mechanical properties

  7. NTT DoCoMo, Active Click (1) Mechanism • Active Click • A mechanism to provide tactile feedback with any touch panel device <Body mounted actuator> <Panel mounted actuator> <Body mounted actuator for large display> Single Pulse Short Burst <Example of driving signal for click feel> Fukumoto Masaaki, and Sugimura Toshiaki, “Active Click: Tactile Feedback for Touch Panels”, CHI '01 extended abstracts on Human factors in computing systems, ACM, pp. 121-122, 2001.

  8. NTT DoCoMo, Active Click (2) Experiment • Experiment • Equipments • Palm TRG Pro with Palm-OS calculator • 10 participants • Independent Condition • Noise level: silent (40 dB), noisy (70 dB) • Click feedback: click (single pulse), beep (Palm OS default) • Task • Adding 5 pairs of 4-digit numbers • 20 trials on each condition • Measurement • Task completion time normalized to BS condition

  9. Sony, Ambient Touch – Actuator Design (1) • Mobile Tactile Display • There was no appropriate mobile tactile display • Multi-layered Piezoelectric Cantilever • Two-layer piezoelectric cantilever model • Low-voltage operation by multi-layering Force Displacement Input Voltage Thickness Piezoelectric Coefficient w, l: beam dimension <Piezoelectric cantilever actuator> I. Poupyrev, S. Maruyama, and J. Rekimoto, "Ambient touch: designing tactile interfaces for handheld devices," UIST '02: Proceedings of the 15th annual ACM symposium on User interface software and technology, ACM, pp. 51-60, 2002.

  10. Sony, Ambient Touch – Actuator Design (2) • Multi-layered Piezoelectric Cantilever(Cont’d) • Electric characteristic of piezoelectric cantilever is similar to a capacitor <Latency of 99% displacement on the piezoelectric cantilever> Input Voltage <Capacitive charging curve> <Displacement of piezoelectric cantilever>

  11. Sony, Ambient Touch – Hardware Systems • Equipments • Sony Clie PEG-N700C with amplification system • Two-axis accelerometer (ADXL202) for measuring tilt motion • Operation bandwidth: 5 Hz • Sampling bandwidth: 40 Hz

  12. Sony, Ambient Touch – Experiment (1) • Experiment Task • Experiment Condition • Independent Variables • Distance: 1 / 6 / 12 lines • Existence of tactile feedbacks • Dependent Variables • Completion time, overshoot <1D scrolling through text lists> <Design space of mobile tactile interfaces>

  13. Sony, Ambient Touch – Experiment (2) • Experiment Design • Ten male participants (19 ~ 35 years old) • Six sessions with seven trials each • The first trial was for warming up • Randomized six trials of three distances • Result

  14. Sony, Tactile Interfaces – Tactile Feedback System (1) • Goal • To verify the effect of tactile feedback for small screen with basic GUI elements • Hardware System • Four actuators at the corners of the touchscreen Click Soft click Springy <Tactile feedbacks> <Tactile interface concept for small touch screens> I. Poupyrev and S. Maruyama, "Tactile interfaces for small touch screens,“ UIST '03: Proceedings of the 16th annual ACM symposium on User interface software and technology. ACM, pp. 217-220, 2003.

  15. Sony, Tactile Interfaces – Tactile Feedback System (2) • Tactile feedbacks • Touch down (T1) • Button – click • Menu / scroll bar – springy • Dragging (T2) • Menu / scroll bar – click • Holding (T3) • Repeating button – click • Menu / scroll bar – soft click • Lift off (T4 and T5) • T4 – same as T1 • T5 – no tactile feedbacks <Structure of touch screen gesture>

  16. Univ. North Carolina,Tactile Virtual Buttons – Feedback Scenario • Goal • To add the tactile feedback of real buttons to virtual buttons with touch screen • Basic Technique • In the case of a pressure sensitive screen • Edge (A / C) • Pop vibration • Button (B) • Increasing frequency of vibration as the pressure increases • Pop vibration on the click situation • Linger Technique • In the case of a non-pressure sensitive screen • Linger time is used as the touch pressure A. Nashel and S. Razzaque, "Tactile virtual buttons for mobile devices,“ CHI '03 extended abstracts on Human factors in computing systems, ACM, pp. 854-855, 2003.

  17. Univ. North Carolina,Tactile Virtual Buttons – Experiment • Experiment Design • 12 participants • Result • Differentiation between button and no button was possible • Differentiation of the row was possible 40 Hz 200 Hz 250 Hz 350 Hz <Mobile phone used in the experiment>

  18. Nokia, Piezoelectric Actuator– Hardware System Design • Goal • Piezoelectric actuator design for haptic feedback and integration to the mobile device • Piezoelectric Actuator • Layered bender-actuator • Amplitudes • From few um to hundreds of um • Latency • ~ 10ms • Frequency • < 1 kHz <Piezoelectric actuator design in mobile device> <Example of single pulse shapes> P. Laitinen, and J. Maenpaa, “Enabling Mobile Haptic Deseign: Piezoelectric Actuator Technology Properties in Hand Held Devices” HAVE, IEEE International Workshop on Haptic Audio Visual Environments and their Applications, pp.40-43, 2006.

  19. Nokia, Piezoelectric Actuator– Tactile Feedback Design • Considerations • Different Holding Types • Varying Surroundings • Context of the Application <Different types of touch interaction>

  20. Nokia, Integration of Haptic and Audio– Hardware Systems • Goal • To find the perceived intensity of the haptic feedback generated by piezo actuators • Tactile Actuator • Bending piezo-electric actuator • Satisfies requirements in the previous paper <Nokia 770 with touch display> V. Tikka, and P. Laitinen, “Designing Haptic Feedback for Touch Display: Experimental Study of Perceived Intensity and Integration of Haptic and Audio“ HAID International Workshop on Haptic and Audio Interaction, pp. 36-44, 2006.

  21. Nokia, Integration of Haptic and Audio– Feedback Signals • Tactile Feedback • Impulse modulation of driving voltage • Amplitude • 3 ~ 180 um • Rising Time • 3 ~ 7 ms • Falling Time • 5 ms fixed • Audio Feedback • Intrinsic sound generated from tactile feedback • 28 ~ 60 dB at 35 cm distance from the device <Tactile stimuli>

  22. Nokia, Integration of Haptic and Audio– Experiment (1) • User Task • Pressing the 5 button on the virtual keypad while other hand holds the PDA • Experiment Condition • Independent Variables • Tactile feedback only / tactile and audio feedback • Dependent Variables • User rating on stimulus intensity (1, clearly too weak – 5, clearly too strong) • Experiment Design • 5 males and 3 females on each condition • 16 stimuli with three repetitions

  23. Nokia, Integration of Haptic and Audio– Experiment (2) • Results • Perceived intensity increases with integration of haptic and audio feedbacks

  24. Nokia, Feel Good Touch– Hardware Systems • Goal • Comparing the best pleasant vibrations with the piezoelectric actuator and the vibration motor • Tactile Actuator • Bending piezo-electric actuator • DC vibration motor <Touchscreen device with two actuators> E. Koskinen, T. Kaaresoja, and P. Laitinen, “Feel-Good Touch: Finding the Most Pleasant Tactile Feedback for a Mobile Touch Screen Button “ International Conference on Multimodal Interfaces, pp. 297-304, 2006.

  25. Nokia, Feel Good Touch– Experiments 1 • User Task • Pressing A and B buttons and selecting the more pleasant button • Experiment Condition • Independent Variables • Tactile feedback only / tactile and audio feedback • Dependent Variables • Frequency of preference • Experiment Design • Six males and four females (average 29 years old) • 21 stimuli pairs with two repetitions <Piezoelectric feedback parameters> <Frequency of preference of piezoelectric actuator>

  26. Nokia, Feel Good Touch– Experiments 2 • User Task • Pressing A and B buttons and selecting the more pleasant button • Experiment Condition • Independent Variables • Drive time • Dependent Variables • Frequency of preference • Experiment Design • Nine males and one females (average 28 years old) • 15 stimuli pairs with three repetitions <Vibration motor parameters> <Frequency of preference of vibration motor>

  27. Nokia, Feel Good Touch– Experiments 3 (1) • User Task • Input three digits appeared on the display and press # • Experiment Condition • Independent Variables • Vibration motor feedback / piezoelectric actuator feedback / no feedback

  28. Nokia, Feel Good Touch– Experiments 3 (2) • Experiment Condition(Cont’d) • Dependent Variables • Performance - completion time, average error rate • Subjective rating with 1 ~ 7 Likert scale • Result • Piezoelectric feedback showed shorter completion time and low error rate, with no statistical significance <Subjective ratings>

  29. Nokia, Crossmodal Congruence– Hardware Systems and Visual Feedback Design • Goal • To make a match of the best fit tactile feedback to the visual button • Visual Feedback • Extracted by analyzing ten kinds of real buttons • Shape • Circle / rectangular • Size • Small (6 x 4.5 mm) / Large (9 x 7 mm) • Height • Raised / flat <Nokia 770 internet tablet> <Example of raised and flat buttons> <Visual feedback to click event> E. Hoggan, T. Kaaresoja, P. Laitinen, and S. Brewster, “Crossmodal Congruence: The Look, Feel and Sound of Touchscreen Widgets“ International Conference on Multimodal Interfaces, pp. 157-164, 2006.

  30. Nokia, Feel Good Touch– Tactile Feedback Design • Tactile Feedback • Designed along with the kinetic energy • Piezoelectric actuator shakes 49 grams • Vibration motor shakes 183 grams • Audio Feedback Properties • Compared real keyboard sound and intrinsic vibration sound • Vibration motor feedback • Weaker sound intensity • Piezoelectric actuator feedback • Sharper sound intensity <Tactile feedback parameters> <Audio feedback parameters>

  31. Nokia, Feel Good Touch– Experiment (1) • Experiment Condition • Independent Variables • Eight visual buttons • Four tactile feedbacks • Dependent Variables • Votes • Subjective rating with Likert scale • Low quality – High quality • Experiment Design • Eight males and four females (23 ~ 47 years old) • 48 tasks with combination pairs of visual and tactile feedbacks <Experiment task interface on the Nokia 770>

  32. Nokia, Feel Good Touch– Experiment (2) • Result • Votes and average quality rating had a significant correlation of 0.79 with p = 0.05

  33. Nokia, Feel Good Touch– Summary • Circular touchscreen buttons are most congruent with • Small • Flat – soft piezo clicks • Raised – soft vibra clicks • Large • Flat / raised – soft vibra clicks • Rectangular touchscreen buttons are most congruent with • Small • Flat – short sharp piezo clicks • Raised – soft piezo clicks • Large • Flat – short sharp piezo clicks • Raised – soft vibra clicks

  34. Univ. Glasgow, Tactile Feedback – Hardware Systems and Tactile Feedback • Goal • To see if tactile displays can offer other benefits for touchscreen devices • Equipments • iPAQ with C2 tactor and stylus • Tactile Feedback • Successive click • 800 ms, 250 Hz sine wave • Error click • 800 ms, 250 Hz sine wave modulated with 30 Hz sine wave • Error case: slip, double taps • Small (6 x 4.5 mm) / Large (9 x 7 mm) <C2 tactor with iPAQ> S. Brewster, F. Chohan, and L. Brown, "Tactile feedback for mobile interactions,” CHI '07: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 159-162, 2007.

  35. Univ. Glasgow, Tactile Feedback – Experiment 1 • User Task • To type poems for 10 minutes on each condition with virtual keyboard in laboratory • Experiment Condition • Independent Variables • Tactile feedback / no feedback (standard) • Dependent Variables • Number of lines entered • Total errors • Number of errors uncorrected • Experiment Design • 12 right-handed participants

  36. Univ. Glasgow, Tactile Feedback – Experiment 2 • User Task • To type poems for 10 minutes on each condition with virtual keyboard seated on a train • Experiment Condition • Independent Variables • Tactile feedback / no feedback (standard) • Dependent Variables • Number of lines entered • Total errors • Number of errors uncorrected • NASA TLX • Experiment Design • 6 right-handed participants

  37. Univ. Glasgow, Tactile Feedback – Results • Tactile feedback reduces mental load in virtual keyboard typing <NASA TLX results for the mobile study> <Performance of two experiments>

  38. Univ. Glasgow, Mobile Touchscreens – Hardware Systems and Tactile Feedback • Equipments • A Palm Treo with a physical keyboard • A Samsung i718 with a LRA actuator • Two C2 Tactors for the vibration localization • Tactile Feedback • Fingertip-over on button edges • 300 ms, 250 Hz sine wave with ramp up/down time • Home keys • ‘F’, ‘J’ – 300 ms, 250 Hz modulated sine wave • Fingertip-click event • 30 ms, 175 square wave • Fingertip-slip event • 3-beat 500 ms 175 Hz modulated sine wave <A Palm Treo 750 and a Samsung i718> <Fingertip-over waveform> S. Brewster, F. Chohan, and L. Brown, "Tactile feedback for mobile interactions,” CHI '07: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 159-162, 2007.

  39. Univ. Glasgow, Tactile Feedback – Experiment 1 (1) • Experiment Condition • Independent Variables • Physical keyboard / tactile feedback / no feedback (standard) • Laboratory / train • Dependent Variables • Accuracy • Keystrokes per character • Completion time • NASA TLX • Experiment Design • Nine males and three females (18 ~ 38 years old) • Counterbalanced • 30 Phrases <User Task>

  40. Univ. Glasgow, Tactile Feedback – Experiment 1 (2)

  41. Univ. Glasgow, Tactile Feedback – Experiment 2 (1) • Experiment Condition • Independent Variables • Physical keyboard / tactile feedback / no feedback (standard) / direction cue added tactile feedback • Laboratory / train • Dependent Variables • Accuracy • Keystrokes per character • Completion time • NASA TLX • Experiment Design • 12 participants • Counterbalanced • 30 Phrases <Dell Axim PDA for two C2 tactors>

  42. Univ. Glasgow, Tactile Feedback – Experiment 2 (2)

  43. Univ. Keio, Realization of Button Click – Button Click Model and Experiment 1 • Push Button Model • Dominant Parameter to Click Feeling • 6 males <Buckling and restitution of dome> <F-S curve on real push buttons> <Subjective rating on real push buttons> K. Tashiro, Y. Shiokawa, T. Aono, and T. Maeno, “Realization of Button Click Feeling by Use of Ultrasonic vibration and Force Feedback” Third Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 1-6, 2009.

  44. Univ. Glasgow, Tactile Feedback – Experiment 2 • Dominant Parameter Values to Click Feeling • Four males and two females • Evaluated each term with -2 (very small) to 2 scale (very large) <Force and stroke parameters of real buttons> <Correlation coefficient of force and stroke between sensory score and F-S characteristic value>

  45. Univ. Glasgow, Tactile Feedback – Experiment 3 (1) • Equipments • Human finger slip condition • -0.1 < coefficient of dynamic friction < 0.1 • > 5 um in vibration amplitude level <Overall view of push button display (left) and ultrasonic vibrator (right)> <Displaying area (left) and decreasing coefficient of dynamic friction (right)>

  46. Univ. Glasgow, Tactile Feedback – Experiment 3 (2) • Experiment Condition • Independent variables • Vibration amplitude, vibration time, buckling force • Dependent variables • Subjective rating with 5 scale (1 – not similar, 5 – similar) • Experiment Design • Six males • Results

  47. Univ. Glasgow, Tactile Feedback – Experiment 4 • Independent variables were used with orthogonal array method <Variance of each sensory evaluation score of the 3 levels>

  48. Summary on Click Modeling • Vibration Rendering • Short impulse or short sine wave signal • NTT DoCoMo, Sony, Nokia, Univ. Glasgow • Sine wave or modulated signal • Univ. North Carolina, Univ. Glasgow • Friction Force Rendering • Ultrasonic actuator with squeeze film effect • Univ. Keio

  49. Appendix A. Capacitor Charge Time • Capacitor Charge • Vc(0) = 0 at capacitor, and i(t) = R/V0 at t = 0 <DC RC circuit>

  50. Appendix B. NASA TLX - Definitions • Designed to measure mental workload for pilots • Dimensions • Each of the 6 dimensions is rated on a 20-step bipolar scale • Mental demand (MD) • Physical demand (PD) • Temporal demand (TD) • Performance (OP) • Effort (EF) • Frustration (FR)

More Related