Mobile usability testing
Download
1 / 52

- PowerPoint PPT Presentation


  • 226 Views
  • Updated On :

Lecture :: „Mobile Usability Testing“. Mobile Usability Testing. Topics Motivation & Definition Mobile Devices Challenges in Mobile Usability Testing Methods & Approaches Heuristics & Guidelines Examples & Ideas. Inst. f. Softwaretechnik und Interaktive Systeme

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '' - opa


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Mobile usability testing l.jpg

Lecture :: „Mobile Usability Testing“

Mobile Usability Testing

  • Topics

  • Motivation & Definition

  • Mobile Devices

  • Challenges in Mobile Usability Testing

  • Methods & Approaches

  • Heuristics & Guidelines

  • Examples & Ideas

Inst. f. Softwaretechnik und Interaktive Systeme

qse.ifs.tuwien.ac.at, [email protected], [email protected]


Mobile usability testing2 l.jpg

Goals ::What do we want to achieve in this lecture?

Mobile Usability Testing

  • What we want:

  • Enable you to set up a Mobile Usability Test based on Quantitative as well as Qualitative Methods.

  • Give you Inspirations & Ideas about what can be done to make mobile applications more usable.

*vgl.: Schilit and Theimer (1994)

2


Mobile usability testing3 l.jpg

Methods ::How do we want to achieve our goals?

  • How we want to do it:

  • Creating an awareness for the challenges of mobile-Usability

  • Compare common Usability Testing Methods

  • Overview about possible Heuristics & Guidelines for Mobile Usability Testing

Mobile Usability Testing

*vgl.: Schilit and Theimer (1994)

3


I motivation definition l.jpg

Definition ::„Mobile Usability Testing“

I. Motivation & Definition

  • Considerations for Mobile Testing:

  • Results are strongly influenced by surrounding environment

  • Results are influenced by devices used for testing

  • Collected Data will be „richer“ (Gesture, Voice, …)

4



I motivation definition6 l.jpg

Costs of late Usability research

I. Motivation & Definition

For every dollar spent acquiring a customer you will spend $100 dollars reacquiring them after they leave because of poor usability or bad customer service. (*)

*vgl.: MauroNewMedia (2002)

6


I motivation definition7 l.jpg

Usability Engineering in the Software Development Process

I. Motivation & Definition

*vgl.: INTERACT 2001 Workshop, Jan Gulliksen, Inger Boivie)

7


Mobile usability testing8 l.jpg

Mobile Usability Testing

II. MobileDevices


Ii mobile devices l.jpg

Mobile Devices :: Definition

Are used “on -the-run and for activities that may last only a few seconds or are highly context dependent” (*)

II. Mobile Devices

*vgl.: (Vetere et al., 2003, p.1)

9


Ii mobile devices10 l.jpg

Mobile Devices :: Realistic View 1

  • „Mobile Devices: One Generation From Useful“(*)

  • Tighter Integration needed:

  • Devices do not work together well with each other. Synchronization with other Applications

  • Different Features packed into one device do notact as one entity  PDAs with GSM modules

  • Ad-hoc Networking is still far from realisation Network coverage & Roaming (WLAN, UMTS, GPRS, GSM) Industry Standards (Bluetooth compatibility, vulnerability)

II. Mobile Devices

*vgl.: (Jakob Nielsen 2003, Alertbox , Aug. 18th.)

10


Ii mobile devices11 l.jpg

Mobile Devices :: Realistic View 2

  • Design / Interface Problems still to be solved:

  • Deck-of-Card Size(*) limits the Screen Size Higher Screen-Resolutions, better viewing angle.

  • One Dimensional Interfaces (Scroll wheels) are not suitablefor 2D – Screens.

  • Text-input is still a great Problem.Small Devices  small Buttons.  new Button Alignments?

II. Mobile Devices

*vgl.: (Jakob Nielsen 2003, Alertbox , Aug. 18th.)

11


Ii mobile devices12 l.jpg

Mobile Devices :: Realistic View 3

  • Fundamental Problems:

  • Quality of Service of local Network Providers.Things that „could be done“ just can‘t because of local Service-Limitations or lack of network coverage. (Broadband, UMTS, etc.)

  • Online Services must specialize for Mobile useMuch shorter Articles, more use of XML, simplyfied Navigation

  • Reconsider the way email is used  not just forward every mail to the PDA (Attachments, Executeables)

II. Mobile Devices

12


Mobile usability testing13 l.jpg

Mobile Usability Testing

III. Challenges


Iii challenges l.jpg

Challenges ::of Mobile Usability Testing (*)

  • 1. Device Proliferation

    • Handling many different Devices, Rendering Methods

  • 2. Application Modality

    • Handling simultaneous voice / Data User interactions

  • 3. User Mobility

    • Users are likely to be distracted during use

  • 4. Data Collection

    • Recording eye-movement and video taping will not work everywhere

III. Challenges

*vgl.: http://www.littlespringsdesign.com/analysis/utest.html

14


Iii challenges15 l.jpg

Challenges :: 1 : Device Proliferation (*)

  • Devices can be

  • As small as possible, optimized for voice communication

  • Quite large, optimized for data display

  • Optimized for gaming

  • Optimized for multimedia

III. Challenges

  • Applications are perceived differently

  • Reading News-Bulletin on a point-matrix phone display  content is forgotten 3 hours later.

  • Reading News-Bulletin on a java-enabled 19“ CRT Monitor Will be stored in long-term memory

*vgl.: http://www.littlespringsdesign.com/analysis/utest.html

15


Iii challenges16 l.jpg

Challenges :: 2 : Application Modality(*)

  • Mobile Applications often combine Graphic / Data and Voice Elements.

  • Difficult to test in an early stage of development(Software isn‘t fully functional / not yet written.)

  • Test must be able to provide simultaneous experiences

III. Challenges

*vgl.: http://www.littlespringsdesign.com/analysis/utest.html

16


Iii challenges17 l.jpg

Challenges :: 3 : User Mobility

  • Mobile Users are very likely to be distracted.

  • Natural Environment is not always manageable/affordable to be simulated in a lab

  • Distractions and „Normal anomalies“ (Waiter interrupting you in a restaurant to take your order) have to be part of the test

III. Challenges

17


Iii challenges18 l.jpg

Challenges :: 4: Data Collection 1

  • Acquiring Data outside the Lab

  • Eye tracking will hardly work on tiny screens and under mobile conditions.

  • Recording tools interfere with the users‘ interaction with the device.(Cameras mounted on a cell phone make the user hold it in an unnatural position)

III. Challenges

“It is cold and snowing and you do not know from where your bus leaves in 5 minutes. You pick up your WAP phone to check: The mobile user runs to catch her bus, after her run three researchers with cameras and microphones..(*)“

*(vgl.: Per-Ola Rasmussen ExarbII 2003)

18


Iii challenges19 l.jpg

Challenges :: 4: Data Collection 2

  • Mobile Users interact not just with the screen and keyboard.

  • Test needs to record, gestures, face-expressions, voice, body-language, etc.

  •  Much richer Data

    (People leaning left and right while playing a Formula-1 racing Game on their Java-enabled Phone.)

  • Record what is the user doing, what is he/she NOT doing.

III. Challenges

19


Mobile usability testing20 l.jpg

Mobile Usability Testing

IV. Methods & Approaches


Iv methods approaches l.jpg

Methods :: Approaches to Mobile Usability Testing 1

  • The common Methods

IV. Methods & Approaches

*(vgl.: Tomas Lindroth, Stefan Nilsson & Per-Ola Rasmussen, ExarbII – HT2000)

21


Iv methods approaches22 l.jpg

Methods :: Approaches to Mobile Usability Testing 2

  • The common Methods

IV. Methods & Approaches

*(vgl.: Tomas Lindroth, Stefan Nilsson & Per-Ola Rasmussen, ExarbII – HT2000)

22


Iv methods approaches23 l.jpg

Methods :: Approaches to Mobile Usability Testing 3

  • The common Methods

IV. Methods & Approaches

*(vgl.: Tomas Lindroth, Stefan Nilsson & Per-Ola Rasmussen, ExarbII – HT2000)

23


Iv methods approaches24 l.jpg

Methods :: Approaches to Mobile Usability Testing 4

  • The common Methods

IV. Methods & Approaches

*(vgl.: Tomas Lindroth, Stefan Nilsson & Per-Ola Rasmussen, ExarbII – HT2000)

24


Iv methods approaches25 l.jpg

Methods :: Lab vs. natural environment

The larger the number of factors that is under control in a test, the more scientific rigour is emphasized. The more natural like the test setting is, the more relevant and applicable the results will be.

IV. Methods & Approaches

*(vgl.: Mason 1988, Järvinen, 1999)

25


Iv methods approaches26 l.jpg

Methods :: Approaches to Mobile Usability Testing 5

  • The Ideal Test

  • Natural situation / environment

  • Application is fully functional

  • All possible forms of devices are being tested

  • Users are free to do what they would normally doUsers don't feel „tested“

  • Tester can record every audio / visual / voice / movement / screen datafrom the user without affecting the users behavior.

  • Tester sees the Application the way the users see it.

IV. Methods & Approaches

26


Iv methods approaches27 l.jpg

Methods :: General Mobile Usability Testing Process 1

  • Preparation

  • Define goals, methods and tasks/scenarios for the test.

    Introduction, Warm-up

  • Introduce the test to the user

  • Start with easier tasks, give time for a short warm-up phase.

    Testing

  • Perform the actual test with as less interaction between user and tester as possible

IV. Methods & Approaches

27


Iv methods approaches28 l.jpg

Methods :: General Mobile Usability Testing Process 2

Test situation

  • Give the user time to get out of the test-situation.

  • Then start reviewing his/her opinions, impressions and suggestions.

  • Make sure to discuss special occurrences that may have happened during the test with the user.

IV. Methods & Approaches

28


Iv methods approaches29 l.jpg

Remember ::Challenges to deal with (*)

  • 1. Device Proliferation

  • 2. Application Modality

  • 3. User Mobility

  • 4. Data Collection

IV. Methods & Approaches

*vgl.: http://www.littlespringsdesign.com/analysis/utest.html

29


Iv methods approaches30 l.jpg

Methods :: Handling User Mobility 1

  • For informal, problem identifying tests

  • Ask participants to use the application maybe over lunch

  • Offering a compensation helps „motivating“ the participant

  • Don‘t forget to have them sign an informed consent statement

  • Interruptions (waiter, etc.) are welcome Watch what happens when the users resumes the task and see what difficulties occur.

IV. Methods & Approaches

30


Iv methods approaches31 l.jpg

Methods :: 3 : Handling User Mobility 2

  • For formal, statistically precise tests

  • Don‘t try to introduce distractions into the test unless you are testing with a greater number of participants

  • Referring to Nielson, the marginal benefit will decrease if you are testing with more then 10 UsersThesis and formula is questioned from many researchers.

IV. Methods & Approaches

31


Iv methods approaches32 l.jpg

Methods :: 3 : Simulate the natural environment? 1

  • Chances(*):

  • Reproducible conditions

  • Easier / more complete documentation

  • Use of more sophisticated tools

    Risks

  • Unusual environment for the user

  • Restrictions due to simulation

  • Non recording of the original work surrounding field (office atmosphere, disturbances, etc.)

IV. Methods & Approaches

*(vgl.: akziv. Requirements from users point of view. 2004)

32


Iv methods approaches33 l.jpg

Interactive Example

  • Which environmental factors are reproduceable in a laboratory surrounding.

    Environmental Conditions BT 041118.xls

IV. Methods & Approaches

33


Iv methods approaches34 l.jpg

Methods :: 3 : Simulate the natural environment? 2

  • „Mobile Devices are build to be mobile so take them out into the field“ (*)

  • Take the lab to the user, not the user to the lab.

IV. Methods & Approaches

*(vgl.: akziv, „wearability“. 2004

34


Iv methods approaches35 l.jpg

Methods :: 3 : Lab Test vs. Field Test(*) 1

  • Example Application: Using SMS Service on a PDA while walking

  • In the lab: on a treadmill

  • In the field: on a pedestrian street

IV. Methods & Approaches

*(vgl.: Jesper Kjeldskov, Aalborg University Denmark)

35


Iv methods approaches36 l.jpg

Methods :: 3 : Lab Test vs. Field Test(*) 2

IV. Methods & Approaches

*(vgl.: Jesper Kjeldskov, Aalborg University Denmark)

36


Iv methods approaches37 l.jpg

Methods :: 3 : Lab Test vs. Field Test(*) 3

Usability problems identified by the test subjects

  • Numbers are basically equal

  • Notice that while sitting on a desk the cosmetic problemsidentified by the users were far more!

IV. Methods & Approaches

*(vgl.: Jesper Kjeldskov, Aalborg University Denmark)

37


Iv methods approaches38 l.jpg

Methods :: 3 : Lab Test vs. Field Test(*) 4

Are Lab tests superior?

  • Consider Cost / benefit of different techniques and settingsTime and effort per problem found

  • Can you afford NOT to find a problem?

  • Costs of missing Usability:

    • lost of repurchases

    • increased calls at helpdesk

    • lost of repurchases

    • lost of brand reputation

    • necessary redesign in late state or next version

    • law suits

    • ……

IV. Methods & Approaches

*(vgl.: Jesper Kjeldskov, Aalborg University Denmark)

38


Iv methods approaches39 l.jpg

Methods :: 4 : Handling Data Collection 1

  • Mobile devices are extremely personal. Users may pick them up, gesture, or lean back with them.(*)

  • Record Device Screen + Users Face at the same time

  • Use wireless tracking & recording technology(WLAN, Bluetooth, small radio cameras, etc.)

  •  Users should not be handicapped by the testing equipment

IV. Methods & Approaches

*(vgl.: Little Springs Inc. 2004)

39


Iv methods approaches40 l.jpg

Methods :: 4 : Handling Data Collection 2

  • Recording a mobile phone‘s screen and the users face at the same time with two cameras mounted on the phone(*)

IV. Methods & Approaches

*(vgl.: Little Springs Inc. 2004)

40


Iv methods approaches41 l.jpg

Methods :: 4 : Handling Data Collection 3

  • Recording „Soft Information“

  • Define methods to integrate „soft-information“E. g.: Users starts shaking the Phone to make it start the application faster; user‘s thumb tends to cover up parts of the device‘s display)

  • Many qualitative information can be quantized.

    • Measuring heart-beats / second to determine the stress-level

    • Recording subconscious hand and leg movements.

    • Record number of extra-applicational interactions(user answered 2 phone calls and asked his colleague for help during the use of the application for 5. min.)

IV. Methods & Approaches

*(vgl.: Little Springs Inc. 2004)

41


Mobile usability testing42 l.jpg

Mobile Usability Testing

V. Possible Heuristics & Guidelines


V heuristics guidelines l.jpg

Heuristics :: General Guidelines for mobile Applications 1

  • Highly functional design

  • Don‘t use fancy designs if they don‘t bring a real benefit for the User

  • Consistent usage of icons / buttons / names and labels

  • Consider the users mental-models when you introduce new functions or name buttons.

  • Integrated content navigation

  • Help the user tracking it‘s way through the information, always provide a clear exit – point.

  • Consider Shortcuts

V. Heuristics & Guidelines

*(vgl.: Little Springs Inc. 2004)

43


V heuristics guidelines44 l.jpg

Heuristics :: GeneralGuidelines for mobile Applications 2

  • Reduced HCI interactions

  • Especially when done on mobile devices interactions with the user are often difficult and time-consuming (text-input on a mobile phone)

  • Reduce Interactions by any means possible (Location based services, Heuristics, default-values, etc.)

  • Offer intelligent search-functions

  • Assist the user in finding the information, as any unnecessary interaction makes the applications less usable for the user.

V. Heuristics & Guidelines

*(vgl.: Little Springs Inc. 2004)

44


V heuristics guidelines45 l.jpg

Heuristics :: Mobile Games 1

  • For Mobile Games the Rules are

  • a little different

  • Navigation Consistency?

  • The User should not feel like using his/her phone, he/she should experience the Game World

V. Heuristics & Guidelines

*(vgl.: NOKIASeries 60 Developer Platform 2.0: Usability Guidelines For J2ME™ Games

45


V heuristics guidelines46 l.jpg

Heuristics :: Mobile Games 2

  • Game experience vs. Social acceptable behavior

  • Sound, Light and Vibration enhance the users Game experience

  •  Typically, games are played in locations where it is not suitable or socially acceptable to have the sound on.

  • During Mobile Usab. Testing consider there are usually other people close by when the user plays the game.

V. Heuristics & Guidelines

*(vgl.: NOKIASeries 60 Developer Platform 2.0: Usability Guidelines For J2ME™ Games

46


Mobile usability testing47 l.jpg

Mobile Usability Testing

VI. Examples & Ideas


Vi examples ideas l.jpg

Text Interface :: Projecting the image

  • A standard-sized Query-Keyboard is projectedby laser on any given surface.

  • The Users input is recognised by a small camera in the cigarette-pack sized device.

  • Note: Being announced in 1999 the product is still under development and may never reach market maturity. (2004)(*)

  • www.virtualdevices.net

VI. Examples & Ideas

*(vgl.: http://www.ibizpda.com)

48


Vi examples ideas49 l.jpg

Text Interface :: Breaking the Qwerty Paradigm

  • Standard-sized keys aligned for one-handed use.

  • Can be used under mobile conditions (doesn't need chair + desk environment as similar fold-up keyboards for PDAs)

VI. Examples & Ideas

*(vgl.: www.frogpad.com/)

49


Vi examples ideas50 l.jpg

Remote Controll :: Force Push

  • Operate household devices with agesture of your fingertip.

  • IR-Led points at Device to be controlled

  • Touch & Acceleration Sensors combinedwith Software recognise gestures and execute command

VI. Examples & Ideas

*(vgl.: Koji Tsukada, mobiquitous.com/pub/apchi2002-ubi-finger.pdf/)

50


Vi examples ideas51 l.jpg

Natural Interaction :: Stick with the basics

  • Real „Pick and Drop“

  • PDA – User picks up a file on his screen by tapping on it with a digital pen.

  • He passes on the pen to his colleague who drops the pen on his PDA-screen.

  • The file is copied.

VI. Examples & Ideas

*(vgl.: http://news.bbc.co.uk/2/hi/technology/3774747.stm)

51


Mobile usability testing52 l.jpg

Mobile Usability Testing

For more information see:

http://qse.ifs.tuwien.ac.at/courses/Usability/VO_Usability_Engineering.htm


ad