User interface evaluation
Download
1 / 56

User Interface Evaluation - PowerPoint PPT Presentation


  • 108 Views
  • Uploaded on

User Interface Evaluation. Usability Testing Methods http://jthom.best.vwh.net/usability/ http://www.cs.umd.edu/~zzj/UsabilityHome.html. Usability Testing Methods. Caring out experiments to find out specific information about a design and/or product. Basis comes from experimental psychology.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' User Interface Evaluation' - jackson-gonzalez


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
User interface evaluation

User Interface Evaluation

Usability Testing Methods

http://jthom.best.vwh.net/usability/

http://www.cs.umd.edu/~zzj/UsabilityHome.html


Usability testing methods
Usability Testing Methods

  • Caring out experiments to find out specific information about a design and/or product.

  • Basis comes from experimental psychology.

  • Uses statistical data methods

    • Quantitative and Qualitative


Usability testing methods1
Usability Testing Methods

  • During usability testing, users work on specific tasks using the interface/product and evaluators use the results to evaluate and modify the interface/product.

  • Widely used in practice, but not appropriately used.

  • Often abused by developers that consider themselves to be usability experts.

  • Can be very expensive and time consuming.


Usability testing methods2
Usability Testing Methods

  • Performance Measurement

  • Thinking-aloud Protocol

  • Question-asking Protocol

  • Coaching Method


Usability testing methods3
Usability Testing Methods

  • Co-discovery Learning

  • Teaching Method

  • Retrospective Testing

  • Remote Testing


Performance measurement
Performance Measurement

  • Applicable Stages:

    • Design, Code, Test & Deployment

  • Personnel

    • Usability Experts, approximately 1.

    • Developers, 0.

    • Users, 6.


Performance measurement1
Performance Measurement

  • Usability Issues Covered

    • Effectiveness: Yes

    • Efficiency: Yes

    • Satisfaction: No

  • Quantitative Data is collected.

  • Can NOT be conducted remotely.

  • Can be used on any system.


Performance measurement2
Performance Measurement

  • What is it?

  • Used to collect quantitative data.

  • Typically, you will be looking for benchmark data.

  • Objectives MUST be quantifiable

    • 75% of users shall be able to complete the basic task in less than 30 minutes.


Performance measurement3
Performance Measurement

  • How can I do it?

  • Define the goals that you expect users to perform


Performance measurement4
Performance Measurement

  • How can I do it?

  • Quantify the goals

    • The time users take to complete a specific task.

    • The Ratio between successful interactions and errors.

    • The time spent recovering from errors.

    • The number of user errors.

    • The number of commands or other features that were never used by the user.

    • The number of system features the user can remember during a debriefing after the test.

    • The proportion of users who say that they would prefer using the system over some specified competitor.


Performance measurement5
Performance Measurement

  • How can I do it?

  • Get participants for the experiments

  • Conduct very controlled experiments

    • All variables must remain consistent across users

  • http://www.cs.umd.edu/~zzj/PerfMeas.htm


Performance measurement6
Performance Measurement

  • Problems With Performance Measurement

  • No qualitative data.


Thinking aloud protocol
Thinking-aloud Protocol

  • Applicable Stages:

    • Design, Code, Test & Deployment

  • Personnel

    • Usability Experts, approximately 1.

    • Developers, 0.

    • Users, 4.


Thinking aloud protocol1
Thinking-aloud Protocol

  • Usability Issues Covered

    • Effectiveness: Yes

    • Efficiency: No

    • Satisfaction: Yes

  • Quantitative Data is NOT collected.

  • Can NOT be conducted remotely.

  • Can be used on any system.


Thinking aloud protocol2
Thinking-aloud Protocol

  • What is it?

  • Technique where the participant is asked to vocalize his or her thoughts, feelings, and opinions while interacting with the product. .


Thinking aloud protocol3
Thinking-aloud Protocol

  • How can I do it?

  • Select the participants, who will be involved?

  • Select the tasks and design scenarios.

  • Ask the participant to perform a task using the software.


Thinking aloud protocol4
Thinking-aloud Protocol

  • How can I do it?

  • During the task, ask the user to vocalize

    • Thoughts, opinions, feelings, etc.

  • http://www.cs.umd.edu/~zzj/ThinkAlo.htm


Thinking aloud protocol5
Thinking-aloud Protocol

  • Problem With Thinking-Aloud Protocol

  • Cognitive Overload

    • Can you walk & chew gum at the same time?

    • Asking the participants to do too much.


Question asking protocol
Question-asking Protocol

  • Applicable Stages:

    • Design, Code, Test & Deployment

  • Personnel

    • Usability Experts, approximately 1.

    • Developers, 0.

    • Users, 4.


Question asking protocol1
Question-asking Protocol

  • Usability Issues Covered

    • Effectiveness: Yes

    • Efficiency: No

    • Satisfaction: Yes

  • Quantitative Data is NOT collected.

  • Can NOT be conducted remotely.

  • Can be used on any system.


Question asking protocol2
Question-asking Protocol

  • What is it?

  • Similar to Thinking-aloud protocol.

  • Instead of participant saying what they are thinking, the evaluator prompts the participant with questions while using the system.


Question asking protocol3
Question-asking Protocol

  • How can I do it?

  • Select the participants, who will be involved?

  • Select the tasks and design scenarios.

  • Ask the participant to perform a task using the software.


Question asking protocol4
Question-asking Protocol

  • How can I do it?

  • During the task, ask the user to questions about the product

    • Thoughts, opinions, feelings, etc.

  • http://www.cs.umd.edu/~zzj/Question.htm


Question asking protocol5
Question-asking Protocol

  • Problem With Thinking-Aloud Protocol

  • Cognitive Overload++

    • Can you walk, chew gum & talk at the same time?

    • Asking the participants to do too much.

    • Added pressure when the evaluator asks questions.

    • Can be frustrating on novice users.


Coaching method
Coaching Method

  • Applicable Stages:

    • Design, Code, Test & Deployment

  • Personnel

    • Usability Experts, approximately 1.

    • Developers, 0.

    • Users, 4.


Coaching method1
Coaching Method

  • Usability Issues Covered

    • Effectiveness: Yes

    • Efficiency: No

    • Satisfaction: Yes

  • Quantitative Data is NOT collected.

  • Can NOT be conducted remotely.

  • Can be used on any system.


Coaching method2
Coaching Method

  • What is it?

  • A system expert sits with the participant and acts as a coach.

  • Expert answers the participant’s questions.

  • The evaluator observes their interaction.


Coaching method3
Coaching Method

  • How can I do it?

  • Select the participants, who will be involved?

  • Select the tasks and design scenarios.

  • Ask the participant to perform a task using the software in the presence of a coach/expert.


Coaching method4
Coaching Method

  • How can I do it?

  • During the task, the user will ask the expert questions about the product

  • http://www.cs.umd.edu/~zzj/Coaching.htm


Coaching method5
Coaching Method

  • Problem With Coaching Method

  • In reality, there will not be a coach present.

  • This is good for creating a coaching system, but not for evaluating the interface.


Co discovery learning
Co-Discovery Learning

  • Applicable Stages:

    • Design, Code, Test & Deployment

  • Personnel

    • Usability Experts, approximately 1.

    • Developers, 0.

    • Users, 6.


Co discovery learning1
Co-Discovery Learning

  • Usability Issues Covered

    • Effectiveness: Yes

    • Efficiency: No

    • Satisfaction: Yes

  • Quantitative Data is NOT collected.

  • Can NOT be conducted remotely.

  • Can be used on any system.


Co discovery learning2
Co-Discovery Learning

  • What is it?

  • Two test users attempt to perform tasks together while being observed.

  • They are to help each other in the same manner as they would if they were working together to accomplish a common goal using the product.

  • They are encouraged to explain what they are thinking about while working on the tasks.

    • Thinking Aloud, but more natural because of partner.


Co discovery learning3
Co-Discovery Learning

  • How can I do it?

  • Select the participants, who will be involved?

  • Select the tasks and design scenarios.

  • Ask the participants to perform a task using the software.


Co discovery learning4
Co-Discovery Learning

  • How can I do it?

  • During the task, the users will help each other and voice their thoughts by talking to each other.

  • http://www.cs.umd.edu/~zzj/Codiscov.htm


Co discovery learning5
Co-Discovery Learning

  • Problem With Co-Discovery Learning

  • Neither is an expert

    • The blind leading the blind.


Teaching method
Teaching Method

  • Applicable Stages:

    • Design, Code, Test & Deployment

  • Personnel

    • Usability Experts, approximately 1.

    • Developers, 0.

    • Users, 4.


Teaching method1
Teaching Method

  • Usability Issues Covered

    • Effectiveness: Yes

    • Efficiency: No

    • Satisfaction: Yes

  • Quantitative Data is NOT collected.

  • Can NOT be conducted remotely.

  • Can be used on any system.


Teaching method2
Teaching Method

  • What is it?

  • You have 1 participant use the system.

  • Ask the participant to teach a new novice participant how to use the system.


Teaching method3
Teaching Method

  • How can I do it?

  • Select the participants, who will be involved?

  • Select the tasks and design scenarios.

  • Ask the 1st participant to perform a task using the software.

  • Ask the 1st participant to teach a new participant.


Teaching method4
Teaching Method

  • How can I do it?

  • Observe their interactions.

  • http://www.cs.umd.edu/~zzj/Teaching.htm


Teaching method5
Teaching Method

  • Problem With Teaching Method

  • Neither is an expert

    • The blind leading the blind.

    • Possible to discover some interesting things about the learn-ability of your interfaces.


Retrospective testing
Retrospective Testing

  • Applicable Stages:

    • Design, Code, Test & Deployment

  • Personnel

    • Usability Experts, approximately 1.

    • Developers, 0.

    • Users, 4.


Retrospective testing1
Retrospective Testing

  • Usability Issues Covered

    • Effectiveness: Yes

    • Efficiency: Yes

    • Satisfaction: Yes

  • Quantitative Data can be collected.

  • Can NOT be conducted remotely.

  • Can be used on any system.


Retrospective testing2
Retrospective Testing

  • What is it?

  • A videotape of the session is observed by the usability expert and the participants.


Retrospective testing3
Retrospective Testing

  • How can I do it?

  • Select the participants, who will be involved?

  • Select the tasks and design scenarios.

  • Use one of the usability testing methods that we have discussed.

  • Videotape the session.


Retrospective testing4
Retrospective Testing

  • How can I do it?

  • Review the videotape with the users.

  • http://www.cs.umd.edu/~zzj/Retrospe.htm


Retrospective testing5
Retrospective Testing

  • Problem With Retrospective Testing

  • Extremely time consuming!


Remote testing
Remote Testing

  • Applicable Stages:

    • Design, Code, Test & Deployment

  • Personnel

    • Usability Experts, approximately 1.

    • Developers, 0.

    • Users, 5.


Remote testing1
Remote Testing

  • Usability Issues Covered

    • Effectiveness: Yes

    • Efficiency: Yes

    • Satisfaction: Yes

  • Quantitative Data can be collected.

  • Can be conducted remotely.

  • Can be used on any system.


Remote testing2
Remote Testing

  • What is it?

  • The participants are separated from the evaluators.

  • No formal observation.

  • No usability lab.


Remote testing3
Remote Testing

  • How can I do it?

  • Give the product/software to participants.

  • Collect information about how they use your software/product.

  • Methods

    • Same-Time Different Place

    • Different-Time Different Place


Remote testing4
Remote Testing

  • How can I do it?

  • Lotus Video Cam, [email protected], SnagIt

  • Usability Logger

    • http://www.usabletools.com/

  • Journaled Sessions

  • http://www.cs.umd.edu/~zzj/Retrospe.htm


Remote testing5
Remote Testing

  • Problem With Remote Testing

  • The evaluator is not there.

  • Can’t observe facial expressions.

  • Great for Web based systems.


Usability testing methods4
Usability Testing Methods

  • Select the method that works best for you.

  • Select the method that fits your implementation.

  • Be thorough during your experiments.

  • The more data, the better.


Usability testing methods5
Usability Testing Methods

  • Hawthorne Effect

    • The tendency for people to improve their performance after any change when they know their performance is being studied.


ad