hci460 week 5 lecture n.
Skip this Video
Loading SlideShow in 5 Seconds..
HCI460: Week 5 Lecture PowerPoint Presentation
Download Presentation
HCI460: Week 5 Lecture

Loading in 2 Seconds...

play fullscreen
1 / 36

HCI460: Week 5 Lecture - PowerPoint PPT Presentation

  • Uploaded on

HCI460: Week 5 Lecture. October 7, 2009. Outline. A few more words about project 1b (Expert Evaluation Report) Plan for next Wednesday Test Plan: Common feedback themes Moderating usability sessions Observing sessions and note taking Guest lecture:

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

HCI460: Week 5 Lecture

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
hci460 week 5 lecture
HCI460: Week 5 Lecture
  • October 7, 2009
  • A few more words about project 1b (Expert Evaluation Report)
  • Plan for next Wednesday
  • Test Plan: Common feedback themes
  • Moderating usability sessions
  • Observing sessions and note taking
  • Guest lecture:
    • Speaker: Ed Israelski, Human Factors Manager, Abbott
    • Title: “Usability Methods in Healthcare”
  • Time for questions, feedback on your Moderator’s Guide and Screener, group work
can i resubmit

A Few More Words About Project 1b

Can I Resubmit?
  • Everyone can resubmit their Project 1b.
    • No grade change
    • No deadline
    • Feedback will be provided verbally during office hours
  • We will look at how you have improved from the first report to the second and third.
formative usability testing of

Plan for Next Wednesday

Formative Usability Testing of:

Samsung TV remote control

TRUEtrack glucose meter

Policy declarations page for an auto insurance company

Flip Video Mino HD

Franklin Spelling Ace

Tenori-On digital musical instrument (synthesizer)

Livescribe Pulse Smartpen

Corner Bakery printable pdf menu

Slacker Radio application on the BlackBerry Storm

Google Maps

Brookstone AM/FM radio alarm clock


Plan for Next Wednesday

  • Address:
    • 500 North Michigan Avenue
    • Floor 16
    • Suite 1610 – User Centric, Inc.
  • Directions:
    • http://www.usercentric.com/directions_chicago.html

Plan for Next Wednesday


Group 1: Dietz, Dulski, Dzienisowicz, Remington

Group 2: Birdseye, Dash, Gamboa, Welense

Group 3: Elliott, Petlick

Group4: Berberian, Devlin, Shah, Sienkiewicz

Group 5: Albarracin, Ginez, Schwarz

Group 6: Epps, Jones, Lund

Group 7: Barbera, Schulte, Schwantes, Young

Group 8: Canady, Doshi, Roberts

Group 9: Garcia, Goldberg, Haines, Wickenkamp

Group 10: Diemer, Komosa, Ranguelov, Young

Group 11: Cheng, Freeman, Taylor

arrival times

Plan for Next Wednesday

Arrival Times
  • Groups 3, 5, 6, 8, 9 and 10 should arrive before 5:45pm.
    • If not all team members can arrive early, whoever can come in early should bring the product you are testing.
  • Groups 1, 2, 4, 7 and 11 should arrive at 7:15pm.
    • Your early arrival will help us make sure that the lab setup works with your product.
other important details

Plan for Next Wednesday

Other Important Details
  • There will be only one test session per group.
    • You should conduct the rest of your sessions (min 5 total) on your own.
  • Limit your session to 20-25 minutes.
    • Typical session lengths: 30, 45, 60, 90, 120 minutes but class time is limited, so we will have shorter sessions.
    • You can conduct longer sessions on your own if you’d like.
  • Participants will be provided.
  • The moderator will be selected randomly from each group, so you all should be prepared to moderate.
    • Do a dry run before Wed session to make sure you can fit the session into a 20-25-minute time slot.
  • Don’t forget to bring what you are testing.

Plan for Next Wednesday

  • You will receive feedback from:
    • Gavin or Aga
    • Your participant
    • Your observers (other groups observing your test session)
  • Feedback will be provided on:
    • The moderation
    • The procedure
  • Each observer will fill out simple feedback sheets:
    • 1 – 3 things that you think the moderator did well
    • 1 – 3 things that the moderator should improve on
    • 1 – 3 good things about the procedure
    • 1 – 3 ways in which the procedure could be improved
  • All feedback will be compiled and posted on COL by EOD 17th.

Test Plans: Common Themes

  • Narrow down your scope.
    • Keep in mind session length
    • Too many objectives / tasks
    • Comparing products instead of testing a single product
      • Good idea but not for formative testing with a few very short sessions.
business objective

Test Plans: Common Themes

Business Objective
  • Provide a business objective.
    • Not critical but very helpful
    • In real life, you should get business objectives from stakeholders
    • Why are we testing this product?
      • To lower the return rate?
      • To decrease the volume of calls to the call center?
objectives and tasks

Test Plans: Common Themes

Objectives and Tasks
  • Objectives and tasks/questions must be in perfect alignment.
    • Objectives without tasks/questions can lead to disappointed stakeholders.
    • Tasks/questions not addressing any objectives are a waste of time.
      • Did you forget to list an important objective?
stimuli and measures

Test Plans: Common Themes

Stimuli and Measures
  • What are stimuli?
    • Stimuli = artifacts = stuff that will be tested
    • E.g., XYZ alarm clock, XYZ glucose meter, XYZ website
  • Use of quantitative measures (e.g., time on task, ratings)
    • Appropriate in summative testing
    • Why and how should we use quantitative measures in formative testing?

Moderating Usability Sessions

  • Who has moderated?
    • Explain insights/lessons learned
  • What makes a good moderator?
need for objectivity

Moderating Usability Sessions

Need for Objectivity

One basic tenet of usability testing is that it is virtually impossible to remain objective when usability testing your own product

Separate design and test teams

Reality is that some testing is better than no testing at all

characteristics of a good moderator rubin

Moderating Usability Sessions

Characteristics of a Good Moderator (Rubin)

Knowledgeable in UCD

Quick learner

Builds a rapport with participants / good communicator

Excellent memory

Good listener

Comfortable with ambiguity


Long attention span

Empathic “people person”

“Big picture” thinker

characteristics of a good moderator lew bojko

Moderating Usability Sessions

Characteristics of a Good Moderator (Lew/Bojko)

Reading the guide vs. just reading the guide

Biggest fear (study)

Biggest fear (formative session)

depends on technique

Moderating Usability Sessions

Depends on Technique
  • State task and participant does task
    • Discussions are limited
  • Think aloud protocol
    • Pros / cons
  • Retrospective review
    • Pros / cons
moderators should avoid rubin

Moderating Usability Sessions

Moderators Should Avoid… (Rubin)

Leading…instead, enable

Being overly consumed with data collection

Acting too knowledgeable

Being rigidly adhering to moderator’s guide

Not relating well to every participant

Jumping to conclusions

moderators should remember lew bojko

Moderating Usability Sessions

Moderators Should Remember… (Lew & Bojko)
  • When in doubt, read
  • The power of silence (simply pause)
  • What it is like to cut video clips
  • When trying to nail a point, avoid questions with yes/no, instead ask for elaboration
  • Manage time, but also take your time (yeah, I know)
  • Don’t jumping to conclusions, BUT be aware of trends
    • Especially when you are the single moderator
simultaneous testing sessions

Moderating Usability Sessions

Simultaneous Testing Sessions
  • How many sessions can be run in a day?
    • 60 min sessions?
    • 90 min sessions?
    • 120 min sessions?
  • Consider sample size of N=12
    • N=20
    • N=20, but with a between groups design (N=40)
  • Implications
    • Parallel sessions
    • Ramp up
communication during session

Moderating Usability Sessions

Communication During Session
  • Pros and cons
  • Notes handed back
  • Walk outside
  • Ear buds
  • IM
  • Do not get flustered
    • Okay to wait a moment
what if your mobile usability lab

Moderating Usability Sessions

What if Your Mobile Usability Lab…
  • Somehow it looks worse after the luggage handlers touched it…
    • It is 7:45am. The first session starts at 9 am.
what if the prototype is not ready

Moderating Usability Sessions

What if the Prototype is Not Ready?
  • Cancel study?
    • Cost / timeline
  • Ready the day before?
    • Technique
team size

Observing Sessions and Note Taking

Team Size
  • Two-person test teams vs. one-person test teams
    • Single session
    • One moderator and one note taker
    • Pros/Cons
  • Switching considerations
    • One and only one moderator?
managing the observation room

Observing Sessions and Note Taking

Managing the Observation Room
  • Manage observation room
    • Redirect
    • Trends vs. outliers
      • Note trends on a whiteboard
    • Assess politics
    • Achievable recommendations
questions about the mod guide screener

Screener and Moderator’s Guide

Questions About the Mod Guide & Screener
  • Both the Screener and Moderator’s Guide are due on Friday 11:59 pm Central Time for in-class students (Tuesday for DL students).
  • Again, we will provide you with feedback.
  • Grade?
    • Project 2 will be graded after the final report is turned in.
    • As appendices, you should turn in:
      • Test Plan
      • Moderator’s Guide improved based on our feedback
      • Screener
  • Questions:
    • How will the screener be used?
    • ….?