Needs analysis design through discovery
Download
1 / 51

Needs Analysis: Design through Discovery - PowerPoint PPT Presentation


  • 130 Views
  • Updated On :

Needs Analysis: Design through Discovery. Philip Kortum, Ph.D. Robert Bushey, Ph.D. SBC Laboratories Human Factors. Tonight's Agenda. Who we are (5 min) Basic needs analysis techniques (60 min) Virtual Lab tour (10 Min) Break (15 min) Needs analysis example (55 min) Conclusions (5 min).

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Needs Analysis: Design through Discovery' - redford


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Needs analysis design through discovery l.jpg

Needs Analysis:Design through Discovery

Philip Kortum, Ph.D.

Robert Bushey, Ph.D.

SBC Laboratories

Human Factors


Tonight s agenda l.jpg
Tonight's Agenda

  • Who we are (5 min)

  • Basic needs analysis techniques (60 min)

  • Virtual Lab tour (10 Min)

  • Break (15 min)

  • Needs analysis example (55 min)

  • Conclusions (5 min)


Who is sbc l.jpg
Who is SBC?

  • Short Answer:

    • We’re the phone company

  • Long Answer:

    • 2nd largest telecommunications providers the US

      • Wireless

      • Long Distance

      • Data

      • Residential/Business phone service

  • Over 60 million access lines

  • $41 Billion in Revenue (# 27 on the Fortune 500)


Who is sbc laboratories l.jpg

Technology Focus Areas

Who is SBC Laboratories?

SBC Laboratories is the Applied Research and Development arm of SBC.

About 250 Advanced-degreed Scientists and Engineers work at SBC’s laboratories in Austin and the Bay area in California.

Broadband

Architecture, Infrastructure, & Services

Intelligent

Networks

Internet

Wireless

Systems

Information

Technology


What is needs analysis l.jpg
What is Needs Analysis?

Needs analysis is the method of uncovering user requirements through direct or indirect interaction with that user


Why do we perform needs analysis l.jpg
Why do we perform needs analysis?

  • We are not the user

  • Users may not know what they really need

    • User is unable to articulate the need

    • True needs are often masked

  • 3 main goals of needs analysis


Goals of needs analysis 1 understand the user s goals l.jpg
Goals of needs analysis1: Understand the user’s goals


Goals of needs analysis 2 uncover user s conceptual models l.jpg
Goals of needs analysis2: Uncover user’s conceptual models

All I know is I push a few buttons here, pull a few sticks there,and up she goes. It seems to be some sort of magic


Goals of needs analysis 3 understand user s knowledge l.jpg
Goals of needs analysis3:Understand user’s knowledge



Good match between needs and implementation leads to superior efficiency and usability l.jpg
Good match between needs and implementation leads to superior efficiency and usability

“This is so cool! I’m flying this thing

completely on my Palm Pilot!”


How is needs analysis different than task analysis l.jpg
How is Needs Analysis different than Task Analysis? superior

  • Needs Analysis – trying to uncover the underlying motivations

  • Task Analysis – trying to uncover the procedural steps


How is needs analysis different than task analysis13 l.jpg

Task Analysis superior

Setting up a VCR

Programming a VCR

Needs Analysis

The ways a person uses a VCR and the reasons behind that use

What they record

What they do with what they record

Time-shifting

Remote control use use

How Is Needs Analysis Different Than Task Analysis?


How is needs analysis different than marketing l.jpg
How Is Needs Analysis Different Than Marketing? superior

  • They both deal with the user

  • They both try to determine what the user ‘needs’

  • Some of the techniques are similar

  • Some of the information collected is the same


How is needs analysis different than marketing15 l.jpg

Marketing superior

Demographics

Where people live, shop

How much they make

Physical attributes

Purchasing behavior patterns

What people will buy

Price points

Needs Analysis

User behavior patterns

What people do

What users know

Users environment

Users mental models

How Is Needs Analysis Different Than Marketing?


Needs vs marketing example a device that records off of the television l.jpg

Marketing superior

How much money do the users have to spend?

What other similar devices do they have?

What kinds of technology do they own?

How much do they use these technologies

Needs Analysis

Why would they want to use an image recorder?

How do they use current devices?

When do they use such devices?

What other technology interactions are there?

Needs vs. Marketing ExampleA device that records off of the television


A simple if unfair characterization l.jpg
A Simple (if unfair) Characterization: superior

Marketing deals with wants

Needs deals with needs


A word about semantics l.jpg
A word about Semantics…. superior

  • Many techniques are similar and are called different things by those who use them (even within the usability community)

  • Good practitioners almost always use a mix of techniques, and often call these mixtures by a new name.


Needs analysis l.jpg
Needs Analysis superior

Can be very focused and limited in scope

Doesn’t have to ‘big sky’


Basic types of needs analysis l.jpg
Basic Types of Needs Analysis superior

  • Questionnaires

  • Interviews

  • Observational Research

  • Hybrid methodologies


Questionnaires l.jpg
Questionnaires superior

“A set of questions for obtaining statistically useful or personal information from users”Webster


Questionnaire advantages l.jpg
Questionnaire Advantages superior

  • Easy to administer

    • Inexpensive

    • Can be administered by untrained staff

  • Can be administered remotely

  • Can have a relatively large number of questions and data samples

  • Can more easily target highly specific users

  • Can branch to reliably capture pertinent information

  • Data can be simple to code


Questionnaire disadvantages l.jpg
Questionnaire Disadvantages superior

  • Difficult to construct a good test instrument

  • Must know a priori exactly what you want to ask

    • No follow-up questions

  • Question intent may be open to interpretation

    • Can result in unknown bad results

  • Open ended responses highly variable

  • Open ended questions are difficult to code

  • Validation/verification difficult if study is not closely controlled


Questionnaire disadvantages fixes l.jpg
Questionnaire Disadvantages: Fixes superior

  • Difficult to construct a good test instrument

    • Follow scientifically designed questionnaire guides (Kirakowsi, Gillham, USARI)

    • Test for validity, reliability, repeatability

  • Must know a priori exactly what you want to ask

    • Conducted pilot tests to try out the questions

    • Conduct interviews to help build a reasonable first pass

    • Use domain experts to help capture detail questions


Questionnaire disadvantages fixes25 l.jpg
Questionnaire Disadvantages: Fixes superior

  • Question intent may be open to interpretation

    • Conduct pilot tests to check for weaknesses

    • Use experts and novices to examine interpretation differences

    • Use unambiguous, quantifiable, anchored language


Ambiguous question example l.jpg
Ambiguous Question Example superior

How bad was your last car accident?

Really bad bad not really bad not bad at all

2 respondents, who both dented the front bumper slightly, no injury

Respondent 1: ‘Really Bad’

Respondent 2: ‘Not bad at all’

“I’m only 16, it’s my first accident and it was my dad’s new sports car, and I wasn’t on the insurance for the car. I’m going to be grounded for life!

“Well, compared to that 23 car roll-over collision I had with that semi last month, this was nothing. Not to mention that I got the collision waiver on the rental!

Class: How could we fix this?


Questionnaire disadvantages fixes27 l.jpg
Questionnaire Disadvantages: Fixes superior

  • Open ended responses highly variable

    • Provide hints, examples to guide the responses

    • Use multi-part questions that are more precise

  • Open ended questions are difficult to code

    • Use more choices, or more questions, to remove need for open ended questions

    • Focus on variables of interest and use ‘other’ to capture low probability events that are not pertinent

    • Devise keyword codebooks to help code open ended data


Questionnaire disadvantages fixes28 l.jpg
Questionnaire Disadvantages: Fixes superior

  • Validation/verification difficult if study is not closely controlled

    • Key code questionnaires to match user and instrument

    • Lock-out multiple take attempts

    • Ask qualifying questions as part of a pretest, or branch


Interviews l.jpg
Interviews superior

“A meeting at which information

is obtained from a person”

Webster


Interviews30 l.jpg
Interviews superior

  • Generally classified into two types:

  • Structured

  • Unstructured


Interview advantages l.jpg
Interview Advantages superior

  • Small numbers of interviews can be performed quickly and inexpensively (although not always)

  • Can be conducted remotely

  • Can be conducted in groups or individually

  • Can be conducted with little notice, if needed

  • Can change the interview for different levels of users


Interview advantages32 l.jpg
Interview Advantages superior

  • Open for opportunistic data discovery

  • Get non-verbal clues to help guide the interview

  • Good technique for ‘fishing expeditions’

  • Useful for gathering preliminary information to guide later needs analysis


Interview disadvantages l.jpg
Interview Disadvantages superior

  • Hard to conduct a ‘good’ interview

    • Easy to lead the respondent, with both verbal and non-verbal cues

    • Easy to unknowingly ask loaded questions

      • “How does it feel to be ugly?”

  • Harder still to conduct a good group interview

    • herd mentality, strong leader

  • Hard to stay ‘up’ and focused for a large number of interviews


Interview disadvantages34 l.jpg
Interview Disadvantages superior

  • Data can be difficult to code and quantify

  • Can be large differences between interviewers

  • Expensive and time consuming to conduct large numbers of interviews

  • Usually a low upper limit to the number of questions


Interview disadvantages35 l.jpg
Interview Disadvantages superior

Interview Example


Interview disadvantages36 l.jpg
Interview Disadvantages superior

  • Interview data may not match reality

Question

What was said

What the data suggests

How many offers do reps make per call?

Reps make 1 offer per call

Almost no reps make 1 offer, some do 0 some do 2+

What kind of rep is the least desirable?

‘order takers’ are the least desirable

Order takers are good -

they drive accessibility

and are among top $ performers

What makes the customer happy?

Taking lots of calls is what makes customers happy

No supporting data


Interview disadvantages fixes l.jpg
Interview Disadvantages: Fixes superior

  • Hard to conduct a ‘good’ interviews

    • Use scientifically developed methods (i.e. Weiss’s Learning from Strangers)

    • Pre-develop a core set of questions

    • Manage group dynamics

    • Use hypothetical questions to elicit more detailed answers

  • Hard to stay ‘up’ and focused for a large number of interviews

    • Limit the number of interviews

    • Use tag teams, alternate sessions

    • Use well trained ‘people persons’


Interview disadvantages fixes38 l.jpg
Interview Disadvantages: Fixes superior

  • Data can be difficult to code and quantify

    • use core set of questions

    • use data code books to help quantify responses

    • use affinity diagrams to aid in coding

  • Can be large differences between interviewers

    • use single interviewer

    • use visible or hidden teams

    • conduct pilot training to level the interviewers


Interview disadvantages fixes39 l.jpg
Interview Disadvantages: Fixes superior

  • Expensive and time consuming to conduct large numbers of interviews

  • Usually a low upper limit to the number of questions

    • target exact population

    • supplement with other techniques

    • use remote techniques

    • use core question set, based on pre-test

  • Interview data may not match reality

    • Always verify data


Observational research l.jpg
Observational Research superior

“An act of recognizing and

noting a fact or occurrence…” Webster


Observational research41 l.jpg
Observational Research superior

  • Observation doesn’t have to occur in real time

  • Observation doesn’t have to be visually based

    • audio recordings

    • diaries

    • telemetry (GPS, web logs, biometrics, equipment monitors) (e.g. car snooper)


Observational advantages l.jpg
Observational Advantages superior

  • Good for understanding complex needs

  • Results are not defined by the design of the method

  • Capable of capturing unknown or undocumented behaviors

  • Gather data on specific artifacts

  • Can uncover inter-dependencies

  • If documented appropriately, can review observations as many times as necessary

  • Can parse expert/novice distinction


Observational disadvantages l.jpg
Observational Disadvantages superior

  • Some phenomenon is too infrequent to catch

  • Heisenberg Principle (i.e. driving tester)

  • Can be more difficult to obtain a reviewable record

  • Desired behavior may be interspersed with other unrelated activity

  • Slow, expensive to collect data

  • Can easily get ‘off-target’

  • Data set can be overwhelming


Observational disadvantages fixes l.jpg
Observational Disadvantages: Fixes superior

  • Some phenomenon is too infrequent to catch

    • use high sample rates to find low probability events

    • create low probability (disaster training)

    • sample in areas known to have these events

    • be lucky

  • Heisenberg Principle (i.e. driving tester)

    • observe for extended periods of time

    • use remote measuring techniques

    • use low/no interaction models

    • clearly understand the cost-benefit of direct interaction

    • work in the users environment


Observational disadvantages fixes45 l.jpg
Observational Disadvantages: Fixes superior

  • Can be more difficult to obtain a reviewable record

    • use alternate recording technologies

    • work in teams

    • Use advanced behavioral software (e.g. Noldus Observer)

  • Desired behavior may be interspersed with other unrelated activity

    • use specific artifact techniques

    • use remote data collection techniques to time compress

    • don’t assume that unrelated activity is actually unrelated (e.g. using the restroom while waiting)


Observational disadvantages fixes46 l.jpg
Observational Disadvantages: Fixes superior

  • Slow, expensive to collect data

    • use remote or automated collection where possible

    • understand required sample size

    • carefully select person/situation/setting to be observed

  • Can easily get ‘off-target’

    • use pilot observations to help identify key behaviors

    • be open to the fact that off-target behaviors may be pertinent


Observational disadvantages fixes47 l.jpg
Observational Disadvantages: Fixes superior

  • Data set can be overwhelming

    • use data reduction/consolidation techniques

      • training samples

      • affinity diagrams

      • physical flow/sequence structures


Observational research example watching television l.jpg
Observational Research Example superior Watching Television


Observational research example watching television49 l.jpg
Observational Research Example superior Watching Television

Nathen, et al, 1985


Hybrid methodologies l.jpg
Hybrid Methodologies superior

  • Ethnography

  • Contextual Inquiry

  • Empathic Design

  • Participatory learning

    • actors

    • rotational management assignments


Questions l.jpg
Questions? superior


ad