1 / 31

Presentation - PowerPoint PPT Presentation

  • Uploaded on

Defining a Product Vision using Remote Research Methodologies presented to The 14 th Annual Usability Professionals’ Association Conference July 1 st , 2005 Mark Safire Director of User Experience Research Sachs Insights. Presentation. Who am I The Products & The Vision

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Presentation' - viveka

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Defining a Product Vision usingRemote Research Methodologiespresented toThe 14th AnnualUsability Professionals’ AssociationConferenceJuly 1st, 2005Mark SafireDirector of User Experience ResearchSachs Insights


  • Who am I

  • The Products & The Vision

    • The Company – NPD Group

    • The Software Products – Solution Folders & PowerView

    • The User Profiles

    • The Research Challenge

    • Remote Moderated Usability

      • Definition; Benefits; Challenges

      • Audience Participation – Experiences with Remote Research

  • The Transformation of PowerView through Research

    • Research Road Map

    • PowerView Research

      • Phase 1: Remote Contextual Inquiry

      • Phase 2: Remote Moderated Usability

      • The Upshot

  • Q&A

Who am i
Who Am I?

  • Mark Safire

    • Director of User Experience Research, Sachs Insights

      • Qualitative research provider for NPD Group online/offline software products since 2002.

    • Spoken at UPA 2003 (Presentation) and UPA 2004 (Idea Market) on Remote Moderated Usability.

      • Also at NYC and Dallas UPA chapters, and for Nielsen Norman Group conferences in Las Vegas and Amsterdam.

    • 2003 Presentation is online at

Sachs insights
Sachs Insights

Qualitative research consultancy founded in 1987

  • Serve automotive, fashion, financial services, media, pharmaceutical, technology and travel industries.

    Practice Areas Include:

  • Brand & Advertising Research

  • New Product & Service Development

  • User Experience Research

    In our User Experience practice, we conduct:

  • Concept & Process Focus Groups

  • Taxonomy & Nomenclature Focus Groups

  • Usability

  • Contextual Inquiry

The company
The Company

  • A leading provider of sales and marketing information based on combined point-of-sale and consumer data for over a dozen major industries.

    • Apparel, appliances, automotive, beauty, cellular, consumer electronics, food and beverage, foodservice, footwear, housewares, imaging, information technology, movies, music, software, toys, travel, video games, ...*

    • Supplemented by consumer panel data to give a more complete picture – demographics, etc.

    • Data from “over 230 retail partners, representing over 90,000 doors worldwide”*

  • Clients: Manufacturers, Retailers, Analysts

  • Quoted in media regularly


  • The software products
    The Software Products

    • Goals of Software

      • Allow customers to analyze data that NPD has collected and organized.

    • Usage Scenarios

      • Monthly reports: Same reports, different numbers

      • Ad-hoc queries: Different reports each time

    • Sample Queries for Reports

      • What is my market share vs. my competitors?

      • How are inkjet printer sales trending over the past 3 mo.’s?

      • What were the top ten products in $ sold last month?

    The software products1

    Solution Folders


    The Software Products

    The software products2

    Solution Folders

    Make Reports from Templates


    Make Reports from Scratch

    The Software Products




    Little (Online)




    A Lot (Offline)


    Room for Error

    Learning Curve

    Setup Required

    The software products3
    The Software Products


    Harder To Learn




    To Learn





    The user profiles ingoing assumptions
    The User Profiles – Ingoing Assumptions

    • Disseminators

      • Market Research experts – the “gatekeepers”

      • Experienced in performing quantitative analysis

      • Pull, analyze, develop reports for others

    • Users

      • Support personnel, bus. analysts & owners, sales & mktg.

      • Less experienced in performing quantitative analysis

      • Pull, analyze, developer reports for themselves

    • Viewers

      • Bus. analysts & owners, sales & mktg.

      • Less experienced in performing quantitative analysis

      • Review reports created by others

    Research challenge finding respondents
    Research Challenge – Finding Respondents

    • Some had low incidence

      • “User” profile not yet a significant percentage.

    • Geographical dispersion

      • Key retailers not clustered

      • Expensive and time consuming to fly them in

    • 2nd Round of Research – Timing

      • Week before Christmas

    Solution: Remote Moderated Usability

    Usability in a lab
    Usability in a Lab





    Remote moderated usability
    Remote Moderated Usability





    Remote moderated usability1
    Remote Moderated Usability

    • Differences from In-Lab Usability

      • Different Locations

      • Software Tool

      • Teleconference

      • Observers usually apart from Moderator


    Remote moderated usability2
    Remote Moderated Usability

    • Benefits

      • Recruiting benefits

        • Reach geographically dispersed populations

        • Reach hard to recruit populations – don’t leave home/office

      • Avoid cost and inconvenience of travel for all parties involved.

      • Get “buy-in” from stakeholders in distant locations

    • NPD Research Conditions:

      • All had access to high-speed Internet connections

      • Participants & their companies were open to the technology

      • NPD Group had a WebEx license and in-house expertise, used primarily for remote demos and help desk support


    Remote moderated usability3
    Remote Moderated Usability

    • Challenges

      • Body Language

      • Rapport

      • Interruptions (phone, visitors, email, IM, …)

      • Logistics (time zones, ...)

      • Security & Privacy

      • Observer dynamics

      • Broadband connection recommended

      • Not for newbies & less coordinated


    Audience participation
    Audience Participation

    • Any Experiences? 1-2 Min. Summary:

      • Your Name, Title, & Company

      • Research Description: Industry, (Client), Audience, & Objective

      • Why Remote Moderated Usability?

      • Challenges you faced & Tips for dealing with them

    Remote moderated usability tools methods
    Remote Moderated Usability Tools & Methods

    • Many tools available

      • NPD done with WebEx Meeting Manager

      • Others commercial apps/ASPs to consider:

        • IBM Lotus Web Conferencing (f.k.a. IBM Lotus Same Time)

        • Microsoft LiveMeeting (f.k.a. Placeware)

        • Citrix GoToMeeting (f.k.a. DesktopStreaming)

        • Macromedia Breeze

      • Other free or open source solutions to consider:

        • Microsoft Messenger/MSN Messenger (f.k.a. Microsoft NetMeeting)

        • VNC (open source – various implementations, including RealVNC)

      • Others kinds of tools

        • Full-time Remote Control Apps (See

          • pcAnywhere, GoToMyPC, Timbuktu, etc.


    Research roadmap

    Solution Folders

    Initial Launch

    In-Lab Interviews

    Optimize Navigation

    Remote Moderated Usability


    Needs Assessment

    Internal Focus Groups

    Remote Contextual Inquiry

    Prototype before Relaunch (version 5.0)

    Remote Moderated Usability

    Research Roadmap

    Powerview research

    Phase I

    Remote Contextual Inquiry

    How do they use it?

    They define & demo tasks.

    Does it meet actual needs?

    See Role in Work Process

    Ultimate reports delivered

    How easy is it to use?

    First Use / Experienced Use

    Things to change /Things NOT to change

    Phase 2

    Remote Moderated Usability

    Can they use it?

    Use their pre-defined tasks

    Do fixes meet needs?

    Role-play work process

    Observe ease of use.

    First Use / Experienced Use

    Points of pain improved? / Have to relearn a lot?

    PowerView Research

    Phase 1 remote contextual inquiry rci
    Phase 1, Remote Contextual Inquiry (RCI)

    • First time & “Ad Hoc use”

      • Even old-timers stumble when trying to create new reports.

      • Drag & drop interface – good idea, don’t change!

      • Inscrutable 8 character field names

      • Hard to envision report layout from interface

        • Especially for “nested” reports

    • Expert & Repeated use

      • Gargantuan monthly reports: Same thing, different numbers

        • Each report is run individually, not as a “batch”

        • Typically formatted each time in Excel – not saved in tool.

          • Often requires specific colors for company and competitors in for tables and charts.

          • Extra columns created during export to show “suppression.”

            Everyone: Look & Feel “dated” – want standard Windows conventions

    Phase 2 remote moderated usability rmu
    Phase 2, Remote Moderated Usability (RMU)

    • First time & “Ad Hoc use”

      • Drag & drop approach still used.

      • Comprehensible, full text, categorized field names

        • But can change back to 8 char. abbrev’s in preferences...

      • Preview of report layout

    • Expert & Repeated use

      • Run as a batch

        • But “Favorites” menuv – confusing metaphor

        • Final: “MyReports”

      • Save formatting

      • No extra columns!

        • gray or red italicized numbers with asterisks

    Powerview phase 1 rci
    PowerView, Phase 1, RCI

    1) Drag & drop a good idea: don’t change!

    2) Inscrutable 8 character field names, without any categorization

    3) Hard to envision report layout from interface

    4) Look & Feel “dated” – not standard Windows conventions.

    Powerview phase 1 rci1
    PowerView, Phase 1, RCI

    Widget Production Quarterly Trend

    What they need

    Powerview phase 1 rci2
    PowerView, Phase 1, RCI

    5) Extra columns

    6) Formatting not saved

    What they get

    Powerview phase 2 rmu
    PowerView, Phase 2: RMU

    1) Drag & drop not changed!

    2) Full text, categorized field names

    3) Preview of Report Layout

    4) Overall Look & Feel is Contemporary

    Powerview phase 2 rmu1
    PowerView, Phase 2: RMU

    5) No extra columns

    6) Formatting can be saved

    Powerview version 5 0
    PowerView Version 5.0

    • Validation

      • Changes in Prototype were well received.

    • Tweaks

      • “My Favorites” was not an intuitive menu name

        • Worked for Solution Folders, but not here. Since PowerView is not a Web application, the “My Favorites” web metaphor was not effective.

        • Solution: Changed menu name to MyReports for launch.

    The upshot
    The Upshot

    Harder To Learn





    To Learn





    Contact information
    Contact Information

    Mark Safire

    Director of User Experience Research

    Sachs Insights

    200 Varick Street, Suite 500

    New York, NY 10014


    212-924-1600 x168