1 / 31

Presentation

Defining a Product Vision using Remote Research Methodologies presented to The 14 th Annual Usability Professionals’ Association Conference July 1 st , 2005 Mark Safire Director of User Experience Research Sachs Insights. Presentation. Who am I The Products & The Vision

fredrica
Download Presentation

Presentation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Defining a Product Vision usingRemote Research Methodologiespresented toThe 14th AnnualUsability Professionals’ AssociationConferenceJuly 1st, 2005Mark SafireDirector of User Experience ResearchSachs Insights

  2. Presentation • Who am I • The Products & The Vision • The Company – NPD Group • The Software Products – Solution Folders & PowerView • The User Profiles • The Research Challenge • Remote Moderated Usability • Definition; Benefits; Challenges • Audience Participation – Experiences with Remote Research • The Transformation of PowerView through Research • Research Road Map • PowerView Research • Phase 1: Remote Contextual Inquiry • Phase 2: Remote Moderated Usability • The Upshot • Q&A

  3. Who Am I? • Mark Safire • Director of User Experience Research, Sachs Insights • Qualitative research provider for NPD Group online/offline software products since 2002. • Spoken at UPA 2003 (Presentation) and UPA 2004 (Idea Market) on Remote Moderated Usability. • Also at NYC and Dallas UPA chapters, and for Nielsen Norman Group conferences in Las Vegas and Amsterdam. • 2003 Presentation is online at www.sachsinsights.com/services/remotemoderatedusability.html

  4. Sachs Insights Qualitative research consultancy founded in 1987 • Serve automotive, fashion, financial services, media, pharmaceutical, technology and travel industries. Practice Areas Include: • Brand & Advertising Research • New Product & Service Development • User Experience Research In our User Experience practice, we conduct: • Concept & Process Focus Groups • Taxonomy & Nomenclature Focus Groups • Usability • Contextual Inquiry

  5. The Company • A leading provider of sales and marketing information based on combined point-of-sale and consumer data for over a dozen major industries. • Apparel, appliances, automotive, beauty, cellular, consumer electronics, food and beverage, foodservice, footwear, housewares, imaging, information technology, movies, music, software, toys, travel, video games, ...* • Supplemented by consumer panel data to give a more complete picture – demographics, etc. • Data from “over 230 retail partners, representing over 90,000 doors worldwide”* • Clients: Manufacturers, Retailers, Analysts • Quoted in media regularly *Source: www.npd.com

  6. The Software Products • Goals of Software • Allow customers to analyze data that NPD has collected and organized. • Usage Scenarios • Monthly reports: Same reports, different numbers • Ad-hoc queries: Different reports each time • Sample Queries for Reports • What is my market share vs. my competitors? • How are inkjet printer sales trending over the past 3 mo.’s? • What were the top ten products in $ sold last month?

  7. Solution Folders PowerView The Software Products

  8. Solution Folders Make Reports from Templates PowerView Make Reports from Scratch The Software Products Less Less Shorter Little (Online) More More Longer A Lot (Offline) Flexibility Room for Error Learning Curve Setup Required

  9. The Software Products PowerView Harder To Learn Solution Folders Easier To Learn Less Flexible More Flexible

  10. The User Profiles – Ingoing Assumptions • Disseminators • Market Research experts – the “gatekeepers” • Experienced in performing quantitative analysis • Pull, analyze, develop reports for others • Users • Support personnel, bus. analysts & owners, sales & mktg. • Less experienced in performing quantitative analysis • Pull, analyze, developer reports for themselves • Viewers • Bus. analysts & owners, sales & mktg. • Less experienced in performing quantitative analysis • Review reports created by others

  11. Research Challenge – Finding Respondents • Some had low incidence • “User” profile not yet a significant percentage. • Geographical dispersion • Key retailers not clustered • Expensive and time consuming to fly them in • 2nd Round of Research – Timing • Week before Christmas Solution: Remote Moderated Usability

  12. Usability in a Lab Respondent Moderator Observer(s) 4

  13. Remote Moderated Usability Respondent Moderator Observer(s) 7

  14. Remote Moderated Usability • Differences from In-Lab Usability • Different Locations • Software Tool • Teleconference • Observers usually apart from Moderator 8

  15. Remote Moderated Usability • Benefits • Recruiting benefits • Reach geographically dispersed populations • Reach hard to recruit populations – don’t leave home/office • Avoid cost and inconvenience of travel for all parties involved. • Get “buy-in” from stakeholders in distant locations • NPD Research Conditions: • All had access to high-speed Internet connections • Participants & their companies were open to the technology • NPD Group had a WebEx license and in-house expertise, used primarily for remote demos and help desk support 9

  16. Remote Moderated Usability • Challenges • Body Language • Rapport • Interruptions (phone, visitors, email, IM, …) • Logistics (time zones, ...) • Security & Privacy • Observer dynamics • Broadband connection recommended • Not for newbies & less coordinated 10

  17. Audience Participation • Any Experiences? 1-2 Min. Summary: • Your Name, Title, & Company • Research Description: Industry, (Client), Audience, & Objective • Why Remote Moderated Usability? • Challenges you faced & Tips for dealing with them

  18. Remote Moderated Usability Tools & Methods • Many tools available • NPD done with WebEx Meeting Manager • Others commercial apps/ASPs to consider: • IBM Lotus Web Conferencing (f.k.a. IBM Lotus Same Time) • Microsoft LiveMeeting (f.k.a. Placeware) • Citrix GoToMeeting (f.k.a. DesktopStreaming) • Macromedia Breeze • Other free or open source solutions to consider: • Microsoft Messenger/MSN Messenger (f.k.a. Microsoft NetMeeting) • VNC (open source – various implementations, including RealVNC) • Others kinds of tools • Full-time Remote Control Apps (See pcmag.com) • pcAnywhere, GoToMyPC, Timbuktu, etc. 11

  19. Solution Folders Initial Launch In-Lab Interviews Optimize Navigation Remote Moderated Usability PowerView Needs Assessment Internal Focus Groups Remote Contextual Inquiry Prototype before Relaunch (version 5.0) Remote Moderated Usability Research Roadmap

  20. Phase I Remote Contextual Inquiry How do they use it? They define & demo tasks. Does it meet actual needs? See Role in Work Process Ultimate reports delivered How easy is it to use? First Use / Experienced Use Things to change /Things NOT to change Phase 2 Remote Moderated Usability Can they use it? Use their pre-defined tasks Do fixes meet needs? Role-play work process Observe ease of use. First Use / Experienced Use Points of pain improved? / Have to relearn a lot? PowerView Research

  21. Phase 1, Remote Contextual Inquiry (RCI) • First time & “Ad Hoc use” • Even old-timers stumble when trying to create new reports. • Drag & drop interface – good idea, don’t change! • Inscrutable 8 character field names • Hard to envision report layout from interface • Especially for “nested” reports • Expert & Repeated use • Gargantuan monthly reports: Same thing, different numbers • Each report is run individually, not as a “batch” • Typically formatted each time in Excel – not saved in tool. • Often requires specific colors for company and competitors in for tables and charts. • Extra columns created during export to show “suppression.” Everyone: Look & Feel “dated” – want standard Windows conventions

  22. Phase 2, Remote Moderated Usability (RMU) • First time & “Ad Hoc use” • Drag & drop approach still used. • Comprehensible, full text, categorized field names • But can change back to 8 char. abbrev’s in preferences... • Preview of report layout • Expert & Repeated use • Run as a batch • But “Favorites” menuv – confusing metaphor • Final: “MyReports” • Save formatting • No extra columns! • gray or red italicized numbers with asterisks

  23. PowerView, Phase 1, RCI 1) Drag & drop a good idea: don’t change! 2) Inscrutable 8 character field names, without any categorization 3) Hard to envision report layout from interface 4) Look & Feel “dated” – not standard Windows conventions.

  24. PowerView, Phase 1, RCI Widget Production Quarterly Trend What they need

  25. PowerView, Phase 1, RCI 5) Extra columns 6) Formatting not saved What they get

  26. PowerView, Phase 2: RMU 1) Drag & drop not changed! 2) Full text, categorized field names 3) Preview of Report Layout 4) Overall Look & Feel is Contemporary

  27. PowerView, Phase 2: RMU 5) No extra columns 6) Formatting can be saved

  28. PowerView Version 5.0 • Validation • Changes in Prototype were well received. • Tweaks • “My Favorites” was not an intuitive menu name • Worked for Solution Folders, but not here. Since PowerView is not a Web application, the “My Favorites” web metaphor was not effective. • Solution: Changed menu name to MyReports for launch.

  29. The Upshot Harder To Learn PowerView Solution Folders Easier To Learn Less Flexible More Flexible

  30. Contact Information Mark Safire Director of User Experience Research Sachs Insights 200 Varick Street, Suite 500 New York, NY 10014 www.sachsinsights.com msafire @sachsinsights.com 212-924-1600 x168 www.remoteusability.com

More Related