1 / 29

A Scientific Approach to Improving VUI Design

Peter U. Leppik, CEO Rick Rappe, VP of Business Development Vocal Laboratories Inc. A Scientific Approach to Improving VUI Design. Agenda. Principles of Gourmet Customer Service Data Collection Toolbox The Project Life-Cycle Conclusion/Q&A. Principles of Gourmet Customer Service.

hien
Download Presentation

A Scientific Approach to Improving VUI Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Peter U. Leppik, CEORick Rappe, VP of Business DevelopmentVocal Laboratories Inc. A Scientific Approach to Improving VUI Design

  2. Agenda • Principles of Gourmet Customer Service • Data Collection Toolbox • The Project Life-Cycle • Conclusion/Q&A

  3. Principles of Gourmet Customer Service • Nearly every effort to improve service makes at least one of these mistakes: • No data is collected from actual callers. • The data collected does not relate to business goals. • There is not enough data collected to be statistically significant. • There is no context or understanding of the data (for example: what is a “good” satisfactions score?). • No action is taken based on the data collected.

  4. The Scientific Approach • Set Goals • Use relevant business goals, not technical metrics • Gather data • Multiple kinds of tests may be needed to get a complete picture • Take action • Validate • Repeat the tests from Step 2. • Commit: Go back to Step 1.

  5. Data Collection Toolbox • There is no one way to collect data about a VUI or call center. • A VUI designer who only uses one test method is like a carpenter who only owns a hammer.

  6. The Toolbox: General Tools • Test Goals • Statistical Considerations • Margin of Error • Probability of Finding Problems • Prototypes • Functional Prototype (aka “Rapid Prototype”) • Wizard-of-Oz Prototype

  7. The Toolbox: Idea Generating Tools • Heuristics • “Best Practices” • Expert Opinions, “Audits,” “Assessments” • Agent Feedback • Call Recording and Monitoring

  8. The Toolbox: Data Generating Tools • Controlled Testing • Lab Tests • Rapid Assessments • Large Controlled Tests • Follow-Up Surveys • End-of-Call Surveys • Not Recommended due to extreme sample bias

  9. The Toolbox: More Data Generating Tools • Friends & Family & Employee Tests • Not Recommended due to unrepresentative sample • Automated Load Testing • Traversal Testing • Call Logs and Stats

  10. The Project Life-Cycle • Every project goes through certain phases. • Analysis • Design • Code • Test • Release • Operational • Every phase has different testing requirements

  11. Analysis Phase • Lay out the project scope, goals, and budget • Ask the big questions • Who are these callers, and how do they live? • Why are they using the phone and not (e.g.) the web? • What is the competition doing? • Look at the big picture and big ideas: • Generate ideas from consultants, vendors, agents, and customers • Develop baselines & benchmarks

  12. Analysis Phase Tools • Controlled Testing • Benchmarking, competitive analysis, baseline measurements • Need to understand your callers before you start • Agent Feedback • Follow-up Surveys • Call Recordings • Heuristics • “Best practices,” consultants’ opinions, etc.

  13. Design Phase • This is when you plan the details of the VUI • Use a Prototype  Test  Prototype  Test iterative design cycle • The keys to success are speed and flexibility • Test several designs, sometimes in the same day • Don’t blow the entire testing budget on The Usability Test

  14. Design Phase Tools • Prototypes • WOZ or Functional Prototype • Controlled testing • Rapid Assessment or Lab Test • Lets you use an iterative design process • Nothing ends arguments faster than solid data.

  15. Code Phase • Implement the design • Most testing is for debugging • Unit testing and some functional testing • Relatively little VUI testing takes place here • Implementation may require some VUI changes; these should be tested

  16. Test Phase • Last stage before rolling out the application to live callers • This is where the most thorough testing takes place

  17. Test Phase Tools • Traversal test • Ensures that the implementation matches the design • Large controlled tests • Designed to catch even small usability glitches • Automated load testing • Done last because changes in the first two test phases can affect load handling

  18. The Test Phase can also be the Acceptance Phase • Business goals are what are important, not the exact design • They are what should be in the contract. • Tests which measure business goals should be the acceptance criteria. • Many of these tests can be used in the design phase.

  19. Release Phase • This is the process of rolling out the application • By this time, you should be confident that the system works • Confident doesn’t mean cocky • This is the time for a phased roll-out • Some call it a “pilot test” • This is a deployment strategy, not a testing strategy

  20. Operational Phase • The work doesn’t end when the system is live! • Application Drift • External changes • Company • Caller Lifestyle • Competition • This looks a lot like the analysis phase for the next project….

  21. Operational Phase Tools • Benchmarking (Controlled Testing) • Competitive analysis (Controlled Testing) • Follow-up surveys • Agent feedback • Monitoring call stats and logs • Listening to call recordings

  22. Lessons Learned • Good service does not have to be more expensive than bad service • Customers are smarter than machines • Every system can stand some improvement • The best VUI designs come from paying attention to the right things at the right time

  23. Q & A

More Related