Download
debriefing recommendations n.
Skip this Video
Loading SlideShow in 5 Seconds..
Debriefing, Recommendations PowerPoint Presentation
Download Presentation
Debriefing, Recommendations

Debriefing, Recommendations

5 Views Download Presentation
Download Presentation

Debriefing, Recommendations

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Debriefing, Recommendations CSSE 376, Software Quality Assurance Rose-Hulman Institute of Technology May 3, 2007

  2. Outline • Post-test • Questionnaire • Debriefing • Final Report • Findings • Analysis • Recommendations

  3. Post-test Questionnaire • Purpose: collect preference information from participant • May also be used to collect background information

  4. Likert Scales Overall, I found the widget easy to use • strongly agree • agree • neither agree nor disagree • disagree • strongly disagree

  5. Semantic Differentials Circle the number closest to your feelings about the product: Simple ..3..2..1..0..1..2..3.. Complex Easy to use ..3..2..1..0..1..2..3.. Hard to use Familiar ..3..2..1..0..1..2..3.. Unfamiliar Reliable ..3..2..1..0..1..2..3.. Unreliable

  6. Free-form Questions I found the following aspects of the product particularly easy to use ________________________________ ________________________________ ________________________________

  7. First Cartoon of the Day

  8. Debriefing • Purpose: find out why the participant behaved the way they did. • Method: interview • May focus on specific behaviors observed during the test.

  9. Debriefing Guidelines • Review participant's behaviors and post-test answers. • Let participant say whatever is on their mind. • Start with high-level issues and move on to specific issues. • Focus on understanding problems, not on problem-solving.

  10. Debriefing Techniques • What did you remember? When you finished inserting an appointment did you notice any change in the information display? • Devil's advocate Gee, other people we've brought in have responded in quite the opposite way.

  11. Findings • Summarize what you have learned • Performance • Preferences

  12. Performance Findings • Mean time to complete • Median time to complete • Mean number of errors • Median number of errors • Percentage of participants performing successfully

  13. Preference Findings • Limited-choice questions • sum each answer • compute averages to compare questions • Free-form questions • group similar answers

  14. Second Cartoon of the Day

  15. Analysis • Focus on problematic tasks • 70% of participants failed to complete task successfully • Conduct a source of error analysis • look for multiple causes • look at multiple participants • Prioritize problems by criticality Criticality = Severity + Probability

  16. Recommendations (1/2) • Get some perspective • wait until a couple days after testing • collect thoughts from group of testers • get buy-in from developers • Focus on highest impact areas

  17. Recommendations (2/2) • Ignore "political considerations" for first draft • Provide short-term and long-term solutions • short-term: will not significantly affect development schedule • long-term: needed for ultimate success of product