Reporting

1. Reporting

2. Stages of project at which reports are likely • User and task analysis • Who (potential) users are, how they do their work, what they do, what they need • Competitive analysis • What the competition is doing • Prototype testing (heuristics, guidelines, usability testing…) • Performance, strengths, weaknesses, recommended changes • On-going evaluation • Performance indicators • Users characteristics, behavior, opinions • What is working well/poorly

3. Tailoring message to the audience • How • Speak their language • Address their concerns • To whom; what are their specific concerns? E.g., • Management • Quality; scheduling; trade-offs: decide what’s worth time & effort • Design team • Need to understand recommendations and rationale • Have to make Marketing, Product-planning department • How will they sell this? • Tend to focus more on demographics than work practice • Customers/Users (especially internal) • Partners in redesigning work • Have to accept the final product

4. What to report • What they want to know • What you think they need to know • What you did • Scope, Goals • Methods • What you found • Recommendations

5. What you need to convey • Understanding of user/customer • Needs, preferences • Work practices, setting • Reactions to design • Your assessment of the design • Recommendations • At a level of abstraction that allows multiple solutions • Rationale for recommendations • To establish your credibility • To provide a larger context for understanding recommendations

6. Media • Written reports • Circulate widely; persist • Video • Complexity and persuasiveness of images • Labor-intensive • Other presentations • Live • Canned

7. Completeness Answer variety of concerns Provide context for understanding recommendations Protect from attack, e.g., re method Brevity Take little time; come to the point People don’t read Clarity: what are your major findings? People remember a few key points Balance completeness and brevity

8. Balance completeness and brevity II • Executive summary • Graphics • E.g. charts of findings and recommendations • Representations of data • Statistics, graphics • Appendices (where the details go) • Copies of test materials and instruments • More detailed findings, e.g., transcripts of sessions, comprehensive reports of qualitative findings

9. Kinds of data you are likely to have • Quantitative • Survey results • Results from usability testing: usability metrics of various kinds • Server log analyses • Qualitative • Task analyses • Interview findings • Results from focus groups • Comments people make, e.g., during testing • Graphics • Videotape, still photos • Examples (e.g. of forms, documents) • [Audiotape usually used only for your purposes, although sound clips are possible]

10. Qualitative Data • Descriptive info about users and their work • Narratives: stories, scenarios, personae • Graphics • Workflow diagrams • User-task matrices • Physical/space diagrams • images • Possible orientations • Person (follow a person/position/role) • Place (stay in one place and describe what flows through) • Artifact (where an artifact goes, how it gets transformed) • Task (how a task is completed)

11. Quantitative Data • Summarize data • Descriptive statistics • univariate • Frequency , Histograms • Bivariate, multivariate • Cross tabs • Time effects • Graphs – time is always the x axis • Measures of central tendency • Mean, median • Distribution around the mean • Maximum, minimum • Investigate causality or at least correlation • Cross-tabs • Correlations, graphs

12. Converting qualitative to quantitative • Measuring occurrences • E.g., # of times people complain about x • Coding data • Categorizing open-ended responses to questions

13. Quantitative data: topics covered in IS208 • Measures of central tendency (various forms of “averages”) • Frequencies • Histograms • Cross-tabulations • Recoding & transforming data