1 / 37

Empirically Validated Web Page Design Metrics

Empirically Validated Web Page Design Metrics. Melody Y. Ivory, Rashmi R. Sinha, Marti A. Hearst UC Berkeley CHI 2001. 196M new Web sites in the next 5 years [Nielsen99]. ~20,000 user interface professionals [Nielson99]. The Usability Gap.

golda
Download Presentation

Empirically Validated Web Page Design Metrics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Empirically Validated Web Page Design Metrics Melody Y. Ivory, Rashmi R. Sinha, Marti A. Hearst UC Berkeley CHI 2001

  2. 196M new Web sites in the next 5 years [Nielsen99] ~20,000 user interface professionals [Nielson99] The Usability Gap Empirically Validated Web Page Design Metrics

  3. A shortage of user interface professionals [Nielson99] The Usability Gap 196M new Web sites in the next 5 years [Nielsen99] Most sites have inadequate usability [Forrester, Spool, Hurst] (users can’t find what they want 39-66% of the time) Empirically Validated Web Page Design Metrics

  4. The Usability Problem • NON-professionals need to create websites • Guidelines are helpful, but • There are MANY usability guidelines • Survey of 21 web guidelines found little overlap [Ratner et al. 96] • Why? • One idea: because they are not empirically validated • Sometimes imprecise • Sometimes conflict Empirically Validated Web Page Design Metrics

  5. Possible Solutions: Tools to Help Non-Professional Designers • Examples: • A “grammar checker” to assess guideline conformance • Imperfect • Only suggestions – not dogma • Automatic comparison to highly usable pages/sites • Automatic template suggestions • How to create these? Empirically Validated Web Page Design Metrics

  6. Current Design Analysis Tools • Some tools report on easy-to-measure attributes • Compare measures to thresholds • Stein (Rating Game), Theng & Marsden, Thimbley (Gentler) • Not empirically validated • Guideline conformance • CAST (Bobby), Scholtz & Laskowski (WebSAT), Lift Online • Perceptually based heuristics • Faraday (Design Advisor) • Small subset of features [Brajnik00] • Simplistic • Not empirically validated Empirically Validated Web Page Design Metrics

  7. The WebTANGO Approach • Models of good design by looking at existing designs • Empirical foundation for easy-to-measure attributes • Focus on information-centric sites • First work to take a large set of sites and analyze them Empirically Validated Web Page Design Metrics

  8. The Investigation • Using quantitative measures to predict Web site ratings • Followup investigation [HFW00] • Given • 1898 pages from 400+ sites • 11 quantitative measures to assess various Web page aspects • Questions • Which features distinguish well-designed web pages? • Can metrics predict ratings? • Are there differences for categories of pages? Empirically Validated Web Page Design Metrics

  9. Webby Awards 2000 • 2000 sites initially • 27 topical categories • We studied sites from information-centric categories • Finance, education, community, living, health, services • 100 judges • International Academy of Digital Arts & Sciences • 3 rounds of judging Empirically Validated Web Page Design Metrics

  10. Webby Awards 2000 • 6 criteria • Content • Structure & navigation • Visual design • Functionality • Interactivity • Overall experience • Scale: 1-10 (highest) • Nearly normally distributed • What are judgments about? Empirically Validated Web Page Design Metrics

  11. Webby Awards 2000 • Content criterion is best predictor • Visual design criterion is worst predictor • User study of 57 sites • Ratings reflect usability Empirically Validated Web Page Design Metrics

  12. Quantitative Measures:Aspects Impacting Usability • Identified 42 attributes from the literature • Roughly characterized: • Page Composition • words, links, images, … • Page Formatting • fonts, lists, colors, … • Overall Page Characteristics • information & layout quality, download speed, … Empirically Validated Web Page Design Metrics

  13. Quantitative Measures: Word Count Empirically Validated Web Page Design Metrics

  14. Quantitative Measures: Body Text % Empirically Validated Web Page Design Metrics

  15. Quantitative Measures: Emphasized Body Text % Empirically Validated Web Page Design Metrics

  16. Quantitative Measures: Text Positioning Count Empirically Validated Web Page Design Metrics

  17. Quantitative Measures: Text Cluster Count Empirically Validated Web Page Design Metrics

  18. Quantitative Measures: Link Count Empirically Validated Web Page Design Metrics

  19. Quantitative Measures: Page Size (Bytes) Empirically Validated Web Page Design Metrics

  20. Quantitative Measures: Graphic % Empirically Validated Web Page Design Metrics

  21. Quantitative Measures: Graphic Count Empirically Validated Web Page Design Metrics

  22. Quantitative Measures: Color Count Empirically Validated Web Page Design Metrics

  23. Quantitative Measures: Font Count Empirically Validated Web Page Design Metrics

  24. Characterizing Measures:A View of Web Site Structure[Newman et al. DIS00] • Information design • structure, categories of information • Navigation design • interaction with information structure • Graphic design • visual presentation Courtesy of Mark Newman Empirically Validated Web Page Design Metrics

  25. Characterizing Measures:Web Site Structure Assessed

  26. Study Method • The Webby factor • Principle components analysis of the 6 criteria • Accounted for 91% of the variance • Two comparisons • Model 1: Highly rated sites (top 33%) vs. the rest • Using the overall Webby score • Model 2: Highly rated sites vs. bottom 33% • Using the Webby factor Empirically Validated Web Page Design Metrics

  27. Findings • We can accurately classify web pages • Linear discriminant analysis • Model 1: For highly-rated sites vs. rest • 67% correct when not considering content categories • 73% correct when taking content categories into account • Model 2: For highly-rated sites vs. bottom • 65% correct when not considering content categories • 80% correct when taking content categories into account Empirically Validated Web Page Design Metrics

  28. Findings • Top vs. bottom • Webby factor • Linear discriminant analysis • Better for categories Empirically Validated Web Page Design Metrics

  29. Deeper Analysis • Which metrics matter? • Linear regression analysis • Backward elimination until adjusted R² reduced • All metrics played a role • Compared small, medium, and large pages • Across the board (preliminary profiles) • Good pages had significantly smaller graphics percentage • Good pages had less emphasized body text • Good pages had more colors (on text) Empirically Validated Web Page Design Metrics

  30. Good small pages have (according to beta coefficients) Slightly more content Smaller page sizes Fewer graphics More font variations Suggests that these pages Have faster download times corroborated by a download time metric Use different fonts for headings vs. the rest of the text Examples Services (Home Pages) Top Bottom Small pages (66 words on average) Empirically Validated Web Page Design Metrics

  31. Medium pages (230 words on average) • Good medium pages • Emphasize less of the body text • Appear to organize text into clusters (e.g., lists and shaded table areas) • Use colors to distinguish headings from body text • Suggests that these pages • Are easier to scan Empirically Validated Web Page Design Metrics

  32. Large pages (827 words on average) • Good large pages have • More headings • More links • Are larger but have fewer graphics • Probably attributable to style sheets • Suggests that good large pages • Are easier to scan and facilitate information seeking Empirically Validated Web Page Design Metrics

  33. Why does this work? • Content is most important predictor • BUT there’s predictive power in other aspects • Visual and navigation design • Verifies preliminary investigation [HFW00] • Possibly: Good design is good design all over • Note: we are NOT suggesting we can characterize: • Aesthetics or subjective preferences Empirically Validated Web Page Design Metrics

  34. Comparable Designs Web Site Design Profiles Analysis Tool Favorite Designs • Prediction • Similarities • Differences • Suggestions How might we use this? Empirically Validated Web Page Design Metrics

  35. Future work • Distinguish according to page role • Home page vs. content vs. index … • Better metrics • More aspects of info, navigation, and graphic design • Site level as well as page level • Category-based profiles • Use clustering to create profiles of good and poor sites • These can be used to suggest alternative designs Empirically Validated Web Page Design Metrics

  36. In Summary • Automated tools should help close the Web Usability Gap • We have a foundation for a new methodology • Empirical, bottom up • We can empirically distinguish good pages • Empirical validation of design guidelines • Can build profiles of good vs. poor sites • Eventually build tools to help users assess designs Empirically Validated Web Page Design Metrics

  37. More Information • http://webtango.berkeley.edu Empirically Validated Web Page Design Metrics

More Related