1 / 55

Empirically Validated Web Page Design Metrics

Explore the usability gap and the need for empirical validation of web page design metrics. Discover possible solutions and current design analysis tools. Learn about quantitative measures impacting usability.

rlerma
Download Presentation

Empirically Validated Web Page Design Metrics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Empirically Validated Web Page Design Metrics Melody Y. Ivory, Rashmi R. Sinha, Marti A. Hearst UC Berkeley CHI 2001

  2. 196M new Web sites in the next 5 years [Nielsen99] ~20,000 user interface professionals [Nielson99] The Usability Gap Empirically Validated Web Page Design Metrics

  3. A shortage of user interface professionals [Nielson99] The Usability Gap 196M new Web sites in the next 5 years [Nielsen99] Most sites have inadequate usability [Forrester, Spool, Hurst] (users can’t find what they want 39-66% of the time) Empirically Validated Web Page Design Metrics

  4. The Usability Problem • NON-professionals need to create websites • Guidelines are helpful, but • There are MANY usability guidelines • Survey of 21 web guidelines found little overlap [Ratner et al. 96] • Why? • One idea: because they are not empirically validated • Sometimes imprecise • Sometimes conflict Empirically Validated Web Page Design Metrics

  5. Possible Solutions: Tools to Help Non-Professional Designers • Examples: • A “grammar checker” to assess guideline conformance • Imperfect • Only suggestions – not dogma • Automatic comparison to highly usable pages/sites • Automatic template suggestions • How to create these? Empirically Validated Web Page Design Metrics

  6. Current Design Analysis Tools • Some tools report on easy-to-measure attributes • Compare measures to thresholds • Stein (Rating Game), Theng & Marsden, Thimbley (Gentler) • Not empirically validated • Guideline conformance • CAST (Bobby), Scholtz & Laskowski (WebSAT), Lift Online • Perceptually based heuristics • Faraday (Design Advisor) • Small subset of features [Brajnik00] • Simplistic • Not empirically validated Empirically Validated Web Page Design Metrics

  7. The WebTANGO Approach • Models of good design by looking at existing designs • Empirical foundation for easy-to-measure attributes • Focus on information-centric sites • First work to take a large set of sites and analyze them Empirically Validated Web Page Design Metrics

  8. The Investigation • Using quantitative measures to predict Web site ratings • Followup investigation [HFW00] • Given • 1898 pages from 400+ sites • 11 quantitative measures to assess various Web page aspects • Questions • Which features distinguish well-designed web pages? • Can metrics predict ratings? • Are there differences for categories of pages? Empirically Validated Web Page Design Metrics

  9. Webby Awards 2000 • 2000 sites initially • 27 topical categories • We studied sites from information-centric categories • Finance, education, community, living, health, services • 100 judges • International Academy of Digital Arts & Sciences • 3 rounds of judging Empirically Validated Web Page Design Metrics

  10. Webby Awards 2000 • 6 criteria • Content • Structure & navigation • Visual design • Functionality • Interactivity • Overall experience • Scale: 1-10 (highest) • Nearly normally distributed • What are judgments about? Empirically Validated Web Page Design Metrics

  11. Webby Awards 2000 • Content criterion is best predictor • Visual design criterion is worst predictor • User study of 57 sites • Ratings reflect usability Empirically Validated Web Page Design Metrics

  12. Quantitative Measures:Aspects Impacting Usability • Identified 42 attributes from the literature • Roughly characterized: • Page Composition • words, links, images, … • Page Formatting • fonts, lists, colors, … • Overall Page Characteristics • information & layout quality, download speed, … Empirically Validated Web Page Design Metrics

  13. Quantitative Measures: Word Count Empirically Validated Web Page Design Metrics

  14. Quantitative Measures: Body Text % Empirically Validated Web Page Design Metrics

  15. Quantitative Measures: Emphasized Body Text % Empirically Validated Web Page Design Metrics

  16. Quantitative Measures: Text Positioning Count Empirically Validated Web Page Design Metrics

  17. Quantitative Measures: Text Cluster Count Empirically Validated Web Page Design Metrics

  18. Quantitative Measures: Link Count Empirically Validated Web Page Design Metrics

  19. Quantitative Measures: Page Size (Bytes) Empirically Validated Web Page Design Metrics

  20. Quantitative Measures: Graphic % Empirically Validated Web Page Design Metrics

  21. Quantitative Measures: Graphic Count Empirically Validated Web Page Design Metrics

  22. Quantitative Measures: Color Count Empirically Validated Web Page Design Metrics

  23. Quantitative Measures: Font Count Empirically Validated Web Page Design Metrics

  24. Characterizing Measures:A View of Web Site Structure[Newman et al. DIS00] • Information design • structure, categories of information • Navigation design • interaction with information structure • Graphic design • visual presentation Courtesy of Mark Newman Empirically Validated Web Page Design Metrics

  25. Characterizing Measures:Web Site Structure Assessed

  26. Study Method • The Webby factor • Principle components analysis of the 6 criteria • Accounted for 91% of the variance • Two comparisons • Model 1: Highly rated sites (top 33%) vs. the rest • Using the overall Webby score • Model 2: Highly rated sites vs. bottom 33% • Using the Webby factor Empirically Validated Web Page Design Metrics

  27. Findings • We can accurately classify web pages • Linear discriminant analysis • Model 1: For highly-rated sites vs. rest • 67% correct when not considering content categories • 73% correct when taking content categories into account • Model 2: For highly-rated sites vs. bottom • 65% correct when not considering content categories • 80% correct when taking content categories into account Empirically Validated Web Page Design Metrics

  28. Findings • Top vs. bottom • Webby factor • Linear discriminant analysis • Better for categories Empirically Validated Web Page Design Metrics

  29. Deeper Analysis • Which metrics matter? • Linear regression analysis • Backward elimination until adjusted R² reduced • All metrics played a role • Compared small, medium, and large pages • Across the board (preliminary profiles) • Good pages had significantly smaller graphics percentage • Good pages had less emphasized body text • Good pages had more colors (on text) Empirically Validated Web Page Design Metrics

  30. Good small pages have (according to beta coefficients) Slightly more content Smaller page sizes Fewer graphics More font variations Suggests that these pages Have faster download times corroborated by a download time metric Use different fonts for headings vs. the rest of the text Examples Services (Home Pages) Top Bottom Small pages (66 words on average) Empirically Validated Web Page Design Metrics

  31. Medium pages (230 words on average) • Good medium pages • Emphasize less of the body text • Appear to organize text into clusters (e.g., lists and shaded table areas) • Use colors to distinguish headings from body text • Suggests that these pages • Are easier to scan Empirically Validated Web Page Design Metrics

  32. Large pages (827 words on average) • Good large pages have • More headings • More links • Are larger but have fewer graphics • Probably attributable to style sheets • Suggests that good large pages • Are easier to scan and facilitate information seeking Empirically Validated Web Page Design Metrics

  33. Why does this work? • Content is most important predictor • BUT there’s predictive power in other aspects • Visual and navigation design • Verifies preliminary investigation [HFW00] • Possibly: Good design is good design all over • Note: we are NOT suggesting we can characterize: • Aesthetics or subjective preferences Empirically Validated Web Page Design Metrics

  34. Comparable Designs Web Site Design Profiles Analysis Tool Favorite Designs • Prediction • Similarities • Differences • Suggestions How might we use this? Empirically Validated Web Page Design Metrics

  35. Future work • Distinguish according to page role • Home page vs. content vs. index … • Better metrics • More aspects of info, navigation, and graphic design • Site level as well as page level • Category-based profiles • Use clustering to create profiles of good and poor sites • These can be used to suggest alternative designs Empirically Validated Web Page Design Metrics

  36. In Summary • Automated tools should help close the Web Usability Gap • We have a foundation for a new methodology • Empirical, bottom up • We can empirically distinguish good pages • Empirical validation of design guidelines • Can build profiles of good vs. poor sites • Eventually build tools to help users assess designs Empirically Validated Web Page Design Metrics

  37. More Information • http://webtango.berkeley.edu Empirically Validated Web Page Design Metrics

  38. Extra Slides • Quantitative Metrics • Statistical Measures • Category Differences Empirically Validated Web Page Design Metrics

  39. Quantitative Metrics: Text Measures • Text Content • Total visible words; also used for Reading Complexity over all text (I,O) • Words that are body text (I) • Words that are display text (i.e., headers) (I) • Gunning Fog Index (O) • ratios of words, sentences and words with 3+ syllables) • computed over prose • Number of words in the page’s title (I) • Number of prose words (for Reading Complexity) (O) • Number of big prose words (for Reading Complexity) (O) • Number of sentences for prose words (for both Reading Complexity measures) (O) Empirically Validated Web Page Design Metrics

  40. Quantitative Metrics: Text Measures • Text Formatting • Body text words that are emphasized (e.g., bold, capitalized or near !'s) (G,O) • Changes in text position from flush left (G,O) • How are aligned text blocks perceived? example • Text areas highlighted with color, bordered regions, rules or lists (G,O) • How are colored clusters perceived? Example1Example2 • Gunning Fog Index (ratios of words, sentences and words with 3+ syllables); computed over all text (O) • Number of big words (for Reading Complexity over all text) (O) Empirically Validated Web Page Design Metrics

  41. Quantitative Metrics: Text Measures • Text Formatting • Minimum font size (G,O) • Maximum font size (G,O) • Average font size (G,O) Empirically Validated Web Page Design Metrics

  42. Quantitative Metrics: Link Measures • Link Content • Words that are link text (I,N) • Average number of words in link text (N) • Total links (N) • Duplicate links (N) • Internal links (N,O) Empirically Validated Web Page Design Metrics

  43. Quantitative Measures: Link Measures • Link Formatting • Total text links (N,I) • Links that are images (N,G,O) • Link areas highlighted with color, bordered regions, rules or lists (G,O) Empirically Validated Web Page Design Metrics

  44. Quantitative Measures: Image Measures • Image Content • Total images (G) • Animated images (G,O) • Images that are links (G,N,O) • Duplicate images (G) • Average image height (G,O) • Average image width (G,O) Empirically Validated Web Page Design Metrics

  45. Quantitative Metrics: Color Measures • Purpose • Quantify color usage (how many, for what) • Assess color quality (Murch) Empirically Validated Web Page Design Metrics

  46. Quantitative Metrics: Color Measures • Color Usage • Total colors (G) • Number of body text colors (G,O) • Number of display text (headers) colors (G,O) • Number of link colors (N,G,O) • Number of default link colors (N,G,O) • Average number of times a color is used (G,O) • What’s the right way to do this? • Number of good color combinations (Murch) (G,O) • Number of bad color combinations (Murch) (G,O) • Number of neutral color combinations (Murch) (G,O) Empirically Validated Web Page Design Metrics

  47. Quantitative Metrics: Page Formatting Measures • Measures • Total fonts (face, size, bold, italic combinations) (G,O) • Height of page in pixels (600 pixel window) (G,O) • Width of page in pixels (800 pixel window) (G,O) • Total page pixels (G,O) • Total graphic pixels (G,O) • Number of horizontal scrolls required (G,O) • Number of vertical scrolls required (G,O) Empirically Validated Web Page Design Metrics

  48. Quantitative Metrics: Download Time Measure • Estimating Download Time • Predictive model based on 56.6K modem (41.2K connection speed) • .957 correlation with measured time • Explains 91% of the variance Empirically Validated Web Page Design Metrics

  49. Quantitative Metrics: Download Time Measure • Model Measures • Total bytes for html (page & stylesheets) (O) • Total bytes for images (O) • Total bytes for scripts (O) • Total bytes for objects (applets, objects, layers) (O) • HTML files (.htm & .css) (O) • Image files (O) • Object files (applets, layer, sound, etc.) (O) • Script files (O) • Total number of scripts, applets, objects, etc. (O) • Number of tables (O) Empirically Validated Web Page Design Metrics

  50. Quantitative Metrics: Other Measures • Measures • Level of non-home pages within the site (N,O) • Number of ads (O) • Number of accessibility syntax errors on page (Bobby) (O) • Number of HTML syntax errors on page (HTML Lint) (O) • Number of text fields, radio boxes, buttons and other form objects (O) • Number of words in meta tags (keyword & description) (I,N) • Functional type (e.g., home, index, content, reference, or other) (O) Empirically Validated Web Page Design Metrics

More Related