1 / 43

Content Metadata and Search Remarks to the Dublin Core Workshop

Content Metadata and Search Remarks to the Dublin Core Workshop. Marti Hearst SIMS, UC Berkeley September 28, 2003. Resource Finding and the Web. Web search vs. collection search When a single page is all that’s needed, web search is fine Although validity is an issue Unsolved problem:

silas
Download Presentation

Content Metadata and Search Remarks to the Dublin Core Workshop

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Content Metadata and SearchRemarks to the Dublin Core Workshop Marti Hearst SIMS, UC Berkeley September 28, 2003

  2. Resource Finding and the Web • Web search vs. collection search • When a single page is all that’s needed, web search is fine • Although validity is an issue • Unsolved problem: • How to make source-focused search more intuitive on the web? • One idea (untested): task-based search Faceted Metadata in Search

  3. What about Content? • Dublin Core takes stances on the “content-neutral” aspects of metadata • Q: What about content? • The Metadata Marsh • Getting agreement on metadata terms is difficult • Even worse when talking about content! • A: Domain-specific solutions • Don’t worry about cross-domain consistency (a necessary drawback) • Success: b-to-b protocols Faceted Metadata in Search

  4. Hypothesis (as yet untested): Assuming we’ve focused on a domain, agreement on category assignment can converge much more quickly by: • Focusing on the applications that will use the category system. • Designing metadata to be used in interfaces that show items represented by many different categories in a highly flexible, but intuitive, manner. Faceted Metadata in Search

  5. One Example: Flamenco Project • Goal: create intuitive, inviting search interfaces that make use of hierarchical faceted metadata • Challenge: How to provide flexibility and power without overwhelming? (Answer: careful interface design) Faceted Metadata in Search

  6. The Flamenco Project Team Brycen Chun Ame Elliott Jennifer English Kevin Li Rashmi Sinha Kirsten Swearingen Ping Yee http://flamenco.berkeley.edu Research funded by: NSF CAREER Grant IIS-9984741 IBM Faculty Fellowship

  7. Our Approach • Integrate the search seamlessly into the information architecture. • Use proper HCI methodologies. • Use faceted metadata: • More flexible than canned hyperlinks • Less complex than full search • Help users see where to go next and return to what happened previously • What’s new? • Putting hierarchical facets into a useable interface. Faceted Metadata in Search

  8. GeoRegion + Time/Date + Topic Metadata: data about dataFacets: orthogonal categories Faceted Metadata in Search

  9. Hierarchical Faceted Metadata Example: Biological Subject Headings 1.Anatomy [A] 2. Organisms [B] 3. Diseases [C] 4. Chemicals and Drugs [D] 5. Analytical, Diagnostic and Therapeutic Techniques and Equipment [E] 6. Psychiatry and Psychology [F] 7. Biological Sciences [G] 8. Physical Sciences [H] 9. Anthropology, Education, Sociology and Social Phenomena [I] 10. Technology and Food and Beverages [J] 11. Humanities [K] 12. Information Science [L] 13. Persons [M] 14. Health Care [N] 15. Geographic Locations [Z] Faceted Metadata in Search

  10. Hierarchical Faced Metadata 1. Anatomy [A]Body Regions [A01] 2. [B] Musculoskeletal System [A02] 3. [C] Digestive System [A03] 4. [D] Respiratory System [A04] 5. [E] Urogenital System [A05] 6. [F] …… 7. [G] 8. Physical Sciences [H] 9. [I] 10. [J] 11. [K] 12. [L] 13. [M] Faceted Metadata in Search

  11. Hierarchical Faceted Metadata 1. Anatomy [A]Body Regions [A01] Abdomen [A01.047] 2. [B] Musculoskeletal System [A02] Back [A01.176] 3. [C] Digestive System [A03] Breast [A01.236] 4. [D] Respiratory System [A04] Extremities [A01.378] 5. [E] Urogenital System [A05] Head [A01.456] 6. [F] …… Neck [A01.598] 7. [G] …. 8. Physical Sciences [H] 9. [I] 10. [J] 11. [K] 12. [L] 13. [M] Faceted Metadata in Search

  12. Hierarchical Faceted Metadata 1. Anatomy [A]Body Regions [A01] Abdomen [A01.047] 2. [B] Musculoskeletal System [A02] Back [A01.176] 3. [C] Digestive System [A03] Breast [A01.236] 4. [D] Respiratory System [A04] Extremities [A01.378] 5. [E] Urogenital System [A05] Head [A01.456] 6. [F] …… Neck [A01.598] 7. [G] …. 8. Physical Sciences [H] Electronics 9. [I] Astronomy 10. [J] Nature 11. [K] Time 12. [L] Weights and Measures 13. [M] …. Faceted Metadata in Search

  13. Hierarchical Faceted Metadata 1. Anatomy [A]Body Regions [A01] Abdomen [A01.047] 2. [B] Musculoskeletal System [A02] Back [A01.176] 3. [C] Digestive System [A03] Breast [A01.236] 4. [D] Respiratory System [A04] Extremities [A01.378] 5. [E] Urogenital System [A05] Head [A01.456] 6. [F] …… Neck [A01.598] 7. [G] …. 8. Physical Sciences [H] Electronics Amplifiers 9. [I] Astronomy Electronics, Medical 10. [J] Nature Transducers 11. [K] Time 12. [L] Weights and Measures 13. [M] …. Faceted Metadata in Search

  14. Hierarchical Faceted Metadata 1. Anatomy [A]Body Regions [A01] Abdomen [A01.047] 2. [B] Musculoskeletal System [A02] Back [A01.176] 3. [C] Digestive System [A03] Breast [A01.236] 4. [D] Respiratory System [A04] Extremities [A01.378] 5. [E] Urogenital System [A05] Head [A01.456] 6. [F] …… Neck [A01.598] 7. [G] …. 8. Physical Sciences [H] Electronics Amplifiers 9. [I] Astronomy Electronics, Medical 10. [J] Nature Transducers 11. [K] Time 12. [L] Weights and Measures Calibration 13. [M] ….Metric System Reference Standard Faceted Metadata in Search

  15. The Interface Design • Chess metaphor • Opening • Middle game • End game Faceted Metadata in Search

  16. Faceted Metadata in Search

  17. Faceted Metadata in Search

  18. Faceted Metadata in Search

  19. Faceted Metadata in Search

  20. Faceted Metadata in Search

  21. Faceted Metadata in Search

  22. Faceted Metadata in Search

  23. Faceted Metadata in Search

  24. Faceted Metadata in Search

  25. The Interface Design • Tightly Integrated Search • Supports Expand as well as Refine • Dynamically Generated Pages • Paths can be taken in any order • Links are idempotent • Consistent Color Coding • Consistent Backup and Bookmarking • Standard HTML • No javascript Faceted Metadata in Search

  26. What is Tricky About This? • It is easy to do it poorly • Yahoo directory structure • It is hard to be not overwhelming • Most users prefer simplicity unless complexity really makes a difference • It is hard to “make it flow” • Can it feel like “browsing the shelves”? • Yes, but we iterated the design 3 times Faceted Metadata in Search

  27. Usability Study • Participants & Collection • 32 Art History Students • ~35,000 images from SF Fine Arts Museum • Study Design • Within-subjects • Each participant sees both interfaces • Balanced in terms of order and tasks • Participants assess each interface after use • Afterwards they compare them directly • Data recorded in behavior logs, server logs, paper-surveys; one or two experienced testers at each trial. • Used 9 point Likert scales. • Session took about 1.5 hours; pay was $15/hour Faceted Metadata in Search

  28. The Baseline System • Floogle • Take the best of the existing keyword-based image search systems Faceted Metadata in Search

  29. sword Faceted Metadata in Search

  30. Faceted Metadata in Search

  31. Faceted Metadata in Search

  32. Faceted Metadata in Search

  33. Hypotheses • We attempted to design tasks to test the following hypotheses: • Participants will experience greater search satisfaction, feel greater confidence in the results, produce higher recall, and encounter fewer dead ends using FC over Baseline • FC will perceived to be more useful and flexible than Baseline • Participants will feel more familiar with the contents of the collection after using FC • Participants will use FC to create multi-faceted queries Faceted Metadata in Search

  34. Four Types of Tasks • Unstructured (3): Search for images of interest • Structured Task (11-14): Gather materials for an art history essay on a given topic, e.g. • Find all woodcuts created in the US • Choose the decade with the most • Select one of the artists in this periods and show all of their woodcuts • Choose a subject depicted in these works and find another artist who treated the same subject in a different way. • Structured Task (10): compare related images • Find images by artists from 2 different countries that depict conflict between groups. • Unstructured (5): search for images of interest Faceted Metadata in Search

  35. Other Points • Participants were NOT walked through the interfaces. • The wording of Task 2 reflected the metadata; not the case for Task 3 • Within tasks, queries were not different in difficulty (t’s<1.7, p >0.05 according to post-task questions) • Flamenco is and order of magnitude slower than Floogle on average. • In task 2 users were allowed 3 more minutes in FC than in Baseline. • Time spent in tasks 2 and 3 were significantly longer in FC (about 2 min more). Faceted Metadata in Search

  36. Post-Interface Assessments All significant at p<.05 except simple and overwhelming Faceted Metadata in Search

  37. 15 16 2 30 1 29 4 28 8 23 6 24 28 3 1 31 2 29 Post-Test Comparison Baseline Which Interface Preferable For: FC Find images of roses Find all works from a given period Find pictures by 2 artists in same media Faceted Metadata in Search

  38. 15 16 2 30 1 29 4 28 8 23 6 24 28 3 1 31 2 29 Post-Test Comparison Baseline FC Which Interface Preferable For: Find images of roses Find all works from a given period Find pictures by 2 artists in same media Overall Assessment: More useful for your tasks Easiest to use Most flexible More likely to result in dead ends Helped you learn more Overall preference Faceted Metadata in Search

  39. Study Results Summary • Strongly positive results for the faceted metadata interface. • Moderate use of multiple facets. • Strong preference over the current state of the art. • Chair of Architecture Dept: “It felt like I was browsing the shelves!” • This kind of enthusiasm is not seen in similarity-based image search interfaces. • Hypotheses are supported. Faceted Metadata in Search

  40. Study Summary • Usability studies done on 3 collections: • Recipes: 13,000 items • Architecture Images: 40,000 items • Fine Arts Images: 35,000 items • Conclusions: • Users like and are successful with the dynamic faceted hierarchical metadata, especially for browsing tasks • Very positive results, in contrast with studies on earlier iterations • Note: it seems you have to care about the contents of the collection to like the interface Faceted Metadata in Search

  41. Advantages of the Approach • Supports different search types • Highly constrained known-item searches • Open-ended, browsing tasks • Can easily switch from one mode to the other midstream • Can both expand and refine • Allows different people to add content without breaking things • Can make use of standard technology Faceted Metadata in Search

  42. Metadata Availability • Many collections already have rich metadata associated with them. • Automated methods are improving. • Have applied this to: • Tobacco documents archive • MEDLINE Faceted Metadata in Search

  43. Back to the Hypothesis • This kind of tool may be helpful for resolving metadata creation wars. • Multiple paths to get to the same item • Different views on different subsets of items • No need to force everything into one hierarchy • What do you think? Faceted Metadata in Search

More Related