Assessment of value
1 / 25

Assessment of Value - PowerPoint PPT Presentation

  • Updated On :

Assessment of Value. Presented by Ben Hunter and Gretchen Scronce INLS 180: Human Information Interactions October 12, 3004. Images courtesy of Characteristics of Usefulness . Authority Quality Relevance Value. Authority.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Assessment of Value' - richard_edik

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Assessment of value l.jpg

Assessment of Value

Presented by Ben Hunter and Gretchen Scronce

INLS 180: Human Information Interactions

October 12, 3004

Images courtesy of

Characteristics of usefulness l.jpg
Characteristics of Usefulness

  • Authority

  • Quality

  • Relevance

  • Value

Authority l.jpg

  • According to Amento: “an authoritative document is one that many other documents link to” (297)

  • Not all links are equally valuable

  • A good authority is linked to by good hubs and a good hub links to good authorities

Playing the system l.jpg
Playing the system?

  • Google bombing

    • 1. Humor

    • 2. Ego

    • 3. Money

    • 4. Justice

  • Deterioration of quality

Quality l.jpg

  • Subjective

  • Amento: “The quality of a web site inherently is a matter of human judgement” (296)

Relevance l.jpg

“Relevance refers to the binary state of whether a document is on the same topic as the query or not.” (Glover, E.J., et al)

Value l.jpg

  • Based on individual human judgment

  • Subjective

  • Determined by user’s information need

  • Offering search constraints can improve likelihood of value

Does authority mean quality l.jpg
Does “Authority” Mean Quality?

  • Brian Amento, Loren Terveen and Will Hill

  • Published in 2000

Problems faced by internet users l.jpg
Problems faced by Internet users

  • Finding collections of items relevant to their interests

  • Identifying high-quality items

  • Finding items that contain a certain category of information

  • Creating and maintaining personalized subsets of items

Link based algorithms l.jpg
Link-based Algorithms

  • Do their results correlate with human standards of quality?

  • Are human standards of quality consistent? Does a shared notion of quality even exist?

  • Do the different link-based algorithms produce consistent results?

Research method l.jpg
Research Method

  • 40 subjects from a local university identified sites relevant to specific popular-culture themes

  • 16 self-identified experts rated the quality of each site

  • Various link and content-based algorithms used to evaluate sites

Results of the study l.jpg
Results of the Study

  • Generally, experts agreed with each other

  • Search engines were relatively consistent with each other

  • Experts and link-based algorithms came up with similar results

Surprises for the researchers l.jpg
Surprises for the Researchers

  • In-degree performed at least as well as more sophisticated methods

  • Simple page count was almost as good as link-analysis methods

Conclusions l.jpg

  • Topic experts make consistent quality judgments

  • Link-based metrics and a simple content metric both do a good job of identifying high quality items

Problems with study l.jpg
Problems with Study

  • Self-selecting, probably homogenous test group (self-identified expert?)

  • Are broad topics really what people typically search for?

  • Does pop-culture represent other uses of search engines?

Web search your way l.jpg
Web Search--Your Way

Eric J. Glover, Steve Lawrence, Michael D. Gordon, William P. Birmingham, and C. Lee Giles (2000)

Standard search engine l.jpg
Standard search engine

  • Searches based on user-generated keyword queries

  • Ordering policy determines which results are listed first

Metasearch engine l.jpg
Metasearch engine

  • Single interface to multiple search engines

  • Results are ordered into one list

  • Metasearches often have difficulty effectively ordering results taken from different search engines--quality is inconsistent

Inquirus 2 l.jpg
Inquirus 2

  • Source selection based on user preferences: information need categories

  • Query modification: prepends/appends

  • Ordering policy: Multi-Attribute Utility Theory

Benefits drawbacks l.jpg

  • User-controlled

  • May produce results of higher value

  • Constructing search is more involved for user

  • Designed for sophisticated searchers--confusing to average Google user?

Conclusion l.jpg

The authors believe that searching features such as those used by Inquirus 2 will enable search engines to produce results of higher value to the user.

Keep in mind l.jpg
Keep in mind…

  • Not a user study

  • Inquirus 2 is used by NEC Research Institute, a sophisticated user group

  • Authors work for NEC

  • Now at Inquirus 3--searches non-Web based sources

Discussion questions l.jpg
Discussion Questions

  • Does one characteristic of usefulness seem more important than the others?

  • Does the popularity of Google, a search engine that uses a link-based metric to assign quality, validate the success of such metrics? Do you think such systems erode over time (e.g. Google bombing) or improve?

  • How might a typical user react to a complicated search engine like Inquirus 2?

  • What impact does the continued evolution of the web have on the concepts discussed in these articles?