1 / 55

Relevance Ranking in the Scholarly Domain

Relevance Ranking in the Scholarly Domain. Dr. Tamar Sadeh LIBER Conference Tartu, Estonia, June 2012. The top three keys for success. 1. 2. 3. Content. 1. Speed. 2. Relevance Ranking. 3.

odetta
Download Presentation

Relevance Ranking in the Scholarly Domain

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Relevance Ranking in the Scholarly Domain • Dr. Tamar Sadeh • LIBER Conference • Tartu, Estonia, June 2012

  2. The top three keys for success 1 2 3

  3. Content 1

  4. Speed 2

  5. Relevance Ranking 3

  6. Relevance is the measure of correspondence between a document and a query as determined by a user Based on Saracevic, 1975

  7. ?

  8. System or algorithmic relevance • Topical or subject relevance • Cognitive relevance or pertinence • Situational relevance or utility • Effective relevance • System or algorithmic relevance • Topical or subject relevance • Cognitive relevance or pertinence • Situational relevance or utility • Affective relevance

  9. There is no absolute relevance

  10. The ScholarRank Project

  11. The Goal Enhance the Primo relevance ranking algorithm

  12. Relevance ranking was not new to us.

  13. Methodology

  14. Setting up a team • Building test environment, tools, and procedures • Defining metrics to evaluate our current success and the improvements we make • Defining measurements to assess the success of the changes, once implemented

  15. Evaluation

  16. Working with researchers • Researchers’ evaluation quantified • Enhancements introduced and checked in the lab, using defined metrics • Enhancements launched and usage patterns monitored • Improvements are introduced on an ongoing basis

  17. How is relevance ranking calculated?

  18. ?

  19. abstractauthordatefull text journallanguagetypepublishersubjecttitle citationsdownloadsjournal impactfactoreigenfactorpagerank

  20. academic degree discipline(s)languagelocationprevious selections search history

  21. ? broad-topic search currency exact-item search material type narrow-topic search

  22. Narrow-topic query Known-item query Broad-topic query Author-related query

  23. ?

  24. ?

  25. The match: traditional information retrieval methods, adapted to the scholarly environment

  26. 1 100 20

  27. ? no. of citations; no. of selections; recency; type; peer review

  28. 1 20 100

  29. Academic degree, discipline ?

  30. Computer Science

  31. Psychology

  32. Mathematics

  33. Physics

  34. Author-related query, known-item query, broad-topic query… ?

  35. Before

More Related