pairs n.
Skip this Video
Loading SlideShow in 5 Seconds..
PAIRS PowerPoint Presentation
Download Presentation

Loading in 2 Seconds...

play fullscreen
1 / 19

PAIRS - PowerPoint PPT Presentation

  • Uploaded on

PAIRS. Forming a ranked list using mined, pairwise comparisons. Reed A. Coke, David C. Anastasiu , Byron J. Gao. PAIRS. Pairwise Automatic Inferential Ranking System. d The Problem.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript


Forming a ranked list using mined, pairwise comparisons

Reed A. Coke, David C. Anastasiu, Byron J. Gao

  • Pairwise Automatic Inferential Ranking System

the problem
The Problem
  • Given a list of items, as well as an optional attribute, how best to generate a ranked list in an online system
  • What is the fastest way to get an accurate result?
  • What is the most accurate way to get a fast result?
previous approaches
Previous approaches
  • NLP techniques are likely the best
  • Can be very costly time-wise
    • Especially with nonstandard grammar of internet
  • PAIRS is an attempt at finding a balance between speed and accuracy
overall architecture
Overall Architecture
  • 1. Query Parsing (fast)
  • 2. Comparison Location (slow)
  • 3. Comparison Evaluation (fast)
  • 4. Ranking (fast)
query parsing
Query Parsing
  • Separates list into pairs:
    • i.e. (A, B, C)-> (A,B), (A,C), (B,C)
      • Leads to rapid explosion of searches
  • Each pair then is expanded into 4 queries
    • i.e. “A vs. B”, “A, B”, etc.
  • Finally, each query is sent alternatingly through Yahoo and Google, thanks to AbstractSearch2
comparison location
Comparison Location
  • Text is retrieved from each unique URL in the search results.
  • The text is then sent to a Java program which tags the part of speech of each word.
  • Line by line, the program determines whether or not the sentence is comparative.
  • Experimental results for Comparison Location
    • PAIRS keyword list: 50% recall, 80% precision
    • Ganapathibotla & Liu list: 97.7% recall, 32% precision
location continued
Location (continued)
  • A comparative sentence is one that meets the following criteria:
    • Contains a comparative word
    • Contains both nouns (stemmed) in the pair
      • Special cases:
        • Pronouns and ellipsis, keep track of “relevancy” of past nouns
        • Phrases
  • Any comparison is then evaluated immediately
special case pronouns
Special Case: Pronouns
  • Jaguars are big. They are bigger than wolves.
  • John loves computers. In fact, he loves them more than Sally.
  • John loves computers. Sally does too. However, he loves them more.
  • John likes Michael Jordan. He is a much more loyal fan than Sally.
  • John likes Michael Jordan. He dunks more impressively than Sally.
  • (on a discussion board) I respectfully disagree with you.
special case ellipsis
Special Case: Ellipsis
  • Wolves are big. However, jaguars are bigger.
  • Wolves are big. Jaguars are bigger.
  • Wolves are annoying, but don't get me started on coyotes.
  • Wolves are annoying, but turtles aren’t.
  • Wolves are annoying and turtles aren’t.
relevance dictionary
Relevance Dictionary
  • Keep track of all nouns
  • Score is affected by recency and frequency
comparison evaluation
Comparison Evaluation
  • 86% of time, people mention the noun that they prefer first.
    • i.e. n1 is better than n2, not n2 is worse than n1
    • Better methods have been found, but not quicker ones
  • Ultimately, will need a list of + and – comparisons
    • This will have to be done by domain:
      • Rocky has fought more than Drago. (+)
      • My son has fought more than your son. (-)
creating the ranking
Creating the Ranking
  • Create a graph with weight edges
  • Brute force the score of the path from each node to every other node within the connected component
  • This results in a ranked list for each component
  • Still slow
  • Query parsing needs experiments to determine just how many queries are needed per pair
  • System is untested as a whole. Must be tested on a closed set of docs to determined total precision/recall
  • Comparison evaluation could be more graceful
  • Graph traversal algorithm could be better
  • PAIRS has several interesting applications
    • College decisions
    • Product comparison
    • Any sort of popularity contest
    • Taking a majority vote
future research
Future Research
  • Polishing each component of PAIRS
  • Testing PAIRS on a closed system
  • BridgeFinder
  • PAIRS was built from the ground up. The only pre-programmed component of PAIRS was the Stanford POS tagger.
  • Things I learned about research
    • How to formulate a research topic
    • How to research previous work in a topic
    • Experimentation
    • How to write a technical report
    • How to give a presentation
  • [1] X. Ding and B. Liu. The utility of linguistic rules in opinion mining. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR '07, pages 811{812, New York, NY, USA, 2007. ACM.
  • [2] X. Ding, B. Liu, and P. S. Yu. A holistic lexicon-based approach to opinion mining. In Proceedings of the international conference on Web search and web data mining, WSDM '08, pages 231{240, New York, NY, USA, 2008. ACM.
  • [3] M. Ganapathibhotla and B. Liu. Mining opinions in comparative sentences. In Proceedings of the 22nd International Conference on Computational Linguistics - Volume 1, COLING '08, pages 241{248, Stroudsburg, PA, USA, 2008. Association for Computational Linguistics.
  • [4] A. Go, R. Bhayani, and L. Huang. Twitter sentiment classification using distant supervision. Technical report, Stanford University, 2010.
  • [5] M. Hu and B. Liu. Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, KDD '04, pages 168{177, New York, NY, USA, 2004. ACM.
  • [6] N. Jindal and B. Liu. Identifying comparative sentences in text documents. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR '06, pages 244{251, New York, NY, USA, 2006. ACM.
  • [7] N. Jindal and B. Liu. Mining comparative sentences and relations. In AAAI'06, pages {1{1, 2006.
  • [8] B. Liu. Web Data Mining. Springer, 2008.
  • [9] K. Toutanova, D. Klein, C. D. Manning, and Y. Singer. Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - Volume 1, NAACL '03, pages 173{180, Stroudsburg, PA, USA, 2003. Association for Computational Linguistics. 10
  • [10] K. Toutanova and C. D. Manning. Enrichingthe knowledge sources used in a maximum entropy part-of-speech tagger. In Proceedings of the 2000 Joint SIGDAT conference on Empirical methods in natural language processing and very large corpora: held in conjunction with the 38th Annual Meeting of the Association for Computational Linguistics Volume 13, EMNLP '00, pages 63{70, Stroudsburg, PA, USA, 2000. Associationfor Computational Linguistics.