1 / 14

Feature Selection for Ranking

Feature Selection for Ranking. Xiubo Geng , Tie-Yan Liu, Tao Qin, Hang Li ---- SIGIR 2007. What’s Ranking?. Ranking: a classical problem in information retrieval S upervised ranking Ranking SVM RankNet. Feature Selection. Well studied in classification Enhance accuracy

silvio
Download Presentation

Feature Selection for Ranking

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Feature Selection for Ranking XiuboGeng, Tie-Yan Liu, Tao Qin, Hang Li ---- SIGIR 2007

  2. What’s Ranking? • Ranking: a classical problem in information retrieval • Supervised ranking • Ranking SVM • RankNet

  3. Feature Selection • Well studied in classification • Enhance accuracy • Improve efficiency • Avoid over-fitting

  4. What’s new for Ranking? • In ranking, a number of ordered categories are used; in classification, the categories are “flat”. • Different evaluation measures: • Ranking focuses more on precision than recall; Classification cares both precision & recall. • Top-n instances are more critical for ranking; whereas instances are weighted equivalently in classification.

  5. What’s new for Ranking? • Mean average precision(MAP) • Suppose you submit a query, and it has 4 relevant docs. • Retrieve Ranking: Y, N, N, Y • MAP = 1x1/4+0.5x0/2+0.3x0/2+0.5x1/4 • =0.375

  6. MAP (A Toy Example) • A query has 5 relevant documents, • Retrieve ranking: Y, Y, N, N • Answer?

  7. Normalized discount cumulative gain (NDCG) • If have multiple level of relevance, then use NDCG • R(j) denotes score for rank j and Zn is a normalization factor to guarantee that a perfect ranking’s NDCG at position n equals 1.

  8. Overview of Algorithm • Goal: select t features from the entire feature set. • Procedure • Define the importance score of each feature • Define the similarity between any two features. • Greedy method to maximize the total importance whereas minimize the total similarity.

  9. Importance & Similarity • Importance: • Rank instances using one feature and using the evaluation measure MAP or NDCG as importance score. • Similarity between features: • Use ranking distance like Kendell’stao measure.

  10. An optimization formula Wi is the importance score, ei,j is similarity between two features. Integer programming, typically not easy to solve. Thus, adopt an greedy algorithm.

  11. Greedy Approach • Construct a undirected graph with each node represent one feature. The node weight is importance score and edges between nodes are the similarity. • For each feature, • Select the node (VKi)with maximum weight • A punishment is conducted to all the other nodes according to their similarities. • Add Vki to the selected features and remove the node and corresponding edges from the graph.

  12. Data Sets • .gov dataset of Web TREC 2004. • 1,053,110 docs, only a subset is used. • 75 queries with binary relevance. • 44 features for each doc. • OHSUMED data • a bibliographical document collection, a subset of MEDLINE database. • 16,140 query-document pairs • three levels of relevance: definitely/possibly/not relevant • 26 features.

  13. Experimental Results(1)

  14. Experimental Results(2)

More Related