1 / 14

Story Link Detection using LCA & Score Normalization

Explore LCA for story smoothing and normalization of scores based on language pair for effective story link detection. Detailed experiments and conclusions.

nichelle-brown
Download Presentation

Story Link Detection using LCA & Score Normalization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. UMass at TDT 2000 James Allan and Victor Lavrenko(with David Frey and Vikas Khandelwal)Center for Intelligent Information RetrievalDepartment of Computer ScienceUniversity of Massachusetts, Amherst

  2. Work on Story Link Detection • Active work on SLD • Not ready in time for official submission • Story “smoothing” using query expansion • Score normalization based on language pair

  3. What is LCA? • Local Context Analysis • Query expansion technique from IR • More stable than other “pseudo RF” approaches • Application for more than document retrieval • Basic idea • Retrieve a set of passages similar to query • Mine those passages for words near query • Ad-hoc weighting designed to do that • Add words to query and re-run

  4. LCA for story smoothing • Convert story to a weighted vector • Inquery weights (incl. Okapi tf component) • Select top 100most highly weighted terms • Find top 20 stories most similar (cosine) • Weight all terms in top 20 stories (LCA) • Select top 100 LCA expansion terms • Add to story (decaying weights from 1.0) • Story now represented by 100-200 terms • Compare smoothed story vectors

  5. Smoothing SLD with LCA • Run on training data (english) • Green line is no smoothing • Blue is smooth with past stories • Pink is smooth with whole corpus (cheating)

  6. Work on Story Link Detection • Story “smoothing” using query expansion • Score normalization based on language pair

  7. Score normalization • Noticed that SYSTRAN documents were throwing scores off substantially • Multilingual SLD was much worse that ENG only • Look at distribution of scores in same-topic and different-topic pairs

  8. Score distributions, same topic ME EE MM

  9. Score distributions, diff topic ME MM EE

  10. Clearly need to normalize • SYSTRAN stories use different vocabulary • Stories are much more likely to be alike • And much less likely to be like true English • Develop normalization based on whether within or cross-language • Convert scores into probabilities • Use distribution plots for each case

  11. Combined distribution(before normalization) Diff. topic Same topic

  12. After normalization(on same data--”cheating”) Diff. topic Same topic Probabilities!

  13. DET plots from normalization • Huge change in distributions • Less pronounced change in DET plot

  14. Conclusions • Story smoothing with LCA works • Need to “smooth” with all stories before later • Need to use different matching for smoothing and then story-story comparison • Score normalization has potential • Other sites have found similar effects • Experiments on source-type (audio, newswire) within language pairs have been inconclusive • Not much training data for doing conversion

More Related