Kbp2014 entity linking scorer
Download
1 / 18

KBP2014 Entity Linking Scorer - PowerPoint PPT Presentation


  • 127 Views
  • Uploaded on

KBP2014 Entity Linking Scorer. Xiaoman Pan, Qi Li and Heng Ji {panx2,liq7,jih}rpi.edu. Overview. We apply two steps to evaluate the KBP2014 Entity Discovery and Linking results. 1. Existing Entity Linking ( Wikification ) F-score 2. NIL Clustering score. Overview.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' KBP2014 Entity Linking Scorer' - lydia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Kbp2014 entity linking scorer

KBP2014 Entity Linking Scorer

Xiaoman Pan, Qi Li and HengJi

{panx2,liq7,jih}rpi.edu


Overview
Overview

  • We applytwostepstoevaluatetheKBP2014EntityDiscoveryandLinkingresults.

    1.Existing Entity Linking (Wikification)F-score

    2.NILClusteringscore


Overview1
Overview

  • We use a tuple ⟨doc-id, start, end, entity-type, kb-id⟩to represent each entity mention, where a special type of kb-id is NIL.

  • Let s be an entity mention in the system output, g be an corresponding gold-standard. An output mention s matches a reference mention giff:

    1. s.doc-id = g.doc-id, 2. s.start = g.start, s.end = g.end, 3. s.entity-type = g.entity-type, 4. and s.kb-id = g.kb-id.


Nil clustering score
NIL Clustering score

  • We will apply three metrics to evaluate the NIL clustering score

    • B-Cubed metric

    • CEAF metric

    • Graph Edit Distance (G-Edit)




CEAF

  • CEAF (Luo 2005)

  • Idea: a mention or entity should not be credited more than once

  • Formulated as a bipartite matching problem

    A special ILP problem

    efficient algorithm: Kuhn-Munkres


Ceaf luo 2005
CEAF (Luo, 2005)


G edit
G-Edit

  • First step, evaluate the name mention tagging F-measure

  • Second step, evaluate the overlapped mentions by using graph-based edit distnace


ad