Inference Protocols for Coreference Resolution

1. Inference Protocols for Coreference Resolution Kai-Wei Chang, Rajhans Samdani, Alla Rozovskaya, Nick Rizzolo, Mark Sammons, and Dan Roth Experiments and Results Pairwise Mention Score Coreference Inference Procedure Compatibility score given by constraints Weight vector learned from training data Coreference Resolution is the task of grouping all the mentions of entities into equivalence classes so that each class represents a discourse entity. In the example below, the mentions are colour-coded to indicate which mentions are co-referent (overlapping mentions have been omitted for clarity). An Americanofficial announced that American President Bill Clinton met hisRussiancounterpart, Vladimir Putin, today. The president said that Russiawas a great country. The inference procedure takes as input a set of pairwise mention scores over a document and aggregates them into globally consistent cliques representing entities. We investigate two techniques: Best-Link and All-Link. Best-Link Inference: For each mention, Best-Link considers the best mention on its left to connect to (according the pairwise score) and creates a link between them if the pairwise score is above some threshold. All-Link Inference: The All-Link inference approach scores a clustering of mentions by including all possible pairwise links in the score. It is also known as correlational clustering (Bansal et al., 2002). ILP Formulation: Both Best-Link and All-Link can be written as an Integer linear programming (ILP) problem: Best-Link: All-Link: wuv is the compatibility score of a pair of mentions, yuv is a binary variable. For the All-Link clustering, we drop one of the three transitivity constraints for each triple of mention variables. Similar to Pascal and Baldridge (2009), we observe that this improves accuracy. • Results on DEV set with predicted mentions • Results on DEV set with gold mentions • Official scores on TEST set: Extracted features A threshold parameter (to be tuned) We use the same features as Bengtson and Roth (2008) with the knowledge extracted from OntoNotes-4.0. Training Procedure • We explored two types of learning strategies, which can be used to learn w in Best-Link and All-Link. The choice of a learning strategy depends on the inference procedure. • Binary Classification: • Following Bengtson and Roth (2008), we learn the pairwise scoring function w on: • Positive examples: for each mention u, we construct a positive example (u, v), where v is the closest preceding mention in u’s equivalence class. • Negative examples: all mention pairs (u, v), where v is a preceding mention of u and u, v are in different classes. • As singleton mentions are not annotated, the sample distributions in the training and inference phases are inconsistent. We apply the mention detector to the training set, and train the classifier using the union set of gold and prediction mentions. • Structured Learning (Structured Perceptron): • We present a structured perceptron algorithm, which is similar to the supervised clustering algorithm of Finley and Joachims (2005) to learn w: CoNLL Shared Task 2011 • Coreference resolution on the OntoNotes-4.0 data set. • Based on Bengtson and Roth (2008), our system is built on Learning Based Java (Rizzolo and Roth, 2010) • We participated in the “closed” track of the shared task. Compared to the ACE 2004 Corpus, the OntoNotes-4.0 data set has two main differences: • Singleton mentions are not annotated in OntoNotes-4.0. • OntoNotes-4.0 takes the largest logical span to represent a mention. Contributions • We investigate two inference methods: Best-Link & All-Link • We provide a flexible architecture for incorporating constraints • We compare and evaluate the two inference approaches and the contribution of constraints System Architecture Discussion and Conclusions • Best-Link outperforms All-Link. This raises a natural algorithmic question regarding the inherent nature of the clustering style most suitable for coreference resolution and regarding possible ways of infusing more knowledge into different coreference clustering styles. • Constraints improve the recall on a subset of mentions. There are other common errors for the system that might be fixed by constraints. • Our approach accommodates infusion of know-ledge via constraints. We have demonstrated its utility in an end-to-end coreference system. Knowledge-based Constraints We define three high precision constraints that improve recall on NPs with definite determiners and mentions whose heads are Named Entities. Examples of mention pairs that are correctly linked by the constraints are: [Governor Bush] and [Bush],[a crucial swing state, Florida] and [Florida], [Sony itself] and [Sony]. Mention Detection • We design a high recall (~90%) and low precision (~35%) rule-based mention detection system. • As a post-processing step, we remove all predicted mentions that remain in singleton clusters after the inference stage. • The system achieves 64.88% in F1 score on TEST set. This research is supported by ARL, and by DARPA, under the Machine Reading Program