1 / 14

Modeling Discriminative Global Inference

Modeling Discriminative Global Inference. Nick Rizzolo and Dan Roth University of Illinois at Urbana-Champaign ICSC ‘07. Motivation. Interesting systems are increasingly complex. Single classifier systems Document categorizer Sentence boundary detector “Local” classifiers + constraints

aliza
Download Presentation

Modeling Discriminative Global Inference

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Modeling Discriminative Global Inference Nick Rizzolo and Dan Roth University of Illinois at Urbana-Champaign ICSC ‘07

  2. Motivation Interesting systems are increasingly complex • Single classifier systems • Document categorizer • Sentence boundary detector • “Local” classifiers + constraints • Chunking • Named Entity Recognition • Semantic Role Labeling • Coreference

  3. Motivation Interesting systems are increasingly complex • Each new system, a massive programming effort • Yet they all solve the same problem: Given related objects… Find the “best” legal labeling • Feature extraction • Learning • Inference

  4. Learning Based Java Eases the programmer’s burden • Organized, integrated feature extraction • Learning operator abstracts training • Convenient constraint programming language

  5. Outline • Motivation • A Model for Discriminative Inference • A Detailed Example • Formalization • Learning Based Java • Support for Integer Linear Programming

  6. Training data wi-1 wi+1 wi wi-1 wi wi+1 wi-1 wi wi+1 wi-1 wi+1 wi wi-1 wi+1 wi wi-1 wi wi+1 Learning wi-1 wi+1 wi open(wi) {“yes”, “no”} w1 w2 w3 w4 w5 w6 w1 w2 w3 w4 w5 w6 w1 w2 w3 w4 w5 w6 Constraints Example: Chunking Inference POS tags Forms Feature extraction

  7. Objective: maximize scoring function object; needs a label possible label inference variable A Discriminative Inference Model • Constrained optimization • We use FOL to represent constraints • Inference algorithms: ILP, Beam search, … • Learning operator

  8. Example: Chunking Modular feature extraction Learn to mimic an oracle From data Using particular features Using particular features With an algorithm Application of the learned function Application of the learned function Declarative, FOL-style syntax

  9. Example: Part of Speech Tagging • 97.4% accuracy on Penn Treebank WSJ • Closed world: baseline feature computed from all data Resulting function returns a conjunctive value LBJ source file 54 total lines LBJ functions become classes; simply instantiate LBJ functions become classes Simply instantiate … … and invoke Java program Effortless integration

  10. Outline • Motivation • A Model for Discriminative Inference • A Detailed Example • Formalization • Learning Based Java • Support for Integer Linear Programming

  11. Integer Linear Programming • Expressive • Linear inequality constraints are hard to design

  12. Final Thoughts • Designers of intelligent systems think about: • What classifiers to learn • How to combine them • LBJ enables programming at this level • Download LBJ from: http://l2r.cs.uiuc.edu/~cogcomp • Future work • More explicit modeling of structure • Applications in NLP and elsewhere

  13. Thank You!

  14. Example: Semantic Role Labeling Declarative, FOL-style constraints written in terms of functions applied to Java objects Inference produces new functions that respect the constraints

More Related