Treebank based wide coverage probabilistic lfg resources
Download
1 / 92

Treebank-Based Wide Coverage Probabilistic LFG Resources - PowerPoint PPT Presentation


  • 308 Views
  • Uploaded on

Treebank-Based Wide Coverage Probabilistic LFG Resources. Josef van Genabith, Aoife Cahill, Grzegorz Chrupala, Jennifer Foster, Deirdre Hogan, Conor Cafferkey, Mick Burke, Ruth O’Donovan, Yvette Graham, Karolina Owczarzak, Yuqing Guo, Ines Rehbein, Natalie Schluter and Djame Sedah

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Treebank-Based Wide Coverage Probabilistic LFG Resources' - arleen


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Treebank based wide coverage probabilistic lfg resources l.jpg

Treebank-Based Wide Coverage Probabilistic LFG Resources

Josef van Genabith, Aoife Cahill, Grzegorz Chrupala, Jennifer Foster, Deirdre Hogan, Conor Cafferkey, Mick Burke, Ruth O’Donovan, Yvette Graham, Karolina Owczarzak, Yuqing Guo, Ines Rehbein, Natalie Schluter and Djame Sedah

National Centre for Language Technology NCLT

School of Computing, Dublin City University

Treebank-Based LFG Resources


Overview l.jpg
Overview

  • Context/Motivation

  • Treebank-Based Acquisition of Wide-Coverage LFG Resources (Penn-II)

    • LFG

    • Automatic F-Structure Annotation Algorithm

    • Acquisition of Lexical Resources

  • Parsing

    • Parsing Architectures

    • LDD-Resolution

    • Comparison with Hand-Crafted (XLE, RASP) and Treebank-Based (CCG, HPSG) Resources

  • Generation

    • Basic Generator

    • Generation Grammar Transforms

    • “History-Based” Generation

  • MT Evaluation

Treebank-Based LFG Resources


Motivation l.jpg
Motivation

  • What do grammars do?

    • Grammars define languages as sets of strings

    • Grammars define what strings are grammatical and what strings are not

    • Grammars tell us about the syntactic structure of (associated with) strings

  • “Shallow” vs. “Deep” grammars

  • Shallow grammars do all of the above

  • Deep grammars (in addition) relate text to information/meaning representation

  • Information: predicate-argument-adjunct structure, deep dependency relations, logical forms, …

  • In natural languages, linguistic material is not always interpreted locally where you encounter it: long-distance dependencies (LDDs)

  • Resolution of LDDs crucial to construct accurate and complete information/meaning representations.

  • Deep grammars := (text <-> meaning) + (LDD resolution)

Treebank-Based LFG Resources


Motivation4 l.jpg
Motivation

  • Constraint-Based Grammar Formalisms (FU, GPSG, PATR-II, …)

    • Lexical-Functional Grammar (LFG)

    • Head-Driven Phrase Structure Grammar (HPSG)

    • Combinatory Categorial Grammar (CCG)

    • Tree-Adjoining Grammar (TAG)

  • Traditionally, deep constraint-based grammars are hand-crafted

  • LFG ParGram, HPSG LingoErg, Core Language Engine CLE, Alvey Tools, RASP, ALPINO, …

  • Wide-coverage, deep constraint-based grammar development is very time consuming, knowledge extensive and expensive!

  • Very hard to scale hand-crafted grammars to unrestricted text!

  • English XLE (Riezler et al. 2002); German XLE (Forst and Rohrer 2006); Japanese XLE (Masuichi and Okuma 2003); RASP (Carroll and Briscoe 2002); ALPINO (Bouma, van Noord and Malouf, 2000)

Treebank-Based LFG Resources


Motivation5 l.jpg
Motivation

  • Instance of “knowledge acquisition bottleneck” familiar from classical “rationalist rule/knowledge-based” AI/NLP

  • Alternative to classical “rationalist” rule/knowledge-based AI/NLP

  • “Empiricistdata-driven ” research paradigm (AI/NLP):

    • Corpora, …, machine-learning-based and statistical approaches, …

    • Treebank-based grammar acquisition, probabilistic parsing

    • Advantage: grammars can be induced (learned) automatically

    • Very low development cost, wide-coverage, robust, but …

  • Most treebank-based grammar induction/parsing technology produces “shallow” grammars

  • Shallow grammars don’t resolve LDDs (but see (Johnson 2002); …), do not map strings to information/meaning representations …

Treebank-Based LFG Resources


Motivation6 l.jpg
Motivation

  • Poses a number of research questions:

  • Can we address the knowledge acquisition bottleneck for deep grammar development by combining insights from rationalist and empiricist research paradigms?

  • Specifically:

  • Can we automatically acquire wide-coverage “deep”, probabilistic, constraint-based grammars from treebanks?

  • How do we use them in parsing?

  • Can we use them for generation?

  • Can we acquire resources for different languages and treebank encodings?

  • How do these resources compare with hand-crafted resources?

  • How do they fare in applications … ?

Treebank-Based LFG Resources


Context l.jpg
Context

  • TAG (Xia, 2001)

  • LFG (Cahill, McCarthy, van Genabith and Way, 2002)

  • CCG (Hockenmaier & Steedman, 2002)

  • HPSG (Miyao and Tsujii, 2003)

  • LFG

  • (van Genabith, Sadler and Way, 1999)

  • (Frank, 2000)

  • (Sadler, van Genabith and Way, 2000)

  • (Frank, Sadler, van Genabith and Way, 2003)

Treebank-Based LFG Resources


Lexical functional grammar lfg l.jpg
Lexical-Functional Grammar (LFG)

Parsing

Treebank-Based LFG Resources


Lfg acquisition for english overview l.jpg
LFG Acquisition for English - Overview

  • Treebank-Based Acquisition of LFG Resources (Penn-II)

    • Lexical Functional Grammar LFG

    • Penn-II Treebank & Preprocessing/Clean-Up

    • F-Str Annotation Algorithm

    • Grammar and Lexicon Extraction

  • Parsing Architectures (LDD Resolution)

  • Comparison with best hand-crafted resources: XLE and RASP

  • Comparison with treebank-based CCG and HPSG resources

Treebank-Based LFG Resources


Lexical functional grammar lfg10 l.jpg
Lexical-Functional Grammar (LFG)

Lexical-Functional Grammar (LFG) (Bresnan & Kaplan 1981, Bresnan 2001, Dalrymple 2001) is a constraint-based theory of grammar.

Two (basic) levels of representation:

  • C-structure: represents surface grammatical configurations such as word order, annotated CFG rules/trees

  • F-structure: represents abstract syntactic functions such as SUBJ(ject), OBJ(ect), OBL(ique), PRED(icate), COMP(lement), ADJ(unct) …, AVM attribute-value matrices/feature structures

    F-structure approximates to basic predicate-argument structure, dependency representation, logical form (van Genabith and Crouch, 1996; 1997)

Treebank-Based LFG Resources


Lexical functional grammar lfg11 l.jpg
Lexical-Functional Grammar (LFG)

Treebank-Based LFG Resources


Lexical functional grammar lfg12 l.jpg
Lexical-Functional Grammar (LFG)

  • Subcategorisation:

    • Semantic forms (subcat frames): see<SUBJ,OBJ>

    • Completeness: all GFs in semantic form present at local f-structure

    • Coherence: only the GFs in semantic form present at local f-structure

  • Long Distance Dependencies (LDDs): resolved at f-structure with

    • Functional Uncertainty Equations (regular expressions specifying paths in f-structure): e.g. TOPICREL = COMP* OBJ

    • subcat frames

    • Completeness/Coherence.

Treebank-Based LFG Resources


Lexical functional grammar lfg13 l.jpg
Lexical-Functional Grammar (LFG)

Treebank-Based LFG Resources


Introduction penn ii lfg l.jpg
Introduction: Penn-II & LFG

  • If we had f-structure annotated version of Penn-II, we could use (standard) machine learning methods to extract probabilistic, wide-coverage LFG resources

  • How do we get f-structure annotated Penn-II?

  • Manually? No: ~50,000 trees …!

  • Automatically! Yes: F-Structure annotation algorithm… !

  • Penn-II is a 2nd generation treebank– contains lots of annotations to support derivation of deep meaning representations:

    • trees, Penn-II “functional” tags (-SBJ, -TMP, -LOC), traces & coindexation

  • f-structure annotation algorithm exploits those.

Treebank-Based LFG Resources


Treebank annotation penn ii lfg l.jpg
Treebank Annotation: Penn-II & LFG

Treebank-Based LFG Resources


Treebank annotation penn ii lfg16 l.jpg
Treebank Annotation: Penn-II & LFG

Treebank-Based LFG Resources


Treebank preprocessing clean up penn ii lfg l.jpg
Treebank Preprocessing/Clean-Up: Penn-II & LFG

  • Penn-II treebank: often flat analyses (coordination, NPs …), a certain amount of noise: inconsistent annotations, errors …

  • No treebank preprocessing or clean-up in the LFG approach (unlike CCG- and HPSG-based approaches)

    • Take Penn-II treebank as is, but

    • Remove all trees with FRAG or X labelled constituents

    • Frag = fragments, X = not known how to annotate

  • Total of 48,424 trees as they are.

Treebank-Based LFG Resources


Treebank annotation penn ii lfg18 l.jpg
Treebank Annotation: Penn-II & LFG

  • Annotation-based (rather than conversion-based)

  • Automatic annotation of nodes in Penn-II treebank trees with f-structure equations

  • Annotation Algorithm exploits:

    • Head information

    • Categorial information

    • Configurational information

    • Penn-II functional tags

    • Trace information

Treebank-Based LFG Resources


Treebank annotation penn ii lfg19 l.jpg
Treebank Annotation: Penn-II & LFG

Architecture of a modular algorithm to assign LFG f-structure equations to trees in the Penn-II treebank:

Head-Lexicalisation [Magerman,1994]

Left-Right Context Annotation Principles

Proto

F-Structures

Coordination Annotation Principles

Proper

F-Structures

Catch-All and Clean-Up

Traces

Treebank-Based LFG Resources


Treebank annotation penn ii lfg20 l.jpg
Treebank Annotation: Penn-II & LFG

  • Head Lexicalisation: modified rules based on (Magerman, 1994)

Treebank-Based LFG Resources


Treebank annotation penn ii lfg21 l.jpg
Treebank Annotation: Penn-II & LFG

Left

Context

Head

Right

Context

Left-Right Context Annotation Principles:

  • Head of NP likely to be rightmost noun …

  • Mother →Left Context Head Right Context

Treebank-Based LFG Resources


Treebank annotation penn ii lfg22 l.jpg
Treebank Annotation: Penn-II & LFG

Left-Right Annotation Matrix

NP:

NP

NP

DT

ADJP

NN

↑=↓

NN

↑spec:det=↓

DT

↓↑adjunct

ADJP

a

RB

JJ

deal

a

RB

JJ

deal

very politicized

very politicized

Treebank-Based LFG Resources


Treebank annotation penn ii lfg23 l.jpg
Treebank Annotation: Penn-II & LFG

Treebank-Based LFG Resources


Treebank annotation penn ii lfg24 l.jpg
Treebank Annotation: Penn-II & LFG

  • Do annotation matrix for each of the monadic categories (without –Fun tags) in Penn-II

  • Based on analysing the most frequent rule types for each categorysuch that

    • sum total of token frequencies of these rule types is greater than 85% of total number of rule tokens for that category

      100% 85% 100% 85%

      • NP 6595 102 VP 10239 307

      • S 2602 20 ADVP 234 6

  • Apply annotation matrix to all (i.e. also unseen) rules/sub-trees, i.e. also those NP-LOC, NP-TMP etc.

Treebank-Based LFG Resources


Treebank annotation penn ii lfg25 l.jpg
Treebank Annotation: Penn-II & LFG

  • Traces Module:

  • Long Distance Dependencies (LDDs)

    • Topicalisation

    • Questions

    • Wh- and wh-less relative clauses

    • Passivisation

    • Control constructions

    • ICH (interpret constituent here)

    • RNR (right node raising)

  • Translate Penn-II traces and coindexation into corresponding reentrancy in f-structure

Treebank-Based LFG Resources


Treebank annotation control wh rel ldd l.jpg
Treebank Annotation: Control & Wh-Rel. LDD

Treebank-Based LFG Resources


Treebank annotation penn ii lfg27 l.jpg
Treebank Annotation: Penn-II & LFG

Head-Lexicalisation [Magerman,1995]

Left-Right Context Annotation Principles

Proto

F-Structures

Coordination Annotation Principles

Proper

F-Structures

Catch-All and Clean-Up

Traces

Constraint Solver

Treebank-Based LFG Resources


Treebank annotation penn ii lfg28 l.jpg
Treebank Annotation: Penn-II & LFG

  • Collect f-structure equations

  • Send to constraint solver

  • Generates f-structures

  • F-structure annotation algorithm in Java, constraint solver in Prolog

  • ~3 min annotating ~50,000 Penn-II trees

  • ~5 min producing ~50,000 f-structures

Treebank-Based LFG Resources


Treebank annotation penn ii lfg29 l.jpg
Treebank Annotation: Penn-II & LFG

Evaluation (Quantitative):

  • Coverage:

    Over 99.8% of Penn-II sentences (without X and FRAG constituents) receive a single covering and connected f-structure:

Treebank-Based LFG Resources


Treebank annotation penn ii lfg30 l.jpg
Treebank Annotation: Penn-II & LFG

  • F-structure quality evaluation against DCU 105Dependency Bank, a manually annotated dependency gold standard of 105 sentences randomly extracted from WSJ section 23.

  • Triples are extracted from the gold standard

  • Evaluation software from (Crouch et al. 2002) and (Riezler et al. 2002)

    relation(predicate~0, argument~1)

Treebank-Based LFG Resources


Treebank annotation penn ii lfg31 l.jpg
Treebank Annotation: Penn-II & LFG

  • Following (Kaplan et al. 2004) evaluation against PARC 700 Dependency Bank calculated for:

    all annotations  PARC features preds-only

  • Mapping required (Burke 2004, 2006)

Treebank-Based LFG Resources


Grammar and lexicon extraction penn ii lfg l.jpg
Grammar and Lexicon Extraction : Penn-II & LFG

Lexical Resources:

  • Lexical information extremely important in modern lexicalised grammar formalisms

  • LFG, HPSG, CCG, TAG, …

  • Lexicon development is time consuming and extremely expensive

  • Rarely if ever complete

  • Familiar knowledge acquisition bottleneck …

  • Treebank-based subcategorisation frame induction (LFG semantic forms) from Penn-II and –III

  • Parser-based induction from British National Corpus (BNC)

  • Evaluation against COMLEX, OALD, Korhonen’s data set

Treebank-Based LFG Resources


Grammar and lexicon extraction penn ii lfg33 l.jpg
Grammar and Lexicon Extraction: Penn-II & LFG

  • Lexicon Construction

    • Manual vs. Automated

  • Our Approach:

  • Subcat Frames not Predefined

  • Functional and/or Categorial Information

  • Parameterised for Prepositions and Particles

  • Active and Passive

  • Long Distance Dependencies

  • Conditional Probabilities

Treebank-Based LFG Resources


Grammar and lexicon extraction penn ii lfg34 l.jpg
Grammar and Lexicon Extraction: Penn-II & LFG

Treebank-Based LFG Resources


Grammar and lexicon extraction penn ii lfg35 l.jpg
Grammar and Lexicon Extraction: Penn-II & LFG

apply<SUBJ,OBL:for>

win<SUBJ,OBJ>

Treebank-Based LFG Resources


Grammar and lexicon extraction penn ii lfg36 l.jpg
Grammar and Lexicon Extraction: Penn-II & LFG

Lexicon extracted from Penn-II (O’Donovan et al 2005):

Treebank-Based LFG Resources


Grammar and lexicon extraction penn ii lfg37 l.jpg
Grammar and Lexicon Extraction: Penn-II & LFG

Treebank-Based LFG Resources


Grammar and lexicon extraction penn ii lfg38 l.jpg
Grammar and Lexicon Extraction: Penn-II & LFG

Parsing-Based Subcat Frame Extraction (O’Donovan 2006):

  • Treebank-based vs. parsing-based subcat frame extraction

  • Parsed British National Corpus BNC (100 million words) with our automatically induced LFGs

  • 19 days on single machine: ~5 million words per day

  • Subcat frame extraction for ~10,000 verb lemmas

  • Evaluation against COMLEX and OALD

  • Evaluation against Korhonen (2002) gold standard

  • Our method is statistically significantly better than Korhonen (2002)

Treebank-Based LFG Resources


Parsing penn ii and lfg l.jpg
Parsing: Penn-II and LFG

  • Overview Parsing Architectures:

    Pipeline & Integrated

  • Long-Distance Dependency (LDD) Resolution at F-Structure

  • Evaluation & Comparison with Hand-Crafted Resources (XLE and RASP)

  • Comparison against Treebank-Based CCG and HPSG Resources

Treebank-Based LFG Resources


Parsing penn ii and lfg40 l.jpg
Parsing: Penn-II and LFG

Treebank-Based LFG Resources


Lexical functional grammar lfg41 l.jpg
Lexical-Functional Grammar (LFG)

Treebank-Based LFG Resources


Parsing penn ii and lfg42 l.jpg
Parsing: Penn-II and LFG

  • Require:

    • subcategorisation frames (O’Donovan et al., 2004, 2005; O’Donovan 2006)

    • functional uncertainty equations

  • Previous Example:

    • claim([subj,comp]), deny([subj,obj])

    •  topicrel =  comp* obj (search along a path of 0 or more comps)

Treebank-Based LFG Resources


Parsing penn ii and lfg43 l.jpg
Parsing: Penn-II and LFG

Subcat frames: as above (O’Donovan et al. 2004, 2005)

Functional Uncertainty equations:

  • Automatically acquire finite approximations of FU-equations

  • Extract paths between co-indexed material in automatically generated f-structures from sections 02-21 from Penn-II

  • 26 TOPIC, 60 TOPICREL, 13 FOCUS path types

  • 99.69% coverage of paths in WSJ Section 23

  • Each path type associated with a probability

    LDD resolution ranked by Path x Subcat probabilities (Cahill et al., 2004)

Treebank-Based LFG Resources


Parsing penn ii and lfg44 l.jpg
Parsing: Penn-II and LFG

  • How do treebank-based constraint grammars compare to deep hand-crafted grammars like XLE and RASP?

  • XLE (Riezler et al. 2002, Kaplan et al. 2004)

    • hand-crafted, wide-coverage, deep, state-of-the-art English LFG and XLE parsing system with log-linear-based probability models for disambiguation

    • PARC 700 Dependency Bank gold standard (King et al. 2003), Penn-II Section 23-based

  • RASP (Carroll and Briscoe 2002)

    • hand-crafted, wide-coverage, deep, state-of-the-art English probabilistic unification grammar and parsing system (RASP Rapid Accurate Statistical Parsing)

    • CBS 500 Dependency Bank gold standard (Carroll, Briscoe and Sanfillippo 1999), Susanne-based

Treebank-Based LFG Resources


Parsing penn ii and lfg45 l.jpg
Parsing: Penn-II and LFG

  • (Bikel 2002) retrained to retain Penn-II functional tags (-SBJ, -SBJ, -LOC,-TMP, -CLR, -LGS, etc.)

  • Pipeline architecture:

  • tag textBikel retrained + f-structure annotation algorithm + LDD resolution f-structures  automatic conversion  evaluation against XLE/RASP gold standards PARC-700/CBS-500 Dependency Banks

Treebank-Based LFG Resources


Parsing penn ii and lfg46 l.jpg
Parsing: Penn-II and LFG

  • Systematic differences between f-structures and PARC 700 and CBS 500 dependency representations

  • Automatic conversion of f-structures to PARC 700 / CBS 500 -like structures (Burke et al. 2004, Burke 2006, Cahill et al. 2008)

  • Evaluation software (Crouch et al. 2002) and (Carroll and Briscoe 2002)

  • Approximate Randomisation Test (Noreen 1989) for statistical significance

Treebank-Based LFG Resources


Parsing penn ii and lfg47 l.jpg
Parsing: Penn-II and LFG

  • Result dependency f-scores (CL 2008 paper):

    PARC 700 XLE vs. DCU-LFG

    • 80.55% XLE

    • 82.73% DCU-LFG (+2.18%)

      CBS 500 RASP vs. DCU-LFG

    • 76.57% RASP

    • 80.23% DCU-LFG (+3.66%)

  • Results statistically significant at  95% level (Noreen 1989)

  • Best result now against PARC 700 84.00% (+3.45%) Charniak + Reranker + Grzegorz’ Penn-II function-tag labeler

  • Treebank-Based LFG Resources


    Parsing penn ii and lfg48 l.jpg
    Parsing: Penn-II and LFG

    PARC 700 Evaluation:

    Treebank-Based LFG Resources


    Parsing penn ii and lfg49 l.jpg
    Parsing: Penn-II and LFG

    Treebank-Based LFG Resources


    Parsing penn ii and lfg50 l.jpg
    Parsing: Penn-II and LFG

    Treebank-Based LFG Resources


    Parsing penn ii and lfg51 l.jpg
    Parsing: Penn-II and LFG

    Treebank-Based LFG Resources


    Parsing penn ii and lfg52 l.jpg
    Parsing: Penn-II and LFG

    Treebank-Based LFG Resources


    Parsing penn ii and lfg53 l.jpg
    Parsing: Penn-II and LFG

    Treebank-Based LFG Resources


    Evaluation against treebank based ccg and hpsg l.jpg
    Evaluation against Treebank-Based CCG and HPSG

    • CCG = Combinatory Categorial Grammar (Steedman 2000)

    • HPSG = Head-Driven Phrase Structure Grammar (Pollard & Sag 1994)

      • Both constraint-based grammar formalisms

      • Treebank-based CCG resources (Hockenmaier & Steedman 2002, Hockenmaier 2003, Clark & Curran 2004, …)

      • Treebank-based HPSG resources (Miyao, Ninomiya & Tsujii 2003, Miyao & Tsujii 2004, …)

    • DepBank = reannotated version of PARC 700 (Briscoe & Carroll 2006) with CBS 500–style GRs

    • RASP (version 2) (Briscoe & Carroll 2006)

    Treebank-Based LFG Resources


    Evaluation against treebank based ccg and hpsg55 l.jpg
    Evaluation against Treebank-Based CCG and HPSG

    • CCG:

      • Small set of basic categories: {NP, N, PP, S}

      • Complex categories: VP = S\NP Vi = S\NP Vdi = (S\NP)/NP

      • Small set of combination rules:

        • X/Y Y  X

        • Y X\Y  X

        • X/Y Y/Z  X/Z

    Treebank-Based LFG Resources


    Evaluation against treebank based ccg and hpsg56 l.jpg
    Evaluation against Treebank-Based CCG and HPSG

    • HPSG:

      • Uniform representation: typed feature structures and inheritance

      • Sign: PHON, SYNSEM, DTRS

      • Inheritance hierarchy

      • Principles (HEAD-FEATURE, VALENCE, …)

      • Id-Schemata (HEAD-COMP, HEAD-MOD, …)

    Treebank-Based LFG Resources





    Probability models penn ii lfg l.jpg
    Probability Models: Penn-II & LFG

    Treebank-Based LFG Resources


    Probability models penn ii lfg61 l.jpg
    Probability Models: Penn-II & LFG

    Evaluation Results:

    Treebank-Based LFG Resources


    Probability models penn ii lfg62 l.jpg
    Probability Models: Penn-II & LFG

    Results are interesting as:

    • Extensive treebank preprocessing (clean-up, correction and restructuring) in CCG and (some in) HPSG

    • none in LFG

    • Custom-designed parsers and sophisticated (log-linear, max ent) parse selection probability models in HPSG and CCG

    • Mix of off-the-shelf and custom designed components, each with their own probability model in early-disambiguation processing pipeline in LFG, no proper overall probability model, but an approximation at best …

    • Still competitive results …

    Treebank-Based LFG Resources


    Probability models penn ii lfg63 l.jpg
    Probability Models: Penn-II & LFG

    Probability Models:

    • Our approach does not constitute proper probability model (Abney, 1996)

    • Why? Probability model leaks:

    • Highest ranking parse tree may feature f-structure equations that cannot be resolved into f-structure

    • Probability associated with that parse tree is lost

    • Doesn’t happen often in practice (coverage >99.5% on unseen data)

    • Research on appropriate discriminative, log-linear or maximum entropy models is important (Miyao and Tsujii, 2002) (Riezler et al. 2002)

    Treebank-Based LFG Resources


    Demo system l.jpg
    Demo System

    • http://lfg-demo.computing.dcu.ie/lfgparser.html

    Treebank-Based LFG Resources


    Applications generation l.jpg
    Applications: Generation

    Applications: Generation

    Treebank-Based LFG Resources


    Applications generation66 l.jpg
    Applications: Generation

    Research Question:

    • Can we make the automatically induced LFG resources reversible/bi-directional?

    • Can they be used for both (probabilistic) parsing and generation?

    Treebank-Based LFG Resources


    Generation penn ii lfg l.jpg
    Generation: Penn-II & LFG

    Treebank-Based LFG Resources


    Generation penn ii lfg68 l.jpg
    Generation: Penn-II & LFG

    Treebank-Based LFG Resources


    Generation penn ii lfg69 l.jpg
    Generation: Penn-II & LFG

    Treebank-Based LFG Resources


    Generation penn ii lfg70 l.jpg
    Generation: Penn-II & LFG

    Treebank-Based LFG Resources


    Generation penn ii lfg71 l.jpg
    Generation: Penn-II & LFG

    Treebank-Based LFG Resources


    Generation penn ii lfg72 l.jpg
    Generation: Penn-II & LFG

    Treebank-Based LFG Resources


    Generation penn ii lfg73 l.jpg
    Generation: Penn-II & LFG

    Treebank-Based LFG Resources


    Generation penn ii lfg74 l.jpg
    Generation: Penn-II & LFG

    Problem: conditioning of generation rules on purely local f-str features

    Solution I: generation grammar transformation (Cahill et al. 2006)

    Solution II: history-based probabilistic generation (Hogan et al. 2007, Cafferkey et al. 2007): condition generation rules on parent GF

    Treebank-Based LFG Resources


    Generation penn ii lfg75 l.jpg
    Generation: Penn-II & LFG

    Treebank-Based LFG Resources


    Generation penn ii lfg76 l.jpg
    Generation: Penn-II & LFG

    Treebank-Based LFG Resources


    Generation penn ii lfg77 l.jpg
    Generation: Penn-II & LFG

    Treebank-Based LFG Resources


    Generation the good the bad and the ugly l.jpg
    Generation: the Good, the Bad and the Ugly

    • Orig: Supporters of the legislation view the bill as an effort to add stability and certainty to the airline-acquisition process , and to preserve the safety and fitness of the industry .

    • Gen: Supporters of the legislation view the bill as an effort to add stability and certainty to the airline-acquisition process , and to preserve the safety and fitness of the industry.

    • Orig: The upshot of the downshoot is that the A 's go into San Francisco 's Candlestick Park tonight up two games to none in the best-of-seven fest .

    • Gen: The upshot of the downshoot is that the A 's tonight go into San Francisco 's Candlestick Park up two games to none in the best-of-seven fest .

    • Orig: By this time , it was 4:30 a.m. in New York , and Mr. Smith fielded a call from a New York customer wanting an opinion on the British stock market , which had been having troubles of its own even before Friday 's New York market break .

    • Gen: Mr. Smith fielded a call from New a customer York wanting an opinion on the market British stock which had been having troubles of its own even before Friday 's New York market break by this timeand in New York , it was 4:30 a.m. .

    • Orig: Only half the usual lunchtime crowd gathered at the tony Corney & Barrow wine bar on Old Broad Street nearby .

    • Gen: At wine tony Corney & Barrow the bar on Old Broad Street nearby gathered usual , lunchtime only half the crowd , .

    Treebank-Based LFG Resources


    Generation penn ii lfg79 l.jpg
    Generation: Penn-II & LFG

    Treebank-Based LFG Resources


    Generation penn ii lfg80 l.jpg
    Generation: Penn-II & LFG

    Problem: conditioning of generation rules on purely local f-str features

    Solution: generation grammar transformation (Cahill et al. 2006)

    Solution: history-based probabilistic generation (Hogan et al. 2007, Cafferkey et al. 2007): condition generation rules on parent GF

    Treebank-Based LFG Resources


    Generation the good the bad and the ugly81 l.jpg
    Generation: the Good, the Bad and the Ugly

    • Orig:By this time , it was 4:30 a.m. in New York , and Mr. Smith fielded a call from a New York customer wanting an opinion on the British stock market , which had been having troubles of its own even before Friday 's New York market break .

    • Gen:Mr. Smith fielded a call from New a customer York wanting an opinion on the market British stock which had been having troubles of its own even before Friday 's New York market break by this time and in New York , it was 4:30 a.m. . (Cahill et al. 2006) GGT

    • Gen: By this time , in New York , it was 4:30 a.m. , and Mr. Smith fielded a call from New a customer York , wanting an opinion on the market British stock which had been having troubles of its own even before Friday ’s New York market break . (Hogan et al. 2007) HB

    • Gen: By this time , in New York , it was 4:30 a.m. , and Mr. Smith fielded a call from a New York customer , wanting an opinion on the market British stock which had been having troubles of its own even before Friday ’s New York market break . (Hogan et al. 2007) HB + MWU

    Treebank-Based LFG Resources


    Generation chinese ctb2 l.jpg
    Generation: Chinese CTB2

    • CTB2 (Yuqing Guo - Toshiba China Beijing R&D Lab)

    • (Cahill et al. 2006) out of the box

    • Training articles 1-270 (3,480 sentences)

    • Testing articles 301-325 (351 sentences)

    Treebank-Based LFG Resources


    Applications machine translation l.jpg
    Applications: Machine Translation

    Applications: Machine Translation

    • Labelled Dependency-Based MT Evaluation (LaDEva)

    • Automatic Acquisition of Transfer Rules

    Treebank-Based LFG Resources


    Applications machine translation84 l.jpg
    Applications: Machine Translation

    Labelled-Dependency-Based MT Evaluation

    • Most automatic MT evaluation metrics (BLEU, NIST) are string (n-gram) based.

    • They unfairly punish perfectly legitimate syntactic and lexical variation:

      • Yesterday John resigned.

      • John resignedyesterday.

      • Yesterday John quit.

  • Legitimate lexical variation: throw in WordNet synonyms into the string match

  • What about syntactic variation?

  • Treebank-Based LFG Resources


    Applications machine translation85 l.jpg
    Applications: Machine Translation

    • Idea: use labelled dependencies for MT evaluation

    • Why: dependencies abstract away from some particulars of surface realisation

    • Adjunct placement, order of conjuncts in a coordination, topicalisation, ...

    Treebank-Based LFG Resources


    Applications machine translation86 l.jpg
    Applications: Machine Translation

    • Idea is intuitive

    • To make it happen you need a robust parser that can parse MT output 

    • Treebank-induced parsers parse anything …!

    • How do we judge whether labelled dependency-based method is better than string-based methods?

    • We compare (correlation) with human judgement/evaluation performance …

    • Why: humans not fooled by legitimate syntactic variation

    Treebank-Based LFG Resources


    Applications machine translation87 l.jpg
    Applications: Machine Translation

    • Experiment: use LDC Multiple Translation Chinese (MTC) Parts 2 and 4 data

    • 16,807 translation-reference human score segments

    • 5,007 test, rest for training (weights … etc.)

    • To make this work, we throw in

      • n-best parsing

      • WordNet synonyms

      • partial matching

      • training weights

      • etc …

    Treebank-Based LFG Resources


    Applications machine translation88 l.jpg
    Applications: Machine Translation

    Treebank-Based LFG Resources


    Applications machine translation89 l.jpg
    Applications: Machine Translation

    Treebank-Based LFG Resources


    References mt eval l.jpg
    References (MT Eval)

    • Karolina Owczarzak, Yvette Graham and Josef van Genabith: Using F-structures in Machine Translation Evaluation. In Proceedings of the 12th International Conference on Lexical Functional Grammar, July 28-30, 2007, Stanford, CA

    • Karolina Owczarzak, Josef van Genabith, and Andy Way. Labelled Dependencies in Machine Translation Evaluation. In Proceedings of ACL 2007 Workshop on Statistical Machine Translation, pages 104-111, Prague, Czech Republic

    • Karolina Owczarzak, Josef van Genabith, and Andy Way. Dependency-Based Automatic Evaluation for Machine Translation. In Proceedings of HLT-NAACL 2007 Workshop on Syntax and Structure in Statistical Translation. Rochester, NY.

    Treebank-Based LFG Resources


    References parsing l.jpg
    References (Parsing)

    • Aoife Cahill, Michael Burke, Ruth O'Donovan, Stefan Riezler, Josef van Genabith and Andy Way. 2008. Wide-Coverage Statistical Parsing Using Automatic Dependency Structure Annotation. Computational Linguistics, Volume 34, 1, MIT Press, March 2008. (accepted for publication)

    • Joachim Wagner, Djamé Seddah, Jennifer Foster and Josef van Genabith: C-Structures and F-Structures for the British National Corpus. In Proceedings of the 12th International Conference on Lexical Functional Grammar, July 28-30, 2007, Stanford, CA

    • A. Cahill, M. Burke, R. O'Donovan, J. van Genabith, and A. Way. Long-Distance Dependency Resolution in Automatically Acquired Wide-Coverage PCFG-Based LFG Approximations, In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), July 21-26 2004, pages 320-327, Barcelona, Spain, 2004

    • Cahill A, M. McCarthy, J. van Genabith and A. Way. Parsing with PCFGs and Automatic F-Structure Annotation, In M. Butt and T. Holloway-King (eds.): Proceedings of the Seventh International Conference on LFG CSLI Publications, Stanford, CA., pp.76--95. 2002

    Treebank-Based LFG Resources


    References generation lex acq l.jpg
    References (Generation, Lex. Acq.)

    • Deirdre Hogan, Conor Cafferkey, Aoife Cahill and Josef van Genabith, Exploiting Multi-Word Units in History-Based Probabilistic Generation, in Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Natural Language Learning (EMNLP-CoNLL 2007), Prague, Czech Republic. pp.267-276

    • A. Cahill and J. Van Genabith, Robust PCFG-Based Generation using Automatically Acquired LFG-Approximations, COLING/ACL 2006, Sydney, Australia

    • R. O'Donovan, M. Burke, A. Cahill, J. van Genabith and A. Way. Large-Scale Induction and Evaluation of Lexical Resources from the Penn-II and Penn-III Treebanks, Computational Linguistics, 2005

    • R. O'Donovan, M. Burke, A. Cahill, J. van Genabith, and A. Way. Large-Scale Induction and Evaluation of Lexical Resources from the Penn-II Treebank, In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), July 21-26 2004, pages 368-375, Barcelona, Spain, 2004

    Treebank-Based LFG Resources


    ad