1 / 35

Approximate Factoring for A* Search

This paper explores the use of approximate factoring to improve A* search in NLP tasks. The authors propose a heuristic design that is both tight and admissible, resulting in efficient computation. The paper presents experiments on bitext parsing and lexicalized parsing to demonstrate the effectiveness of the proposed approach.

wallin
Download Presentation

Approximate Factoring for A* Search

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Approximate Factoring for A* Search Aria Haghighi, John DeNero, and Dan Klein Computer Science Division University of California Berkeley

  2. Inference for NLP Tasks A* Search

  3. Inference as Search Partial Hypothesis a3 a2 a2 y a1

  4. Bitext Parsing as Search S S S S’ NP VP NP VP Target Source translation is hard , la traducción es dificil Weighted Synchronous Grammar Parsing O(n6) Modified CKY over bi-spans (X[i,j],X’[i’,j’])

  5. A* Search y Score So Far Completion Score

  6. A* Search • Heuristic Design • Tight small • Admissible • Efficient to compute This way hypothesis! Optimal Result A* Heuristic Man

  7. A* Example: Bitext Search Bi-Span Viterbi Inside Score Cost So Far

  8. A* Bitext Search Viterbi Outside Score Completion Score Ideal Heuristic O(n6)

  9. Of Stately Projections ¼ S’ S S S’ S S’ NP’ VP’ NP VP S’ S NP’ VP’ NP VP S S’ S S

  10. A* Bitext Search S S S S’ S S’ NP VP NP VP NP VP NP’ VP’ Suppose, Then,

  11. Projection Heuristic O(n6) O(n3) O(n3) Klein and Manning [2003]

  12. When models don’t factorize

  13. When models don’t factorize c(a) y x Át(a) Ás(a) ¼s(y) ¼t(y) ¼t(x) ¼s(x) Pointwise Admissibility

  14. When models don’t factorize y ¼t(y) ¼s(y) Admissibility

  15. Finding Factored Costs How to find Ás and Át? Pointwise Gap

  16. Finding Factored Costs Small gaps

  17. Finding Factored Costs Pointwise Admissibility

  18. Finding Factored Costs

  19. Bitext Experiments Synchronous Tree-to-Tree Transducer • Trained on 40k sentences of English-Spanish Europarl [Galley et. al, 2004] • Rare words replaced with POS tags • Tested on 1,200 sent. max length 5-15 Optimization Problem • Solved only once per grammar • 206K Variables • 160KConstraints • 29 minutes

  20. Bitext Experiments

  21. Bitext Experiments

  22. Bitext Experiments Zhang and Gildea (2006)

  23. Bitext Experiments Zhang and Gildea (2006)

  24. Lexicalized Parsing S-(is,VBZ) VP-(is,VBZ) NP-(translation,NN) (is,VBZ) S (translation, NN) NP VP Klein and Manning [2003]

  25. Lexicalized Parsing

  26. Lexicalized Parsing Too many constraints to efficiently solve! Over 64e13 possible lexicalized rules

  27. Lexicalized Parsing

  28. Lexicalized Parsing

  29. Lexicalized Parsing

  30. Lexicalized Parsing

  31. Lexicalized Model Experiments Standard Setup • Train on section 2-21 of the treebank • Test on section 23 (length · 40) Models Tested • Factored model [Klein and Manning, 2003] • Non-Factored Model

  32. Lexicalized Parsing Factored Model [Klein and Manning, 2003]

  33. Lexicalized Parsing Non-Factored Model

  34. Conclusions • Generaltechnique for generating A* estimates • Can explicitly control admissibility tightness trade-off • Future Work: Explore different objectives and applications

  35. Thanks http://nlp.cs.berkeley.edu

More Related