Loading in 2 Seconds...
Loading in 2 Seconds...
Margin-based Decomposed Amortized Inference Gourab Kundu, Vivek Srikumar, Dan Roth. Amortized Inference. General Recipe.
Gourab Kundu, Vivek Srikumar, Dan Roth
Key Observations: (1) In NLP, we solve a lot of inference problems, at least one per sentence. (2)Redundancy of structures: The number of observed structures (blue solid line) is much smaller than the number of inputs (red dotted line). Moreover, the distribution of observed structures is highly skewed (inset). (Eg. for POS, a small number of tag sequences are much more frequent than the rest.) Pigeon Principle Applies.
Research Question: Can we solve the k-th inference instance much fast than the 1st?
Amortized inference (Srikumar et al 2012) shows how computation from earlier inference instances can be used to speed up inference for new, previously unseen instances.
If CONDITION(problem cache, newproblem)
then (no need to call the solver)
SOLUTION(new problem) = old solution
Call base solver and update cache
+A theorem guaranteeing correctness
2. Decomposed Amortized Inference using Lagrangian Relaxation
1. Margin based Amortization
More redundancy among
Redundancy in components of structures: We extend amortization to cases where the full structured output is not repeatedby storing partial computation for future inference problems.
Semantic Role Labeling
Entities and Relations
[Roth & Yih 2004]
[Punyakanok, et al 2008]
We simulate a long-running NLP process by caching problems and solutions from Gigaword corpus. We used a database engine to cache ILPs and their solutions along their and structured margin. We compare our approaches to a state-of-the-art ILP solver (Gurobi) and also to Theorem 1 from (Srikumar et al.2012).
This work continues the original work on Amortization: Srikumar, Kundu and Roth. On Amortizing Inference Cost for Structured Prediction. EMNLP, 2012
Solve only one in four problems
Solve only one in six problems
Wall clock improvements too
This research is sponsored by the Army Research Laboratory (ARL) under agreement W911NF-09-2-0053, Defense Advanced Research Projects Agency (DARPA) Machine Reading Program under Air Force Research Laboratory (AFRL) prime contract no. FA8750-09-C-0181, DARPA under agreement number FA8750-13-2-0008, Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior National Business Center contract number D11PC20155.