Posterior Regularization for Structured Latent Variable Models. Li Z honghua I2R SMT Reading Group. Outline. Motivation and Introduction Posterior Regularization Application Implementation Some Related Frameworks. Motivation and Introduction. Prior Knowledge
I2R SMT Reading Group
We posses a wealth of prior knowledge about most NLP tasks.
Leveraging Prior Knowledge
Possible approaches and their limitations
Bayesian Approach : Encode prior knowledge with a prior on parameters
Augmenting Model : Encode prior knowledge with additional variables and dependencies.
limitation: may make exact inference intractable
-- Constraint Features & Expectations
-- EM style learning algorithm
Original Objective :
EM style learning algorithm
Computing the Posterior Regularizer
Statistical Word Alignments
IBM Model 1 and HMM
One feature for each source word m, that counts how many times it is aligned to a target word in the alignment y.
Define feature for each target-source position pair i,j . The feature takes the value zero in expectation if a word pair i ,j is aligned with equal probability in both directions.
Learning Tractable Word Alignment Models with Complex Constraints CL10
more info: Modelshttp://sideinfo.wikkii.com
many of my slides get from there