Posterior Regularization for Structured Latent Variable Models. Li Z honghua I2R SMT Reading Group. Outline. Motivation and Introduction Posterior Regularization Application Implementation Some Related Frameworks. Motivation and Introduction. Prior Knowledge
Posterior Regularization for Structured Latent Variable Models
I2R SMT Reading Group
We posses a wealth of prior knowledge about most NLP tasks.
Leveraging Prior Knowledge
Possible approaches and their limitations
Bayesian Approach : Encode prior knowledge with a prior on parameters
Augmenting Model : Encode prior knowledge with additional variables and dependencies.
limitation: may make exact inference intractable
-- Constraint Features & Expectations
-- EM style learning algorithm
Original Objective :
EM style learning algorithm
Computing the Posterior Regularizer
Statistical Word Alignments
IBM Model 1 and HMM
One feature for each source word m, that counts how many times it is aligned to a target word in the alignment y.
Define feature for each target-source position pair i,j . The feature takes the value zero in expectation if a word pair i ,j is aligned with equal probability in both directions.
Learning Tractable Word Alignment Models with Complex Constraints CL10
more info: http://sideinfo.wikkii.com
many of my slides get from there