Posterior regularization for structured latent variable models
Download
1 / 26

Posterior Regularization for Structured Latent Variable Models - PowerPoint PPT Presentation


  • 144 Views
  • Uploaded on

Posterior Regularization for Structured Latent Variable Models. Li Z honghua I2R SMT Reading Group. Outline. Motivation and Introduction Posterior Regularization Application Implementation Some Related Frameworks. Motivation and Introduction. Prior Knowledge

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Posterior Regularization for Structured Latent Variable Models' - krysta


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Posterior regularization for structured latent variable models

Posterior Regularization for Structured Latent Variable Models

Li Zhonghua

I2R SMT Reading Group


Outline
Outline Models

  • Motivation and Introduction

  • Posterior Regularization

  • Application

  • Implementation

  • Some Related Frameworks


Motivation and introduction
Motivation and Introduction Models

Prior Knowledge

We posses a wealth of prior knowledge about most NLP tasks.


Motivation and introduction prior knowledge
Motivation and Introduction Models--Prior Knowledge


Motivation and introduction prior knowledge1
Motivation and Introduction Models--Prior Knowledge


Motivation and introduction1
Motivation and Introduction Models

Leveraging Prior Knowledge

Possible approaches and their limitations


Motivation and introduction limited approach
Motivation and Introduction Models--Limited Approach

Bayesian Approach : Encode prior knowledge with a prior on parameters

  • Limitation:Our prior knowledge is not about parameters!

  • Parameters are difficult to interpret; hard to get desired effect.


Motivation and introduction limited approach1
Motivation and Introduction Models--Limited Approach

Augmenting Model : Encode prior knowledge with additional variables and dependencies.

limitation: may make exact inference intractable


Posterior regularization
Posterior Regularization Models

  • A declarative language for specifying prior knowledge

    -- Constraint Features & Expectations

  • Methods for learning with knowledge in this language

    -- EM style learning algorithm



Posterior regularization2
Posterior Regularization Models

Original Objective :


Posterior regularization3
Posterior Regularization Models

EM style learning algorithm


Posterior regularization4
Posterior Regularization Models

Computing the Posterior Regularizer


Application
Application Models

Statistical Word Alignments

IBM Model 1 and HMM


Application1
Application Models

One feature for each source word m, that counts how many times it is aligned to a target word in the alignment y.


Application2
Application Models

Define feature for each target-source position pair i,j . The feature takes the value zero in expectation if a word pair i ,j is aligned with equal probability in both directions.


Application3
Application Models

Learning Tractable Word Alignment Models with Complex Constraints CL10


Application4
Application Models

  • Six language pairs

  • both types of constraints improve over the HMM in terms of both precision and recall

  • improve over the HMM by 10% to 15%

  • S-HMM performs slightly better than B-HMM

  • S-HMM performs better than B-HMM in 10 out of 12 cases

  • improve over IBM M4 9 times out of 12


Application5
Application Models


Implementation
Implementation Models

  • http://code.google.com/p/pr-toolkit/







more info: Modelshttp://sideinfo.wikkii.com

many of my slides get from there

Thanks!


ad