variational bayes model selection for mixture distribution
Download
Skip this Video
Download Presentation
Variational Bayes Model Selection for Mixture Distribution

Loading in 2 Seconds...

play fullscreen
1 / 18

Variational Bayes Model Selection for Mixture Distribution - PowerPoint PPT Presentation


  • 121 Views
  • Uploaded on

Variational Bayes Model Selection for Mixture Distribution. Authors: Adrian Corduneanu & Christopher M. Bishop. Presented by Shihao Ji Duke University Machine Learning Group Jan. 20, 2006 . Outline. Introduction – model selection Automatic Relevance Determination (ARD) Experimental Results

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Variational Bayes Model Selection for Mixture Distribution' - teenie


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
variational bayes model selection for mixture distribution

Variational Bayes Model Selectionfor Mixture Distribution

Authors: Adrian Corduneanu & Christopher M. Bishop

Presented by Shihao Ji

Duke University Machine Learning Group

Jan. 20, 2006

slide2
Outline
  • Introduction – model selection
  • Automatic Relevance Determination (ARD)
  • Experimental Results
  • Application to HMMs
slide3
Introduction
  • Cross validation
  • Bayesian approaches
    • MCMC and Laplace approximation
    • (Traditional) variational method
    • (Type II) variational method
slide4
Automatic Relevance Determination (ARD)
  • relevance vector regression
  • Given a dataset , we assume is Gaussian

Likelihood:

Prior:

Posterior:

Determination of hyperparameters:

Type II ML

slide5
Automatic Relevance Determination (ARD)
  • mixture of Gaussian
  • Given an observed dataset , we assume each data point is drawn
  • independently from a mixture of Gaussian density

Likelihood:

Prior:

Posterior:

VB

Determination of mixing coefficients:

Type II ML

slide6
Automatic Relevance Determination (ARD)
  • model selection

Bayesian method: ,

Component elimination: if ,

i.e.,

slide7
Experimental Results
  • Bayesian method vs. cross-validation

600 points drawn from a mixture of 5 Gaussians.

slide8
Experimental Results
  • Component elimination

Initially the model had 15 mixtures, finally was pruned down to 3 mixtures

slide10
Automatic Relevance Determination (ARD)
  • hidden Markov model
  • Given an observed dataset , we assume each data sequence is
  • generated independently from an HMM

Likelihood:

Prior:

Posterior:

VB

Determination of p and A:

Type II ML

slide11
Automatic Relevance Determination (ARD)
  • model selection

Bayesian method: ,

State elimination: if ,

Define -- visiting frequency

where

ad