Variational bayes model selection for mixture distribution
This presentation is the property of its rightful owner.
Sponsored Links
1 / 18

Variational Bayes Model Selection for Mixture Distribution PowerPoint PPT Presentation


  • 87 Views
  • Uploaded on
  • Presentation posted in: General

Variational Bayes Model Selection for Mixture Distribution. Authors: Adrian Corduneanu & Christopher M. Bishop. Presented by Shihao Ji Duke University Machine Learning Group Jan. 20, 2006 . Outline. Introduction – model selection Automatic Relevance Determination (ARD) Experimental Results

Download Presentation

Variational Bayes Model Selection for Mixture Distribution

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Variational bayes model selection for mixture distribution

Variational Bayes Model Selectionfor Mixture Distribution

Authors: Adrian Corduneanu & Christopher M. Bishop

Presented by Shihao Ji

Duke University Machine Learning Group

Jan. 20, 2006


Variational bayes model selection for mixture distribution

Outline

  • Introduction – model selection

  • Automatic Relevance Determination (ARD)

  • Experimental Results

  • Application to HMMs


Variational bayes model selection for mixture distribution

Introduction

  • Cross validation

  • Bayesian approaches

    • MCMC and Laplace approximation

    • (Traditional) variational method

    • (Type II) variational method


Variational bayes model selection for mixture distribution

Automatic Relevance Determination (ARD)

  • relevance vector regression

  • Given a dataset , we assume is Gaussian

Likelihood:

Prior:

Posterior:

Determination of hyperparameters:

Type II ML


Variational bayes model selection for mixture distribution

Automatic Relevance Determination (ARD)

  • mixture of Gaussian

  • Given an observed dataset , we assume each data point is drawn

  • independently from a mixture of Gaussian density

Likelihood:

Prior:

Posterior:

VB

Determination of mixing coefficients:

Type II ML


Variational bayes model selection for mixture distribution

Automatic Relevance Determination (ARD)

  • model selection

Bayesian method: ,

Component elimination: if ,

i.e.,


Variational bayes model selection for mixture distribution

Experimental Results

  • Bayesian method vs. cross-validation

600 points drawn from a mixture of 5 Gaussians.


Variational bayes model selection for mixture distribution

Experimental Results

  • Component elimination

Initially the model had 15 mixtures, finally was pruned down to 3 mixtures


Variational bayes model selection for mixture distribution

Experimental Results


Variational bayes model selection for mixture distribution

Automatic Relevance Determination (ARD)

  • hidden Markov model

  • Given an observed dataset , we assume each data sequence is

  • generated independently from an HMM

Likelihood:

Prior:

Posterior:

VB

Determination of p and A:

Type II ML


Variational bayes model selection for mixture distribution

Automatic Relevance Determination (ARD)

  • model selection

Bayesian method: ,

State elimination: if ,

Define -- visiting frequency

where


Variational bayes model selection for mixture distribution

Experimental Results (1)


Variational bayes model selection for mixture distribution

Experimental Results (2)


Variational bayes model selection for mixture distribution

Experimental Results (3)


Variational bayes model selection for mixture distribution

Questions?


  • Login