1 / 18

# Variational Bayes Model Selection for Mixture Distribution - PowerPoint PPT Presentation

Variational Bayes Model Selection for Mixture Distribution. Authors: Adrian Corduneanu & Christopher M. Bishop. Presented by Shihao Ji Duke University Machine Learning Group Jan. 20, 2006 . Outline. Introduction – model selection Automatic Relevance Determination (ARD) Experimental Results

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

## PowerPoint Slideshow about ' Variational Bayes Model Selection for Mixture Distribution' - teenie

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Variational Bayes Model Selectionfor Mixture Distribution

Authors: Adrian Corduneanu & Christopher M. Bishop

Presented by Shihao Ji

Duke University Machine Learning Group

Jan. 20, 2006

• Introduction – model selection

• Automatic Relevance Determination (ARD)

• Experimental Results

• Application to HMMs

• Cross validation

• Bayesian approaches

• MCMC and Laplace approximation

• (Traditional) variational method

• (Type II) variational method

• relevance vector regression

• Given a dataset , we assume is Gaussian

Likelihood:

Prior:

Posterior:

Determination of hyperparameters:

Type II ML

• mixture of Gaussian

• Given an observed dataset , we assume each data point is drawn

• independently from a mixture of Gaussian density

Likelihood:

Prior:

Posterior:

VB

Determination of mixing coefficients:

Type II ML

• model selection

Bayesian method: ,

Component elimination: if ,

i.e.,

• Bayesian method vs. cross-validation

600 points drawn from a mixture of 5 Gaussians.

• Component elimination

Initially the model had 15 mixtures, finally was pruned down to 3 mixtures

• hidden Markov model

• Given an observed dataset , we assume each data sequence is

• generated independently from an HMM

Likelihood:

Prior:

Posterior:

VB

Determination of p and A:

Type II ML

• model selection

Bayesian method: ,

State elimination: if ,

Define -- visiting frequency

where