Improving the Mean Field Approximation via the Use of Mixture Distributions

1. Improving the Mean Field Approximation via the Use of Mixture Distributions Tommi S. Jaakola Michael I. Jordan

2. 1. Introduction • Inference in Graphical Model • make use of conditional dependency with Markov properties : sparse graph without long cycles • mean field approximation : dense graph with many clusters • Improved Mean Field Approximation • multimodal rather than unimodal by mixture distribution • components of mixture are assumed to be simple factorized distributions : independent fluctuations

3. 2. Mean Field Approximation • Target calculation :

4. 3. Mixture Approximation

6. 4.Computing the Mutual Information

7. 5. Finding the Mixture Paramters

8. 6. Discussion • In the paper • presented a general methodology for using mixture distributions in approximate probabilistic calculations • showed that the bound on the likelihood resulting from mixture-based approximations • presented fitting parameters • Bishop, et al (1998) have presented empirical results using mixture approximation : better than simple factorized mean field approximation • Further research • combine the mixture approximation with exact methods