Mean-Field Theory and Its Applications In Computer Vision1

1. Mean-Field Theory and Its Applications In Computer Vision1

2. Introduction • Problem formulation • Mean-field based inference method • Strategy for incorporating different costs

3. Labelling problem Assign a label to each image pixel Object detection Object segmentation Stereo

4. Problem Formulation Find a Labelling that maximize the conditional probability

5. Inference Message Passing Move-Making • Besag. On the Statistical Analysis of Dirty Pictures, JRSS, 1986 • Boykov et al. Fast Approximate Energy Minimization via Graph Cuts, PAMI 2001 • Komodakis et al. Fast Approximate Optimal Solutions for Single and Dynamic MRFs, CVPR, 2007 • Lempitsky et al. Fusion Moves for Markov Random Field Optimization, PAMI, 2010 • T. Minka. Expectation Propagation for Approximate Bayesian Inference, UAI, 2001 • Murphy. Loopy Belief Propagation: An Empirical Study, UAI, 1999 • Jordan et.al. An Introduction to Variational Methods for Graphical Models, ML-1999 • J. Yedidia et al. Generalized Belief Propagation, NIPS, 2001 Other Algorithms Convex Relaxations • Chekuri et al. Approximation Algorithms for Metric Labelling, SODA, 2001 • M. Goemans et al. Improved Approximate Algorithms for Maximum-Cut, JACM, 1995 • M. Muramatsu et al. A New SOCP Relaxation for Max-Cut, JORJ, 2003 • RaviKumar et al. QP Relaxation for Metric Labelling, ICML 2006 • K. Alahari et.al. Dynamic Hybrid Algorithms for MAP Inference, PAMI 2010 • P. Kohli et al. On Partial Optimality in Multilabel MRFs, ICML, 2008 • C. Rother et al. Optimizing Binary MRFs via Extended Roof Duality, CVPR, 2007

6. Inference Message Passing • T. Minka. Expectation Propagation for Approximate Bayesian Inference, UAI, 2001 • Murphy. Loopy Belief Propagation: An Empirical Study, UAI, 1999 • Jordan et.al. An Introduction to Variational Methods for Graphical Models, ML-99 • J. Yedidia et al. Generalized Belief Propagation, NIPS, 2001 • Variational message passing algorithm • We focus on mean-field based inference

7. Mean-field methods • Mean-fields methods (Jordan et.al., 1999) • Intractable inference with distribution • Approximate distribution from tractable family

8. Variational Inference • Minimize the KL-divergence between Q and P

9. Variational Inference • Minimize the KL-divergence between Q and P

10. Variational Inference • Minimize the KL-divergence between Q and P

11. Variational Inference • Minimize the KL-divergence between Q and P

12. Markov Random Field (MRF) • Graph: • A simple MRF Product of potentials defined over cliques

13. Markov Random Field (MRF) • Graph: • In general Un-normalized part

14. Energy minimization • Potential and energy

15. Variational Inference Expectation of cost under Q distribution Entropy of Q

16. Naïve Mean Field • Family : assume all variables are independent

17. Variational Inference • Shannon’s entropy decomposes

18. Variational Inference • Stationary point solution • Marginal update in mean-field • Normalizing constant:

19. Variational Inference • Marginal for variable i taking label l

20. Variational Inference • Marginal for variable i taking label l • An assignment of all variables in clique c

21. Variational Inference • Marginal for variable i taking label l • An assignment of all variables in clique c • An assignment of all variables apart from x_i

22. Variational Inference • Marginal for variable i taking label l • An assignment of all variables in clique c • An assignment of all variables apart from x_i • Marginal distribution of all variables in c apart from x_i

23. Variational Inference • Marginal for variable i taking label l • An assignment of all variables in clique c • An assignment of all variables apart from x_i • Marginal distribution of all variables in c apart from x_i • Summation evaluates the expected value of cost over distribution Q given that x_i takes label l

24. Simple Illustration Naïve mean-field approximation

25. Mean-field algorithm • Iterative algorithm • Iterate till convergence • Update marginals of each variable in each iteration

26. Q distribution

27. Max posterior marginal (MPM) • MPM with approximate distribution: • MAP solution / most likely solution • Empirically achieves very high accuracy:

28. Structured Mean Field • Naïve mean field can lead to poor solution • Structured (higher order) mean-field

29. How to make a mean-field algorithm • Pick a model • Unary, pairwise, higher order cliques • Define a cost • Potts, linear truncated, robust PN • Calculate the marginal • Calculate the expectation of cost defined

30. How to make a mean-field algorithm • Use this plug-in strategy in many different models • Grid pairwise CRF • Dense pairwise CRF • Higher order model • Co-occurrence model • Latent variable model • Product label space