Stochastic approximate inference
This presentation is the property of its rightful owner.
Sponsored Links
1 / 29

Stochastic approximate inference PowerPoint PPT Presentation


  • 68 Views
  • Uploaded on
  • Presentation posted in: General

Stochastic approximate inference. Kay H. Brodersen Computational Neuroeconomics Group Department of Economics University of Zurich Machine Learning and Pattern Recognition Group Department of Computer Science ETH Zurich http://people.inf.ethz.ch/bkay/. When do we need approximate inference?.

Download Presentation

Stochastic approximate inference

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Stochastic approximate inference

Stochastic approximate inference

Kay H. Brodersen

Computational Neuroeconomics GroupDepartment of EconomicsUniversity of Zurich

Machine Learning and Pattern Recognition GroupDepartment of Computer ScienceETH Zurich

http://people.inf.ethz.ch/bkay/


When do we need approximate inference

When do we need approximate inference?

sample from an arbitrary distribution

compute an expectation

compute an expectation

compute an expectation

How to evaluate the posterior distribution of the model parameters?

How to compute the evidence term?

How to compute the expectation of the posterior?

How to make a point prediction?


Which type of approximate inference

Which type of approximate inference?

Stochastic approximationsthrough sampling

  • computationally expensive

  • storage intensive

  • asymptotically exact

  • easily applicable general-purpose algorithms

Deterministic approximationsthrough structural assumptions

  • application often requires mathematical derivations (hard work)

  • systematic error

  • computationally efficient

  • efficient representation

  • learning rules may give additional insight


Themes in stochastic approximate inference

Themes in stochastic approximate inference

Sampling from a desired target distributionwe need to find a way of drawing random numbers from some target distribution

Computing an expectation w.r.t. that target distribution

we can approximate the expectation of using the sample mean:

sampling

z

summary

z


Transformation method

1

Transformation method


Transformation method for sampling from

Transformation method for sampling from

transformation

Idea: we can obtain samples from some distribution by first sampling from the uniform distribution and then transforming these samples.


Transformation method algorithm

Transformation method: algorithm

  • Algorithm for sampling from

    • Draw a random number from the uniform distribution:

    • Transform by applying the inverse cumulative density function (cdf) of the desired target distribution:

    • Repeat both steps for .


Transformation method example

Transformation method: example

  • Example: sampling from the exponential distribution

    • The desired pdf is:

    • The corresponding cdf is:

    • The inverse cdf is:

    • Thus, is a sample from the exponential distribution.

  • Implementation in MATLAB

    for t=1:10000

    z(t) = -1/lambda*log(1-rand);

    end

    hist(z)

    mean(z)


Transformation method summary

Transformation method: summary

  • Discussion

    • yields high-quality samples

    • easy to implement

    • computationally efficient

    • obtaining the inverse cdf can be difficult


Rejection sampling and importance sampling

2

Rejection samplingand importance sampling


Rejection sampling

Rejection sampling

The proposal distribution , scaled by a factor , represents an envelope of .

Idea: when the transformation method cannot be applied, we can resort to a more general method called rejection sampling. Here, we draw random numbers from a simpler proposal distribution and keep only some of these samples.


Rejection sampling algorithm

Rejection sampling: algorithm

  • Algorithm for sampling from

    • Sample from

    • Sample from

    • If , then accept the sample:

    • Otherwise, discard and .

    • Repeat until we have obtained accepted samples.


Importance sampling

Importance sampling

  • Idea: if our goal is to compute the expectation , we can outperform rejection sampling by bypassing the generationg of random samples.

  • Naïve approach

    • A naïve approach would be to approximate the expectation as follows. Rather than sampling from , we discretize -space into a uniform grid and evaluate:

    • There are two problems with this approach:

      • The number of terms in the summation grows exponentially with the dimensionality of .

      • Only a small proportion of the samples will make a significant contribution to the sum. Uniform sampling clearly is very inefficient.


Importance sampling1

Importance sampling

Addressing the two problems of the naive approach above, given a proposal distribution , we can approximate the expectation aswhere the samples are drawn from .

The quantities are known as importance weights, and they correct the bias introduced by sampling from the wrong distribution.

Unlike in the case of rejection sampling, all of the generated samples are retained.


Markov chain monte carlo

3

Markov Chain Monte Carlo


Markov chain monte carlo mcmc

Markov Chain Monte Carlo (MCMC)

Idea: we can sample from a large class of distributions and overcome the problems that previous methods face in high dimensions using a framework called Markov Chain Monte Carlo.


Background on markov chains

Background on Markov chains

  • A first-order Markov chain is defined as a series of random variables such the following conditional-independence property holds:

  • Thus, the graphical model of a Markov chain is a chain:

  • A Markov chain is specified in terms of

    • the initial probability distribution

    • the transition probabilities


Background on markov chains1

Background on Markov chains

Markov chain: state diagram

Equlibrium distribution

2

0.5

1

0.5

0.5

3

1

2

3


Background on markov chains2

Background on Markov chains

Markov chain: state diagram

Equlibrium distribution

2

1

3

1

2

3


The idea behind mcmc

The idea behind MCMC

Desired target distribution

Empirical distribution of samples

z

z

Markov chain whose equilibrium distribution is

1

3

5

7

2

4

6

8


Metropolis algorithm

Metropolis algorithm

  • Algorithm for sampling from

    • Initialize by drawing somehow.

    • At cycle , draw a candidate sample from .Importantly, needs to be symmetric, i.e., .

    • Accept with probability,and otherwise set .

  • Notes

    • In contrast to rejection sampling, each cycle leads to a new sample, even when the candidate is discarded.

    • Note that the sequence is not a set of independent samples from because successive samples are highly correlated.


Metropolis hastings algorithm

Metropolis-Hastings algorithm

  • Algorithm for sampling from

    • Initialize by drawing somehow.

    • At cycle , draw a candidate sample from .In contrast to the Metropolis algorithm (see previous slide), no longer needs to be symmetric.

    • Accept with probability,and otherwise set .


Metropolis accept or reject

Metropolis: accept or reject?

p(z*)

Inrease in density:

p(zτ)

z*

p(zτ)

Decreasein density:

p(z*)

z*


Gibbs sampling

Gibbs sampling

Idea: as an alternative to the Metropolis-Hastings algorithm, Gibbs sampling is less broadly applicable but does away with acceptance tests and can therefore be more efficient.

Suppose we wish to sample from a multivariate distribution , e.g., representing several variables in a model. For example, we might be interested in their joint posterior distribution.

In Gibbs sampling, we update one component at a time.


Gibbs sampling1

Gibbs sampling

  • Algorithm for sampling from

    • Initialize somehow.

    • At cycle , sample , i.e., replace the variable by a new sample, drawn from a distribution that is conditioned of the current values of all other variables. The resulting new vector is our new sample.

    • In the next cycle, replace a different variable . The simplest procedure is to go round Alternatively, could be chosen randomly.


Summary

Summary

  • Throughout Bayesian statistics, we encounter intractable problems. Most of these problems are: (i) evaluating a distribution; or (ii) computing the expectation of a distribution.

  • Sampling methods provide a stochastic alternative to deterministic methods. They are usually computationally less efficient, but are asymptotically correct, broadly applicable, and easy to implement.

  • We looked at three main approaches:

    • Transformation method: efficient sampling from simple distributions

    • Rejection sampling and importance sampling: sampling from arbitrary distributions; direct computation of an expected value

    • Monte Carlo Markov Chain (MCMC): efficient sampling from high-dimensional distributions through the Metropolis-Hastings algorithm or Gibbs sampling


Stochastic approximate inference

Supplementary slides


Transformation method proof

Transformation method: proof

  • Procedure:

  • Proof:. Is ?Moreover: Therefore:


Background on markov chains3

Background on Markov chains

Ergodicity: A Markov chain is ergodic if the distribution converges to an invariant distribution irrespective of the choice of initial distribution . A homogeneous Markov chain will be ergodic, subject to some weak restrictions on the invariant distribution and the transition probabilities.

Equilibrium distribution: Given an ergodic Markov chain, that invariant distribution which any initial distribution converges to is called the equilibrium distribution. (An ergodic Markov chain may have several invariant distributions but only one of these is the equilibrium distribution.)


  • Login