Simulation of the matrix Bingham-von Mises-Fisher distribution, with applications to multivariate an...
Download
1 / 31

Discussion led by Chunping Wang ECE, Duke University July 10, 2009 - PowerPoint PPT Presentation


  • 65 Views
  • Uploaded on

Simulation of the matrix Bingham-von Mises-Fisher distribution, with applications to multivariate and relational data. Peter D. Hoff to appear in Journal of Computational and Graphical Statistics. Discussion led by Chunping Wang ECE, Duke University July 10, 2009. Outline.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Discussion led by Chunping Wang ECE, Duke University July 10, 2009' - shaeleigh-aguirre


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Simulation of the matrix Bingham-von Mises-Fisher distribution, with applications to multivariate and relational data

Peter D. Hoffto appear in Journal of Computational and Graphical Statistics

Discussion led by Chunping Wang

ECE, Duke University

July 10, 2009


Outline
Outline distribution, with applications to multivariate and relational data

  • Introduction and Motivations

  • Sampling from the Vector Von Mises-Fisher (vMF) Distribution (existing method)

  • Sampling from the Matrix Von Mises-Fisher (mMF) Distribution

  • Sampling from the Bingham-Von Mises-Fisher (BMF) Distribution

  • One Example

  • Conclusions

1/21


Introduction
Introduction distribution, with applications to multivariate and relational data

Stiefel manifold: set of rank- orthonormal matrices, denoted

The matrix Bingham-von Mises-Fisher distribution

The matrix von Mises-Fisher distribution – linear term

The matrix Bingham distribution – quadratic term

2/21


Motivations
Motivations distribution, with applications to multivariate and relational data

Sampling orthonormal matrices from distributions is useful for many applications.

Examples:

  • Factor analysis

observed matrix

latent

latent

Given uniform priors over Stiefel manifold,

3/21


Motivations1
Motivations distribution, with applications to multivariate and relational data

  • Principal components

observed matrix, with each row

with

Eigen-value decomposition

Likelihood

Posterior with respect to uniform prior

4/21


Motivations2
Motivations distribution, with applications to multivariate and relational data

  • Network data

, symmetric binary observed matrix, with the 0-1

indicator of a link between nodes i and j.

E: symmetric matrix of independent standard normal noise

Posterior with respect to uniform prior

5/21


Sampling from the vmf distribution wood 1994
Sampling from the vMF Distribution (wood, 1994) distribution, with applications to multivariate and relational data

the modal vector;

, concentration parameter

A distribution on the -sphere in

constant distribution for any given angle

defines the modal direction.

6/21


Sampling from the vmf distribution wood 19941
Sampling from the vMF Distribution (wood, 1994) distribution, with applications to multivariate and relational data

(1) A simple direction

( Proposal envelope )

(2) An arbitrary direction

For a fixed orthogonal matrix ,

7/21


Sampling from the mmf distribution
Sampling from the mMF Distribution distribution, with applications to multivariate and relational data

Rejection sampling scheme 1: uniform envelope

rejection region

a bound

Sample

accept

when

Acceptance region

Extremely inefficient

8/21


Sampling from the mmf distribution1

Y distribution, with applications to multivariate and relational data

Y

Y

Sampling from the mMF Distribution

Rejection sampling scheme 2: based on sampling from vMF

Proposal samples are drawn from vMF density functions with parameter , constrained to be orthogonal to other columns of .

9/21


Sampling from the mmf distribution2

Y distribution, with applications to multivariate and relational data

Y

Y

Sampling from the mMF Distribution

Rejection sampling scheme 2: based on sampling from vMF

Proposal samples are drawn from vMF density functions with parameter , constrained to be orthogonal to other columns of .

9/21


Sampling from the mmf distribution3

Y distribution, with applications to multivariate and relational data

Y

Y

Sampling from the mMF Distribution

Rejection sampling scheme 2: based on sampling from vMF

Proposal samples are drawn from vMF density functions with parameter , constrained to be orthogonal to other columns of .

Rotate the modal direction

9/21


Sampling from the mmf distribution4

Y distribution, with applications to multivariate and relational data

Y

Y

Sampling from the mMF Distribution

Rejection sampling scheme 2: based on sampling from vMF

Proposal samples are drawn from vMF density functions with parameter , constrained to be orthogonal to other columns of .

Rotate the sample to be orthogonal to the previous columns

9/21


Sampling from the mmf distribution5

Y distribution, with applications to multivariate and relational data

Y

Y

Sampling from the mMF Distribution

Rejection sampling scheme 2: based on sampling from vMF

Proposal samples are drawn from vMF density functions with parameter , constrained to be orthogonal to other columns of .

Proposal distribution

9/21


Sampling from the mmf distribution6
Sampling from the mMF Distribution distribution, with applications to multivariate and relational data

Rejection sampling scheme 2: based on sampling from vMF

Sample scheme:

10/21


Sampling from the mmf distribution7
Sampling from the mMF Distribution distribution, with applications to multivariate and relational data

A Gibbs sampling scheme

Sample iteratively

  • Note that . When .

  • remedy: sampling two columns at a time

  • Non-orthogonality among the columns of add to the autocorrelation in the Gibbs sampler.

  • remedy: performing the Gibbs sampler on

11/21


Sampling from the bmf distribution
Sampling from the BMF Distribution distribution, with applications to multivariate and relational data

The vector Bingham distribution

12/21


Sampling from the bmf distribution1
Sampling from the BMF Distribution distribution, with applications to multivariate and relational data

The vector Bingham distribution

12/21


Sampling from the bmf distribution2
Sampling from the BMF Distribution distribution, with applications to multivariate and relational data

The vector Bingham distribution

Better mixing

12/21


Sampling from the bmf distribution3
Sampling from the BMF Distribution distribution, with applications to multivariate and relational data

The vector Bingham distribution

From variable substitution, rejection sampling or grid sampling

12/21


Sampling from the bmf distribution4
Sampling from the BMF Distribution distribution, with applications to multivariate and relational data

The vector Bingham distribution

The density is symmetric about zero

12/21


Sampling from the bmf distribution5
Sampling from the BMF Distribution distribution, with applications to multivariate and relational data

The vector Bingham-von Mises-Fisher distribution

The density is not symmetric about zero any more, is no longer uniformly distributed on . The update of and should be done jointly. The modified step 2(b) and 2(c) are:

13/21


Sampling from the bmf distribution6
Sampling from the BMF Distribution distribution, with applications to multivariate and relational data

The matrix Bingham-von Mises-Fisher distribution

Rewrite

14/21


Sampling from the bmf distribution7
Sampling from the BMF Distribution distribution, with applications to multivariate and relational data

The matrix Bingham-von Mises-Fisher distribution

Sample two columns at a time

Parameterize 2-dimensional orthonormal matrices as

Uniform pairs on the circle

Uniform

15/21


Sampling from the bmf distribution8
Sampling from the BMF Distribution distribution, with applications to multivariate and relational data

The matrix Bingham-von Mises-Fisher distribution

16/21


Example eigenmodel estimation for network data
Example: Eigenmodel estimation for network data distribution, with applications to multivariate and relational data

17/21


Example eigenmodel estimation for network data1
Example: Eigenmodel estimation for network data distribution, with applications to multivariate and relational data

, symmetric binary observed matrix, with the 0-1

indicator of a link between nodes i and j.

E: symmetric matrix of independent standard normal noise

Posterior with respect to uniform prior

BMF distribution with

18/21


Example eigenmodel estimation for network data2
Example: Eigenmodel estimation for network data distribution, with applications to multivariate and relational data

Samples from two independent Markov chains with different starting values

19/21


Example eigenmodel estimation for network data3
Example: Eigenmodel estimation for network data distribution, with applications to multivariate and relational data

20/21


Conclusions
Conclusions distribution, with applications to multivariate and relational data

  • The sampling scheme of a family of exponential distributions over the Stiefel manifold was developed;

  • This enables us to make Bayesian inference for those orthonormal matrices and incorporate prior information during the inference;

  • The author mentioned several application and implemented the sampling scheme on a network data set.

21/21


References
References distribution, with applications to multivariate and relational data

  • Andrew T. A. Wood. Simulation of the von Mises Fisher distribution. Comm. Statist. Simulation Comput., 23:157-164, 1994

  • G. Ulrich. Computer generation of distributions on the m-sphere. Appl. Statist., 33, 158-163, 1984

  • J. G. Saw. A family of distributions on the m-sphere and some hypothesis tests. Biometrika, 65, 69-74, 1978


ad