a gentle introduction to gaussian distribution l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
A gentle introduction to Gaussian distribution PowerPoint Presentation
Download Presentation
A gentle introduction to Gaussian distribution

Loading in 2 Seconds...

play fullscreen
1 / 21

A gentle introduction to Gaussian distribution - PowerPoint PPT Presentation


  • 116 Views
  • Uploaded on

A gentle introduction to Gaussian distribution. X = 0. X = 1. X: Random variable. Review. Random variable Coin flip experiment. P(x). P(x) >= 0. 0. 1. x. Review. Probability mass function (discrete). Any other constraints? Hint: What is the sum?. Example: Coin flip experiment.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'A gentle introduction to Gaussian distribution' - royce


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
review

X = 0

X = 1

X: Random variable

Review
  • Random variable
  • Coin flip experiment
review3

P(x)

P(x) >= 0

0

1

x

Review
  • Probability mass function (discrete)

Any other constraints?

Hint: What is the sum?

Example: Coin flip experiment

review4

f(x)

f(x) >= 0

x

Review
  • Probability density function (continuous)

Unlike discrete,

Density function does not represent

probability but its rate of change called the “likelihood”

Examples?

review5

f(x)

f(x) >= 0

x

Review
  • Probability density function (continuous)

& Integrates to 1.0

x0

X0+dx

P( x0 < x < x0+dx ) = f(x0).dx

But, P( x = x0 ) = 0

the gaussian distribution
The Gaussian Distribution

Courtesy: http://research.microsoft.com/~cmbishop/PRML/index.htm

central limit theorem
Central Limit Theorem

The distribution of the sum of N i.i.d. random variables becomes increasingly Gaussian as N grows.

Example: N uniform [0,1] random variables.

central limit theorem coin flip
Central Limit Theorem (Coin flip)
  • Flip coin N times
  • Each outcome has an associated random variable Xi (=1, if heads, otherwise 0)
  • Number of heads
  • NH is a random variable
    • Sum of N i.i.d. random variables

NH = x1 + x2 + …. + xN

central limit theorem coin flip10
Central Limit Theorem (Coin flip)
  • Probability mass function of NH
    • P(Head) = 0.5 (fair coin)

N = 5

N = 10

N = 40

moments of the multivariate gaussian 1
Moments of the Multivariate Gaussian (1)

thanks to anti-symmetry of z

maximum likelihood
Maximum likelihood
  • Fit a probability density model p(x | θ) to the data
    • Estimate θ
  • Given independent identically distributed (i.i.d.) data X = (x1, x2, …, xN)
    • Likelihood
    • Log likelihood
  • Maximum likelihood: Maximize ln p(X | θ) w.r.t. θ
maximum likelihood for the gaussian 1
Maximum Likelihood for the Gaussian (1)

Given i.i.d. data , the log likelihood function is given by

Sufficient statistics

maximum likelihood for the gaussian 2
Maximum Likelihood for the Gaussian (2)

Set the derivative of the log likelihood function to zero,

and solve to obtain

Similarly

mixtures of gaussians 1
Mixtures of Gaussians (1)

Single Gaussian

Mixture of two Gaussians

Old Faithful data set

mixtures of gaussians 2
Mixtures of Gaussians (2)

Component

Mixing coefficient

K=3

Combine simple models into a complex model:

mixtures of gaussians 4
Mixtures of Gaussians (4)

Log of a sum; no closed form maximum.

Determining parameters ¹, §, and ¼ using maximum log likelihood

Solution: use standard, iterative, numeric optimization methods or the expectation maximization algorithm (Chapter 9).