maximum likelihood estimates and the em algorithms ii l.
Skip this Video
Loading SlideShow in 5 Seconds..
Maximum Likelihood Estimates and the EM Algorithms II PowerPoint Presentation
Download Presentation
Maximum Likelihood Estimates and the EM Algorithms II

Loading in 2 Seconds...

play fullscreen
1 / 69

Maximum Likelihood Estimates and the EM Algorithms II - PowerPoint PPT Presentation

  • Uploaded on

Maximum Likelihood Estimates and the EM Algorithms II. Henry Horng-Shing Lu Institute of Statistics National Chiao Tung University Part 1 Computation Tools. Include Functions in R. source( “ file path ” ) Example

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Maximum Likelihood Estimates and the EM Algorithms II' - mckile

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
maximum likelihood estimates and the em algorithms ii

Maximum Likelihood Estimates and the EM Algorithms II

Henry Horng-Shing Lu

Institute of Statistics

National Chiao Tung University

include functions in r
Include Functions in R
  • source(“file path”)
  • Example
    • In MME.R:
    • In R:
example 1 in genetics 1
Example 1 in Genetics (1)

Two linked loci with alleles A and a, and B and b

A, B: dominant

a, b: recessive

A double heterozygote AaBb will produce gametes of four types: AB, Ab, aB, ab

















F ( Female) 1- r’ r’ (female recombination fraction)

M (Male) 1-r r (male recombination fraction)


example 1 in genetics 2
Example 1 in Genetics (2)

r and r’ are the recombination rates for male and female

Suppose the parental origin of these heterozygote is from the mating of . The problem is to estimate r and r’ from the offspring of selfed heterozygotes.

Fisher, R. A. and Balmukand, B. (1928). The estimation of linkage from the offspring of selfed heterozygotes. Journal of Genetics, 20, 79–92.


example 1 in genetics 4
Example 1 in Genetics (4)

Four distinct phenotypes: A*B*, A*b*, a*B* and a*b*.

A*: the dominant phenotype from (Aa, AA, aA).

a*: the recessive phenotype from aa.

B*: the dominant phenotype from (Bb, BB, bB).

b* : the recessive phenotype from bb.

A*B*: 9 gametic combinations.

A*b*: 3 gametic combinations.

a*B*: 3 gametic combinations.

a*b*: 1 gametic combination.

Total: 16 combinations.


example 1 in genetics 6
Example 1 in Genetics (6)

Hence, the random sample of n from the offspring of selfed heterozygotes will follow a multinomial distribution:


example 1 in genetics 7
Example 1 in Genetics (7)

Suppose that we observe the data of

y = (y1, y2, y3, y4) = (125, 18, 20, 24),

which is a random sample from

Then the probability mass function is


maximum likelihood estimate mle
Maximum Likelihood Estimate (MLE)


Maximize likelihood: Solve the score equations, which are setting the first derivates of likelihood to be zeros.

Under regular conditions, the MLE is consistent, asymptotic efficient and normal!



mle for example 1 1
MLE for Example 1 (1)





mle for example 1 2
MLE for Example 1 (2)






a banach space
A Banach Space
  • A Banach space B is a vector space over the field K such that
    • Every Cauchy sequence of B converges in B (i.e., B is complete).


lipschitz continuous
Lipschitz Continuous
  • A closed subset and mapping
    • F is Lipschitz continuous on A with if .
    • F is a contraction mapping on A if F is Lipschitz continuous and


fixed point theorem
Fixed Point Theorem
  • If F is a contraction mapping on A if F is Lipschitz continuous and
    • F has an unique fixed point such that
    • initial , k=1,2,…



parallel chord method 1
Parallel Chord Method (1)
  • Parallel chord method is also called simple iteration.
define functions for example 1 in r
Define Functions for Example 1 in R

We will define some functions and variables for finding the MLE in Example 1 by R

newton raphson method 1
Newton-Raphson Method (1)
halley s method
Halley’s Method
  • The Newton-Raphson iteration function is
  • It is possible to speed up convergence byusing more expansion terms than the Newton-Raphson method does when the object function is very smooth, like the method by Edmond Halley (1656-1742):


bisection method 1
Bisection Method (1)
  • Assume that and that there exists a number such that . If and have opposite signs, and represents the sequence of midpoints generated by the bisection process, thenand the sequence converges to r.
  • That is, .

( )

secant method
Secant Method

( )

( )

secant bracket method





Secant-Bracket Method
  • The secant-bracket method is also called the regular falsi method.
fisher scoring method
Fisher Scoring Method
  • Fisher scoring method replaces by where is the Fisher information matrix when the parameter may be multivariate.
order of convergence
Order of Convergence
  • Order of convergence is p if

and c<1 for p=1.



Hence, we can use regression to estimate p.

theorem for newton raphson method
Theorem for Newton-Raphson Method
  • If , F is a contraction mapping then p=1 and
  • If exists, has a simple zero, then such that of the Newton-Raphson method is a contraction mapping and p=2.
find convergence order by r 1
Find Convergence Order by R (1)

R=Newton(y1, y2, y3, y4, initial)

#Newton method can be substitute for different method






Write your own programs for those examples presented in this talk.

Write programs for those examples mentioned at the following web page:

Write programs for the other examples that you know.


more exercises 1
More Exercises (1)

Example 3 in genetics: The observed data are (nO, nA, nB, nAB) = (176, 182, 60, 17) ~ Multinomial(r^2, p^2+2pr, q^2+2qr, 2pq), where p, q, and r fall in [0,1] such that p+q+r = 1. Find the MLEs for p, q, and r.


more exercises 2
More Exercises (2)

Example 4 in the positron emission tomography (PET):The observed data are n*(d) ~Poisson(λ*(d)), d = 1, 2, …, D, and

The values of p(b,d) are known and the unknown parameters are λ(b), b = 1, 2, …, B.

Find the MLEs for λ(b), b = 1, 2, …, B.



more exercises 3
More Exercises (3)

Example 5 in the normal mixture:The observed data xi, i = 1, 2, …, n, are random samples from the following probability density function:

Find the MLEs for the following parameters: