1 / 22

STATISTICAL INFERENCE PART I POINT ESTIMATION

STATISTICAL INFERENCE PART I POINT ESTIMATION. . . (. ). STATISTICAL INFERENCE. Determining certain unknown properties of a probability distribution on the basis of a sample (usually, a r.s.) obtained from that distribution. Point Estimation:. Interval Estimation:. Hypothesis Testing:.

udell
Download Presentation

STATISTICAL INFERENCE PART I POINT ESTIMATION

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. STATISTICAL INFERENCEPART IPOINT ESTIMATION

  2.  ( ) STATISTICAL INFERENCE • Determining certain unknown properties of a probability distribution on the basis of a sample (usually, a r.s.) obtained from that distribution Point Estimation: Interval Estimation: Hypothesis Testing:

  3. The family of pdfs { f(x; ),  } STATISTICAL INFERENCE • Parameter Space ( or ): The set of all possible values of an unknown parameter, ; . • A pdf with unknown parameter: f(x; ), . • Estimation: Where in ,  is likely to be?

  4. Estimator of an unknown parameter : A statistic used for estimating . STATISTICAL INFERENCE • Statistic: A function of rvs (usually a sample rvs in an estimation) which does not contain any unknown parameters. An observed value

  5. POINT ESTIMATION • θ: a parameter of interest; unknown • Goal: Find good estimator(s) for θ or its function g(θ).

  6. METHODS OF ESTIMATION Method of Moments Estimation, Maximum Likelihood Estimation

  7. METHOD OF MOMENTS ESTIMATION (MME) • Let X1, X2,…,Xn be a r.s. from a population with pmf or pdf f(x;1, 2,…, k). The MMEs are found by equating the first k population moments to corresponding sample moments and solving the resulting system of equations. Population Moments Sample Moments

  8. METHOD OF MOMENTS ESTIMATION (MME) so on… Continue this until there are enough equations to solve for the unknown parameters.

  9. EXAMPLES • Let X~Exp(). • For a r.s of size n, find the MME of . • For the following sample (assuming it is from Exp()), find the estimate of : 11.37, 3, 0.15, 4.27, 2.56, 0.59.

  10. EXAMPLES • Let X~N(μ,σ²). For a r.s of size n, find the MMEs of μ and σ². • For the following sample (assuming it is from N(μ,σ²)), find the estimates of μ and σ²: 4.93, 6.82, 3.12, 7.57, 3.04, 4.98, 4.62, 4.84, 2.95, 4.22

  11. DRAWBACKS OF MMES • Although sometimes parameters are positive valued, MMEs can be negative. • If moments does not exist, we cannot find MMEs.

  12. MAXIMUM LIKELIHOOD ESTIMATION (MLE) • Let X1, X2,…,Xn be a r.s. from a population with pmf or pdf f(x;1, 2,…, k), the likelihood function is defined by

  13. MAXIMUM LIKELIHOOD ESTIMATION (MLE) • For each sample point (x1,…,xn), let be a parameter value at which L(1,…, k| x1,…,xn) attains its maximum as a function of (1,…, k), with (x1,…,xn) held fixed. A maximum likelihood estimator (MLE) of parameters (1,…, k) based on a sample (X1,…,Xn) is • The MLE is the parameter point for which the observed sample is most likely.

  14. EXAMPLES • Let X~Bin(n,p), where both n and p are unknown. One observation on X is available, and it is known that n is either 2 or 3 and p=1/2 or 1/3. Our objective is to estimate the pair (n,p).

  15. MAXIMUM LIKELIHOOD ESTIMATION (MLE) • It is usually convenient to work with the logarithm of the likelihood function. • Suppose that f(x;1, 2,…, k) is a positive, differentiable function of 1, 2,…, k. If a supremum exists, it must satisfy the likelihood equations • MLE occurring at boundary of  cannot be obtained by differentiation. So, use inspection.

  16. MLE • Moreover, you need to check that you are in fact maximizing the log-likelihood (or likelihood) by checking that the second derivative is negative.

  17. EXAMPLES 1. X~Exp(), >0. For a r.s of size n, find the MLE of .

  18. EXAMPLES 2. X~N(,2). For a r.s. of size n, find the MLEs of  and 2.

  19. EXAMPLES 3. X~Uniform(0,), >0. For a r.s of size n, find the MLE of .

  20. Example:X~N(,2). For a r.s. of size n, the MLE of  is . By the invariance property of MLE, the MLE of 2is INVARIANCE PROPERTY OF THE MLE • If is the MLE of , then for any function (), the MLE of () is .

  21. ADVANTAGES OF MLE • Often yields good estimates, especially for large sample size. • Invariance property of MLEs • Asymptotic distribution of MLE is Normal. • Most widely used estimation technique. • Usually they are consistent estimators. [will define consistency later]

  22. DISADVANTAGES OF MLE • Requires that the pdf or pmf is known except the value of parameters. • MLE may not exist or may not be unique. • MLE may not be obtained explicitly (numerical or search methods may be required.). It is sensitive to the choice of starting values when using numerical estimation.  • MLEs can be heavily biased for small samples.

More Related