1 / 24

STATISTICAL INFERENCE PART III

STATISTICAL INFERENCE PART III. EXPONENTIAL FAMILY, FISHER INFORMATION, CRAMER-RAO LOWER BOUND (CRLB). EXPONENTIAL CLASS OF PDFS. X is a continuous (discrete) rv with pdf f(x; ),  . If the pdf can be written in the following form.

leia
Download Presentation

STATISTICAL INFERENCE PART III

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. STATISTICAL INFERENCEPART III EXPONENTIAL FAMILY, FISHER INFORMATION, CRAMER-RAO LOWER BOUND (CRLB)

  2. EXPONENTIAL CLASS OF PDFS • X is a continuous (discrete) rv with pdf f(x;), . If the pdf can be written in the following form then, the pdf is a member of exponential class of pdfs of the continuous (discrete) type. (Here, k is the number of parameters)

  3. REGULAR CASE OF THE EXPONENTIAL CLASS OF PDFS • We have a regular case of the exponential class of pdfs of the continuous type if • Range of X does not depend on. • c() ≥ 0, w1(),…, wk() are real valued functions of  for . • h(x) ≥ 0, t1(x),…, tk(x) are real valued functions of x. If the range of X depends on , then it is called irregular exponential class or range-dependent exponential class.

  4. EXAMPLES X~Bin(n,p), where n is known. Is this pdf a member of exponential class of pdfs? Why? Binomial family is a member of exponential family of distributions.

  5. EXAMPLES X~Cauchy(1,). Is this pdf a member of exponential class of pdfs? Why? Cauchy is not a member of exponential family.

  6. EXPONENTIAL CLASS and CSS • Random Sample from Regular Exponential Class is a css for . If Y is an UE of , Yis the UMVUE of .

  7. EXAMPLES Let X1,X2,…~Bin(1,p), i.e., Ber(p). This family is a member of exponential family of distributions. is a CSS for p. is UE of p and a function of CSS. is UMVUE of p.

  8. EXAMPLES X~N(,2) where both  and 2 is unknown. Find a css for  and 2 .

  9. FISHER INFORMATION AND INFORMATION CRITERIA • X, f(x;), , xA (not depend on ). Definitions and notations:

  10. FISHER INFORMATION AND INFORMATION CRITERIA The Fisher Information in a random variable X: The Fisher Information in the random sample: Let’s prove the equalities above.

  11. FISHER INFORMATION AND INFORMATION CRITERIA

  12. FISHER INFORMATION AND INFORMATION CRITERIA

  13. FISHER INFORMATION AND INFORMATION CRITERIA The Fisher Information in a random variable X: The Fisher Information in the random sample: Proof of the last equality is available on Casella & Berger (1990), pg. 310-311.

  14. CRAMER-RAO LOWER BOUND (CRLB) • Let X1,X2,…,Xnbe sample random variables. • Range of X does not depend on . • Y=U(X1,X2,…,Xn): a statistic; does’nt contain . • Let E(Y)=m(). • Let prove this!

  15. CRAMER-RAO LOWER BOUND (CRLB) • -1Corr(Y,Z)1 • 0 Corr(Y,Z)21   • Take Z=′(x1,x2,…,xn;) • Then, E(Z)=0 and V(Z)=In() (from previous slides).

  16. CRAMER-RAO LOWER BOUND (CRLB) • Cov(Y,Z)=E(YZ)-E(Y)E(Z)=E(YZ)

  17. CRAMER-RAO LOWER BOUND (CRLB) • E(Y.Z)=mʹ(), Cov(Y,Z)=mʹ(), V(Z)=In() The Cramer-Rao Inequality (Information Inequality)

  18. CRAMER-RAO LOWER BOUND (CRLB) • CRLB is the lower bound for the variance of an unbiased estimator of m(). • When V(Y)=CRLB, Y is the MVUE of m(). • For a r.s., remember that In()=n I(), so,

  19. LIMITING DISTRIBUTION OF MLEs • : MLE of  • X1,X2,…,Xnis a random sample.

  20. EFFICIENT ESTIMATOR • T is an efficient estimator (EE) of  if • T is UE of , and, • Var(T)=CRLB • Y is an efficient estimator (EE) of its expectation, m(), if its variance reaches the CRLB. • An EE of m() may not exist. • The EE of m(), if exists, is unique. • The EE of m() is the unique MVUE of m().

  21. ASYMPTOTIC EFFICIENT ESTIMATOR • Y is an asymptotic EE of m() if

  22. EXAMPLES A r.s. of size n from X~Poi(θ). • Find CRLB for any UE of θ. • Find UMVUE of θ. • Find an EE for θ. • Find CRLB for any UE of exp{-2θ}. Assume n=1, and show that is UMVUE of exp{-2θ}. Is this a reasonable estimator?

  23. EXAMPLE A r.s. of size n from X~Exp(). Find UMVUE of , if exists.

  24. Summary • We covered 3 methods for finding UMVUE: • Rao-Blackwell Theorem (Use a ss T, an UE U, and create a new statistic by E(U|T)) • Lehmann-Scheffe Theorem (Use a css T which is also UE) • Cramer-Rao Lower Bound (Find an UE with variance=CRLB)

More Related