1 / 13

Consistency

Consistency. An estimator is a consistent estimator of θ, if , i.e., if converge in probability to θ. Theorem. An unbiased estimator for θ, is a consistent estimator of θ if Proof:. Example.

Download Presentation

Consistency

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Consistency • An estimator is a consistent estimator of θ, if , i.e., if converge in probability to θ. week 3

  2. Theorem • An unbiased estimator for θ, is a consistent estimator of θ if • Proof: week 3

  3. Example • Suppose X1, X2,…, Xn are i.i.d Poisson(λ). Let then… week 3

  4. Important comment • Consistency is an asymptotic property so we can have a consistent estimator that is biased as long as it is asymptotically unbiased. • Example: Uniform example above. week 3

  5. The Likelihood Function - Introduction • Recall: a statistical model for some data is a set of distributions, one of which corresponds to the true unknown distribution that produced the data. • The distribution fθ can be either a probability density function or a probability mass function. • The joint probability density function or probability mass function of iid random variables X1, …, Xn is week 3

  6. The Likelihood Function • Let x1, …, xn be sample observations taken on corresponding random variables X1, …, Xn whose distribution depends on a parameter θ. The likelihood function defined on the parameter space Ω is given by • Note that for the likelihood function we are fixing the data, x1,…, xn, and varying the value of the parameter. • The value L(θ | x1, …, xn) is called the likelihood of θ. It is the probability of observing the data values we observed given that θ is the true value of the parameter. It is not the probability of θ given that we observed x1, …, xn. week 3

  7. Examples • Suppose we toss a coin n = 10 times and observed 4 heads. With no knowledge whatsoever about the probability of getting a head on a single toss, the appropriate statistical model for the data is the Binomial(10, θ) model. The likelihood function is given by • Suppose X1, …, Xn is a random sample from an Exponential(θ) distribution. The likelihood function is week 3

  8. Sufficiency - Introduction • A statistic that summarizes all the information in the sample about the target parameter is called sufficient statistic. • An estimator is sufficient if we get as much information about θ from as we would from the entire sample X1, …, Xn. • A sufficient statistic T(x1, …, xn) for a model is any function of the data x1, …, xn such that once we know the value of T(x1, …, xn), then we can determine the likelihood function. week 3

  9. Sufficient Statistic • A sufficient statistic is a function T(x1, …, xn) defined on the sample space, such that whenever T(x1, …, xn) = T(y1, …, yn), then for some constant c. • Typically, T(x1, …, xn) will be of lower dimension than x1, …, xn, so we can consider replacing x1, …, xnby T(x1, …, xn) as a data reduction and this simplifies the analysis. • Example… week 3

  10. Minimal Sufficient Statistics • A minimal sufficient statisticT for s model is any sufficient statistic such that once we know a likelihood function L(θ|x1, …, xn) for the model and data then we can determine T(x1, …, xn). • A relevant likelihood function can always be obtained from the value of any sufficient statistic T, but if T is minimal sufficient as well, then we can also obtain the value of T from any likelihood function. • It can be shown that a minimal sufficient statistics gives the maximal reduction of the data. • Example… week 3

  11. Alternative Definition of Sufficient Statistic • Let X1, …, Xn be a random sample from a distribution with unknown parameter θ. The statistic T(x1, …, xn) is said to be sufficient for θ if the conditional distribution of X1, …, Xn given T does not depend on θ. • This definition is much harder to work with as the conditional distribution of the sample X1, …, Xn given the sufficient statistics T is often hard to derive. week 3

  12. Factorization Theorem • Let T be a statistic based on a random sample X1, …, Xn. Then T is a sufficient statistic for θ if i.e. if the likelihood function can be factored into two nonnegative functions one that depend on T(x1, …, xn) and θ and one that depend only on the data x1, …, xn. • Proof: week 3

  13. Examples week 3

More Related