Statistics workshop principles of hypothesis testing j term 2009 bert kritzer
This presentation is the property of its rightful owner.
Sponsored Links
1 / 30

Statistics Workshop Principles of Hypothesis Testing J-term 2009 Bert Kritzer PowerPoint PPT Presentation


  • 63 Views
  • Uploaded on
  • Presentation posted in: General

Statistics Workshop Principles of Hypothesis Testing J-term 2009 Bert Kritzer. Statistical Inference. Inference about populations from samples Inference about underlying processes Could the observed pattern been generated by a random process?

Download Presentation

Statistics Workshop Principles of Hypothesis Testing J-term 2009 Bert Kritzer

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Statistics workshop principles of hypothesis testing j term 2009 bert kritzer

Statistics Workshop Principles of Hypothesis TestingJ-term 2009Bert Kritzer


Statistical inference

Statistical Inference

  • Inference about populations from samples

  • Inference about underlying processes

    • Could the observed pattern been generated by a random process?

  • Inference about systematic vs. random (“stochastic”) components

    Observation = Systematic + Random

    • Sampling

    • Process

  • Observed statistics as random variables


Statistics as random variables

Statistics As Random Variables

μ = 0.564

mean of means = 0.566


Parameters statistics and estimators

Parameters, Statistics, and Estimators


Hypothesis testing

Hypothesis Testing

A procedure for drawing conclusions about characteristics in a population or about a process:

  • Is it reasonable to conclude that there is a relationship between two variables?

  • Is it reasonable to conclude that a populaton parameter exceeds some value?

  • Is it reasonable to conclude that the population parameter differs among two or more groups?


Hypothesis testing research hypotheses

Hypothesis TestingResearch Hypotheses

Are African-Americans more likely to be stopped by the police?

Is the average number of stops for African-Americans different than that for Whites?


Hypothesis testing null hypotheses

Hypothesis TestingNull Hypotheses

Are African-Americans more likely to be stopped by the police?

Is the average number of stops greater for African-Americans than for Whites?


Why the null hypothesis

Why the Null Hypothesis?

  • Can state the null hypothesis with precision, and that allows us to compute the probability of observing a particular result if the null hypothesis is true

  • Logically it is easier to ascertain what is untrue than what is true. If we can dichotomize the possibilities A and B, and then determine that A is not true, it must be the case that B is true.


Hypothesis testing and legal decisionmaking

Hypothesis Testing andLegal Decisionmaking

H0: InnocentHA: Guilty

Legal Decisionmaking:

Pr(Guilty|Evidence)

Hypothesis testing:

Pr(Evidence|Innocent)


Substantive vs statistical significance

Substantive vs. Statistical Significance

  • How big of a difference could be explained by random processes such as sampling?

    • Depends on sample size and characteristics of the underlying distribution

  • How big of a difference is enough that we should care about it?

    • Normative/policy question

    • Depends on “costs” associated with differences

  • Statistical significance refers only to the first of these


Significance level

“Significance Level”

  • How careful do you want to be in reaching a conclusion about your hypothesis?

  • Process will ask whether your null hypothesis can be rejected

  • What probability that you incorrectly rejected the null hypothesis are you willing to accept?

  • This probability is the significance level


Is the coin honest h 0 an honest coin

Is the Coin Honest?H0: An Honest Coin


Directionality

Directionality

  • Is the research hypothesis directional?

    • Do you think that the salaries of men are greater on average than the salaries of women?

    • Do you think that the salaries paid to African-Americans differ from the salaries paid to Whites?

    • Is the coin loaded vs. is the coin loaded toward heads?

  • “One-tailed” (directional) vs. “Two-tailed” (nondirectional) hypotheses

    • If directional hypothesis is correct, you are more likely to reject null hypothesis with one-tailed test


Types of error

TYPE I (α error): Rejecting a Null Hypothesis that is in fact true

Set by the “significance level”

TYPE II (β error): Failing to Reject a Null Hypothesis that is in fact false

Depends on the significance level, the sample size, and how wrong the null is

Types of Error


Steps in hypothesis testing

Steps in Hypothesis Testing

  • State research (“alternative”) hypothesis HA

  • State null hypothesis H0

  • Set decision rule

    • “level of significance” Pr(α error)

    • Directional or nondirectional (“one-tailed” or “two-tailed”) based on research hypothesis

  • Obtain data (set sample size)

  • Compute test statistic and get probability of observing it under H0 (“p-value”)

  • Make decision whether to reject H0


Hypothesis testing example

Hypothesis Testing Example


The concept of power

The Concept of Power

The probability that a hypothesis test will reject a false null hypothesis

β is the probability of a Type II error

1-β is the “power” of a significance test


What determines the power of a test

What Determines the Power of a Test?

  • The characteristics of a particular test

  • The sample size

  • The magnitude of the true effect (difference, regression coefficient, etc.) that we are trying to detect

200 feet vs. 500 feet

A chickadee vs. a bluejay


Power curves

Power Curves


Power curves one tailed

Power Curves-One Tailed


Is the coin honest

Is the Coin Honest?

  • If the coin is honest, the probability of a head on any one flip is .5.

  • Do we suspect that it is loaded one direction or the other?

  • How dishonest do we think it is (i.e., what is the actual probability of a head if that probability is not .5)?

  • How big is the sample (number of flips)?


What are the probabilities of different outcomes for an honest coin

What are the probabilities of different outcomes for an honest coin?

  • The binomial distribution

  • Two parameters are the probability and the “sample size” (number of flips of the coin)

  • Choosing the sample size will affect our ability to make a correct decision about the coin

    • The bigger the sample size the better

  • Knowing the “alternative hypothesis” (how dishonest the coin is) can help in deciding the sample size


Sample size of 12 h 0 an honest coin

Sample Size of 12H0: An Honest Coin


Other sample size options

Other Sample Size Options

Probability of Outcomes for an Honest

Probabilities shown are probabilities of a more extreme outcome


What is our research hypothesis

What Is Our Research Hypothesis?

  • Dishonest?

  • Loaded toward heads?


What significance level do we want to use

What Significance Level Do We Want to Use?

  • How willing are we to decide that the coin is dishonest (loaded toward heads?) when it is actually honest?

  • Some possibilities:

    .10

    .05

    .01

    .001


How big of a sample

How Big of a Sample?

  • How sure do we want to be able to reject the null hypothesis if the coin is in fact loaded toward heads?

    • Power

  • Do we have any idea of how dishonest the coin is?


Power curves 05 one tailed

Power Curves(.05 one-tailed)


15 flips

15 Flips

  • We will need 12 or more heads on our 15 flips to reject the null at .05 (one-tailed) level

  • According to previous chart, if coin has a true probability of heads of .67, we have a 20% of rejecting the null if our research hypothesis is that coin is loaded toward heads


Three frequently used methods of hypothesis testing

Three Frequently Used Methods of Hypothesis Testing

  • Direct comparison to hypothesized value

    • t-tests using t distribution

    • Z-tests using normal distribution

      • occasionally reported as a chi square

  • Comparisons to a set of hypothesized values based on a model

    • “Goodness-of-Fit” test using chi square

  • Reduction in predictive error

    • Analysis of Variance using F-ratio


  • Login