1 / 18

Hypothesis Testing: Understanding the Steps and Significance Levels

Learn about the steps involved in hypothesis testing and the significance levels used to make decisions. Explore the importance of choosing the appropriate test statistic in hypothesis testing.

olen
Download Presentation

Hypothesis Testing: Understanding the Steps and Significance Levels

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Quantitative Methods – Week 7: Inductive Statistics II: Hypothesis Testing Roman Studer Nuffield College roman.studer@nuffield.ox.ac.uk

  2. A standard normal distribution is a normal distribution N(0,1) with mean =0 and standard deviation =1 95% of cases =1 2,5% of cases 2,5% of cases with -1,96 0 +1,96 Introduction In last week’s class on inductive statistics, we learned how to determine and interpret confidence intervals... • Explain!

  3. Introduction (II) … where the t-value tells us how many standard errors either side of the mean we have to add to achieve a certain degree of confidence (90%, 95%, 99%) E.g. 95% confidence level: • With confidence intervals, the implicit hypothesis was that the specific intervals contains a certain value • With hypothesis testing, the hypothesis is now made explicit; so we explicitly want to know whether… • Can the result from one sample be considered different from the result of another sample? Example: Is the mean relief payment in Kent different from the one in Sussex? • Is there really a connection between one variable and another one? Example: Is the regression coefficient on unemployment statistically significant and can therefore help to explain the voting share of the Nazi party? Why do we actually need hypothesis testing ??

  4. The 5 Steps of Hypothesis Testing Specify the hypothesis in an appropriate form for statistical testing Set a level of probability on the basis of which the hypothesis should be rejected Select the relevant test statistic Calculate the test statistic and compare this calculated value with the tabulated theoretical probability distribution in order to reach a decision about the hypothesis Interpret the results of the decision

  5. Step 1: Null and Alternative Hypothesis For testing a hypothesis, the first step is set up a null hypothesis (H0) that can be rejected The null hypothesis is presumed to be true until the data strongly suggests otherwise (like a defendant on trial) The alternative hypothesis H1 specifies the opposite Examples: • H0: The defendant is innocent H1: The defendant is guilty • H0: Unemployment rates had no impact on Nazi votes H1: Unemployment had an impact on Nazi votes • H0: bUnemployment=0 H1: bUnemployment≠0

  6. Step 2: The Level of Probability Type I and Type II error • In hypothesis testing we can make two kinds of mistakes • Type I error: Rejecting the null hypothesis when it is in fact true • Type II error: Failing to reject the null hypothesis when it is actually false • What is the risk we are willing to take of making a Type I error?

  7. Step 2: The Level of Probability (II) • The Significance level, a, is the probability of making a Type I error •  A small probability of a type I error is preferred • To what extent we are willing to take a risk of making wrong conclusion? Common choices for a are… • 10% • 5% (most common) • 1% • 5% level means that we are taking a risk of being wrong five times per 100 trials • Trade-off: If we reduce the probability of a type I error, the probability for a type II error will increase

  8. Step 3: The Test Statistic • What is the probability, that H0 is true given the observed outcome? • For every sample statistic there is a corresponding sampling distribution • Test statistics and critical values in order to test a null hypothesis against an alternative • For the time being, the test statistic we are using is the t-statistic CAUTION! • Always remember the underlying assumptions when testing: • The distribution is approximately normal • Our sample doesn’t suffer from a serious sample bias

  9. 2nd sample y 1st sample b<b x Step 4: Calculate and Assess the Test Statistic Test statistic for a regression coefficient If we could repeat the “social” experiment, we would obtain different values for the error term and consequently for the dependent variable  impact on the slope coefficient How reliable is it to conclude that there is a relationship? Example:

  10. Step 4: Calculate and Assess the Test Statistic (II) Is b significantly different from 0? H0: b=0  t-statistics The nominator b is the regression coefficient derived from the OLS method The denominator corresponds to the estimated standard deviation of the sampling distribution of the coefficient, i.e. the SE(b)

  11. Step 4: Calculate and Assess the Test Statistic (III) Statistical significance using the t-statistics • The t-statistic indicates how many standard deviations the sample regression coefficient is from 0 (we can also express it as deviation from some certain value of interest if we want) • Central Limit Theorem applies • Shape of the sampling distribution becomes normal • With increasing sample size the t-distribution approximates a standard normal distribution  If t-value >1.96, the estimated regression coefficient is more than 1.96 standard deviations from 0 and the probability for such an outcome is less than 5%, if H0 is true

  12. Step 4: Calculate and Assess the Test Statistic (IV) Sampling distribution and critical value 95% of cases 2.5% of cases 2.5% of cases -1.96 +1.96 Reject H0 Accept H0 Reject H0 Confidence level a=0.95

  13. Step 4: Calculate and Assess the Test Statistic (V) Significance level and p-level P-value is the probability that the outcome observed would be present if the null hypothesis is true Small p-values are evidence against H0 P-value<a  Reject H0 , accept H1 Failing to reject the null hypothesis does not necessarily constitute support for H0; it just means that the data or pattern is not sufficiently strong to reject H0

  14. Step 4: Calculate and Assess the Test Statistic (VI) One- and two-tailed tests Two-tailed test: The parameter value is calculated for both tails of the sampling distribution  The critical region is divided equally between the left- and the right-hand tails If the hypothesis is about the directions: One-tailed test: H0: μ1 = μ2; H1: μ1 > μ2 Critical region is in the left- or the right-hand tail  a one-tailed test increases the critical region at one tail Strong a priori reasons must exist!

  15. Step 5: Interpreting the Results Statistical versus historical significance • Statistical significance refers to the probability of type I error (rejecting the null hypothesis when it is in fact true) • Statistical significance is influenced by • magnitude of the parameter • magnitude of the standard error, i.e. sample size (because the standard error decreases with N everything else being equal) • Historical significance: What is the practical significance of rejecting the hypothesis?  The aim is to find statistically significant results that are relevant from a historical perspective.

  16. Computer Class: • Repetition & Hypothesis Testing

  17. Exercises Relief dataset Get the “Relief” dataset at http://www.nuff.ox.ac.uk/users/studer/teaching.htm • Note: In the column “county”, 1 stands for Kent and 2 stands for Sussex. • Look at the dataset: what is the basic unit of measurement here? Is it a time-series, a cross-sectional or a panel dataset? • Look at the variables relief and unemployment (unemp) • Get a first overview by visualising the data: Are the variables approximately normally distributed? • Compute the mean, median, standard deviation, coefficient of variation, kurtosis and skewness for the variables • Are the level of relief and the level of employment statistically different in Kent and Sussex? (Hint: create new variables and test for difference of means.) What is the null hypothesis? Are the means different at a 95% significance level? • Work again with the whole dataset, not just with Kent and Sussex. Compute the correlation coefficient between unemployment and the relief payments. How do you interpret the result? Is the correlation coefficient significant at a 95% and at a 99% level? State the null hypothesis, and test it • Estimate a regression using relief and unemployment. Which one is the dependent, which one the independent variable? Report the results of the regression (a, b, R2). Is the regression coefficient (b) statistically significant at a 95% level? Formulate the null hypothesis and test it. What is the critical t-value shown in the t-statistics table? What is the actual t-value of b? What is the p-value? What does all that mean for the interpretation of the regression results?

  18. Homework • Readings: • Feinstein & Thomas, Ch. 8 & 9 • Problem Set 5: • Finish the exercises from today’s computer class if you haven’t done so already. Include all the results and answers in the file you send me. • Do exercise 1 from Feinstein & Thomas, p. 181.

More Related