1 / 29

Hypothesis Testing

In hypothesis testing, we compare observed results to the null hypothesis to determine if there is a significant relationship. This process involves the risk of two types of errors: Type I and Type II errors. The level of significance regulates the risk of making a Type I error, while the sample size affects the probability of a Type II error. Statistical power is the ability to detect significant relationships. This outline provides an overview of hypothesis testing, including steps and considerations.

charleskelly
Download Presentation

Hypothesis Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Hypothesis Testing

  2. Outline • The Null Hypothesis • Type I and Type II Error • Using Statistics to test the Null Hypothesis • The Logic of Data Analysis

  3. Research Questions and Hypotheses • Research question: • Non-directional: • No stated expectation about outcome • Example: • Do men and women differ in terms of conversational memory? • Hypothesis: • Statement of expected relationship • Directionality of relationship • Example: • Women will have greater conversational memory than men

  4. Grounding Hypotheses in Theory • Hypotheses have an underlying rationale: • Logical reasoning behind the direction of the hypotheses (theoretical rationale – explanation) • Why do we expect women to have better conversational memory? • Theoretical rationale based on: • 1. Past research • 2. Existing theory • 3. Logical reasoning

  5. The Null Hypothesis • Null Hypothesis - the absence of a relationship • E..g., There is no difference between men’s and women’s with regards to conversational memories • Compare observed results to Null Hypothesis • How different are the results from the null hypothesis? • We do not propose a null hypothesis as research hypothesis - need very large sample size / power • Used as point of contrast for testing

  6. Hypotheses testing • When we test observed results against null: • We can make two decisions: • 1. Accept the null • No significant relationship • Observed results similar to the Null Hypothesis • 2. Reject the null • Significant relationship • Observed results different from the Null Hypothesis • Whichever decision, we risk making an error

  7. Type I and Type II Error • 1. Type I Error • Reality: No relationship • Decision: Reject the null • Believe your research hypothesis have received support when in fact you should have disconfirmed it • Analogy: Find an innocent man guilty of a crime • 2. Type II Error • Reality: Relationship • Decision: Accept the null • Believe your research hypothesis has not received support when in fact you should have rejected the null. • Analogy: Find a guilty man innocent of a crime

  8. Potential outcomes of testing Decision Accept NullReject Null R E No A Relationship L I Relationship T Y 1 2 3 4

  9. Potential outcomes of testing Decision Accept NullReject Null R E No A Relationship L I Relationship T Y Correct decision 2 3 4

  10. Potential outcomes of testing Decision Accept NullReject Null R E No A Relationship L I Relationship T Y 1 2 3 Correct decision

  11. Potential outcomes of testing Decision Accept NullReject Null R E No A Relationship L I Relationship T Y 1 Type I Error 3 4

  12. Potential outcomes of testing Decision Accept NullReject Null R E No A Relationship L I Relationship T Y 1 2 4 Type II Error

  13. Potential outcomes of testing Decision Accept NullReject Null R E No A Relationship L I Relationship T Y Correct decision Type I Error Correct decision Type II Error

  14. Function of Statistical Tests • Statistical tests determine: • Accept or Reject the Null Hypothesis • Based on probability of making a Type I error • Observed results compared to the results expected by the Null Hypotheses • What is the probability of getting observed results if Null Hypothesis were true? • If results would occur less than 5% of the time by simple chance then we reject the Null Hypothesis

  15. Start by setting level of risk of making a Type I Error • How dangerous is it to make a Type I Error: • What risk is acceptable?: • 5%? • 1%? • .1%? • Smaller percentages are more conservative in guarding against a Type I Error • Level of acceptable risk is called “Significance level” : • Usually the cutoff - <.05

  16. Conventional Significance Levels • .05 level (5% chance of Type I Error) • .01 level (1% chance of Type I Error) • .001 level (.1% chance of Type I Error) • Rejecting the Null at the .05 level means: • Taking a 5% risk of making a Type I Error

  17. Steps in Hypothesis Testing • 1. State research hypothesis • 2. State null hypothesis • 3.Set significance level (e.g., .05 level) • 4. Observe results • 5. Statistics calculate probability of results if null hypothesis were true • 6. If probability of observed results is less than significance level, then reject the null

  18. Guarding against Type I Error • Significance level regulates Type I Error • Conservative standards reduce Type I Error: • .01 instead of .05, especially with large sample • Reducing the probability of Type I Error: • Increases the probability of Type II Error • Sample size regulates Type II Error • The larger the sample, the lower the probability of Type II Error occurring in conservative testing

  19. Statistical Power • The power to detect significant relationships • The larger the sample size, the more power • The larger the sample size, the lower the probability of Type II Error • Power = 1 – probability of Type II Error

  20. Statistical Analysis • Statistical analysis: • Examines observed data • Calculates the probability that the results could occur by chance (I.e., if Null was true) • Choice of statistical test depends on: • Level of measurement of the variables in question: • Nominal, Ordinal, Interval or Ratio

  21. Logic of data analysis • Univariate analysis • One variable at a time (descriptive) • Bivariate analysis • Two variables at a time (testing relationships) • Multivariate analysis • More than two variables at a time (testing relationships and controlling for other variables)

  22. Variables • Dependent variable: • What we are trying to predict • E.g., Candidate preference • Independent variables: • What we are using as predictors • E.g., Gender, Party affiliation

  23. Testing hypothesis for two nominal variables Variables Null hypothesis Procedure Gender Passing is not Chi-square related to gender Pass/Fail

  24. Testing hypothesis for one nominal and one ratio variable Variables Null hypothesis Procedure Gender Score is not T-test related to gender Test score

  25. Testing hypothesis for one nominal and one ratio variable Variable Null hypothesis Procedure Year in school Score is not related to year in ANOVA school Test score • Can be used when nominal variable has more than two categories and can include more than one independent variable

  26. Testing hypothesis for two ratio variables Variable Null hypothesis Procedure Hours spent studying Score is not related to hours Correlation spent studying Test score

  27. Testing hypothesis for more than two ratio variables Variable Null hypothesis Procedure Hours spent studying Score is not positively related to hours Classes spent studying and Multiple missed not negatively related regression to classes missed Test score

  28. Commonality across all statistical analysis procedures • Set the significance level: • E.g., .05 level • Means that we are willing to conclude that there is a relationship if: • Chance of Type I error is less than 5% • Statistical tests tell us whether: • The observed relationship has less than a 5% chance of occurring by chance

  29. Summary of Statistical Procedures

More Related