The Independent Samples t-Test

1. A Classic! The Independent Samples t-Test PG-17 Feared by Graduate Students Everywhere!

2. Independent Samples • Random Selection: • Everyone from the Specified Population has an Equal Probability • Of being Selected for the study (Yeah Right!) • Random Assignment: • Every participant has an Equal Probability of being in the Treatment • Or Control Groups

3. The Null Hypothesis • Both groups from Same Population • No Treatment Effect • Both Sample Means estimate Same Population Mean • Difference in Sample Means reflect Errors of Estimation of Mu • X-Bar1 + e1 = Mu (Mu – X-Bar1 = e1) • X-Bar2 + e2 = Mu (Mu – X-Bar2 = e2) • Errors are Random and hence Unrelated

4. Expectation • If Both Samples were selected from the Same Population: • How much should the Sample Means Disagree about Mu? • X-Bar1 – X-Bar2 • Errors of Estimation decrease with N • Errors of Estimation increase with Population Heterogeneity

5. The Expected Disagreement • The Standard Error of a Difference: • SEX-Bar1-X-Bar2 • The Average Difference between two Sample Means • The Expected Difference between two Sample Means • When they are Estimating the Same Mu • 68% chance of this much Or Less • 95% chance of (this much x 2) Or Less • Actually this much x 1.96, if you know sigma • Rounded up to 2

6. Expectation: The Standard Error of the Difference The Expected Disagreement between two Sample Means (if H0 true) • SEM for Control Group  SEM for Treatment Group T for Treatment Group C for Control Group Add the Errors and take the Square Root

7. Evaluation • Compare the Difference you Got to the Difference you would Expect • If H0 true What you Got ? What you Expect df = n1 + n2 - 2

8. Evaluation • Compare the Difference you Got to the Difference you would Expect • If H0 true What you Got a) If they agree: Keep H0 ? • If they disagree: Reject H0 Is TOO DAMN BIG! What you Expect

9. Burn This!

10. Power The ability to find a relationship when it exists • Errors of Estimation and Standard Errors of the Difference decrease with N • Use the Largest sample sizes possible • Errors of Estimation increase with Population Heterogeneity • Run all your subjects under Identical Conditions (Experimental Control)

11. Power • What if your data look like this? • Everybody increased their score (X-bar1 – X-Bar2), • but heterogeneity among subjects (SEM1 & SEM2) is large

12. Power • Correlated Samples Designs: • Natural Pairs: E.G.: Father vs. Son • Measuring liberal attitudes • Matched Pairs: Matching pairs of students on I.Q. • One of each pair gets treatment (e.g., teaching with technology • Repeated Measures: • Measure Same Subject Twice (e.g., Pre-, Post-therapy) • Look at differences between Pairs of Data Points, ignoring Between • Subject differences

13. Correlated Samples Smaller denominator Makes t bigger, hence More Power • Same as  usual • Minus strength  of Correlation If r=0, denominator is the same, but df is smaller

14. Effect Size A weighted average of Two estimates of Sigma • What are the Two Ts of Research? • What is better than computing Effect Size?

15. Confidence Interval Use 2-tailed t-value at 95% confidence level With N1 + N2 –2 df  Best Estimate  N-1 df Does the Interval cross Zero?

21. Assumptions of the t-Test • Both (if more than one) population(s): • Normally distributed • Equal variance • Violations of Assumptions: • Robust unless gross • Transform scores (e.g. take log of each score)

22. Power • Power = 1 – Beta • Theoretical (Beta usually unknown) • Reject H0: • Decision is clear, you have a relationship • Fail to reject H0: • Decision is unclear, you may have failed to find a Relationship • due to lack of Power

23. Power • Increases with Effect Size (Mu1 – Mu2) • Increases with Sample Size • If close to p<0.05 add N • Decreases with Standard Error of the Difference (denominator) • Minimize by • Recording data correctly • Use consistent criteria • Maintain consistent experimental conditions (control) • (Increasing N)