Lecture 7-8

1. Lecture 7-8 Validity and Introduction to Inferential Statistics

2. Probability Measure Selection Chain of Reasoning for Inferential Statistics Sample Population Inference

3. Hypothesis Testing

4. A statement about what findings areexpected and are used for inferentialstatistics null hypothesis statements of valueless statistically significant relationships of differences"the two groups will not differ“ alternative hypothesis statements of statistically significant relationships of differences"group A will do better than group B""group A and B will not perform the same" Hypothesis

5. Null is True Null is False Correct Decision Accept Error Type II Error Correct Decision Error Reject Type I Error Possible Outcomes in Hypothesis Testing Type I Error: Rejecting a True Hypothesis Type II Error: Accepting a False Hypothesis

6. Probability Statistical probability is the odds that what we observed in the sample did not occur because of error (random and/or systematic). In other words, the probability associated with a statistic is the level of confidence we have that the sample group that we measured actually represents the total population.

7. Two Tail 2.5% 2.5% 5% region of rejection of null hypothesis Non directional

8. One Tail 5% 5% region of rejection of null hypothesis Directional

9. Null is True Null is False Correct Decision Accept Error Type II Error Correct Decision Error Reject Type I Error Possible Outcomes in Hypothesis Testing Type I Error: Rejecting a True Hypothesis Type II Error: Accepting a False Hypothesis

10. Probability Measure (results) Selection Chain of Reasoning for Inferential Statistics Sample Population Inference

11. When we conduct a study and obtain our RESULTS, we would like to have some confidence that our conclusions are the most plausible explanations for the results we observed

12. confounding variables Relationship of Variables independent variables dependent variables presumed or possible cause presumed results

13. Confounding VariablesThreats to Internal Validity Intervening Variables • Independent variables that have not been or can not be controlled or measured directly Extraneous Variables • those uncontrolled variables (not manipulated by the experimenter) that mayhave a significant influence upon the results of the study

14. The extent that we can be confident in our conclusions is related to the degree of internal validity we have established in our study

15. Judging Internal Validity Explanation credibility • is the study reasonable? • problem, lit review, purpose & hypotheses

16. Judging Internal Validity Translation fidelity • are the operational definitions reasonable • 5 Ws and an H • Who (subjects) • Where (situation) • Why (treatment (the cause)) • What (measurement (the effect)) • How (method of evaluation) • When (procedure)

17. Judging Internal Validity Demonstrated Result • Authenticity of the evidence • Precedence of cause • Presence of effect • Congruence of explanation and evidence

18. Judging Internal Validity Rival explanations eliminated • There are always rival explanations, it is up to the researcher to anticipate as many as possible and design a study that eliminates or minimizes the most threatening • use of comparison groups (control groups) • randomization

19. Judging Internal Validity • Credible Result • the sum of the previous 4 judgements • includes an evaluation of this studies findings in relation to the literature • consistent with previous research • inconsistent ----> why

20. History Maturation Selection Instrumentation Mortality Cross-Sectional Longitudinal Matching Subject Effect Valid Data (self report) Regression to the mean Correlation - Causality Experimenter Effect Instability Observer/rater Effects Order Effects Sampling Bias Statistical Testing Treatment Confound Classical Threats to Internal Validity

21. History When a group of subjects is measured before and after exposure to some treatment, pretest-posttest change (or lack of) can be attributed to something other than the treatment that took place outside of the confines of the study between the pre and post measurements

22. Maturation Changes in the organism (person) over the course of the study may influence the outcome measure

23. Instrumentation Sometimes a measuring instrument’s ability to yield accurate information systematically changes over time, like when norms become obsolete

24. Mortality If subjects drop out of a one-group pretest-posttest design, or there is differential rates of (and reasons) attrition in multi-group designs, conclusions about group differences can be misleading

25. Selection When two or more groups receive different treatments and there is a failure to randomly assign subject to groups, treatment differences might me confounded with initial group differences

26. Cross-Sectional / Longitudinal Attempts to identify developmental trends by studying different age groups (cross-sectional) and not by studying one age group over an extended time period (longitudinal)

27. Matching The attempt to create equivalent groups at the start of a study by selecting specific variable and making sure they are equally distributed in all your groups

28. Subject Effect The subjects in particular group figure out what your study is about and alter their behavior (consciously or unconsciously)

29. Valid Data / Self-Report When a subject (consciously or unconsciously) reports false data.

30. Regression To The Mean If a group of subject are preselected to receive a treatment because they represent an extreme group and the pretest has a correlation lower than +1.00 with the posttest, the preselected group will be less “extreme” on the posttest regardless of the treatment intervention B. Applegate

31. Correlation / Causality Drawing a causal conclusion when the design is not an “experimental design”

32. Experimenter Effect When there are different experimenters and they differentially treat the different experimental groups, or when one experimenter changes the way they treat (react to) a group

33. Instability Sample statistics are estimates of population values (parameters) and thus often have limited generalizability

34. Observer/Rater Effects Two people viewing the same thing will often see and report different things

35. Order Effects Can happen when multiple measuring instruments or treatments are administered and the outcome response is partially dependent on the specific order of presentation.

36. Sampling Bias The use of non-representative (non random) samples and trying to generalize back to the population

37. Statistical(other than instability and regression) The use of the wrong statistical test, or violation of the assumptions underlying the statistical test used

38. Testing People tend to become “more normal” on subsequent testing

39. Treatment Confound When characteristics in the study or experimenter characteristics are confounded with the treatment administration

40. Inferential Statistics • Relationship • bivariate • correlation • multivariate • Regression • Group comparison • IV, DV • t-test • ANOVA • Estimation

41. Relationship

42. Pearson Correlation • Direction of relationship (linear) • positive or negative • Magnitude of relationship (-1.0 to +1.0) • pencil • cigar • football • watermelon • ball

46. Nonlinear Relationship

47. Correlation Interpretation

48. Common Correlations • Pearson Product Moment Correlation • both variables are continuous • Spearman Rank-order Correlation • both variables are measured as rank data • Biserial Correlation • one variable is continuous and one is an ‘artificial’ dichotomy with an underlying normal distribution • Point-Biserial Correlation • one variable is continuous and one is a ‘true’ dichotomy • Phi Coefficient • both variables are ‘true’ dichotomies

49. More Correlations • Tetrachoric Correlation • both variables are ‘artificial’ dichotomies with underlying normal distributions • Polychoric Correlation • both variables are ordinally measured with both having underlying normal distributions • Polyserial Correlation (rps, Dps) • one variable is continuous and one is ordinal with an underlying normal distribution • Kendall Tau-b • measures agreement between two rankings • Kendall’s Coefficient of Concordance • measures of the extent to which members of a set of m distinct rank orderings of N things tend to be similar