research design in clinical psychology l.
Skip this Video
Loading SlideShow in 5 Seconds..
Research Design in Clinical Psychology PowerPoint Presentation
Download Presentation
Research Design in Clinical Psychology

Loading in 2 Seconds...

play fullscreen
1 / 19

Research Design in Clinical Psychology - PowerPoint PPT Presentation

  • Uploaded on

Research Design in Clinical Psychology. Lecture 2 Reliability, Validity, and Artifact/Bias (Chapters 2-4 in Kazdin). Internal Validity. To what extent can the intervention, and not other factors, account for study results History Maturation Testing (Practice) Instrumentation

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

Research Design in Clinical Psychology

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
    Presentation Transcript
    1. Research Design in Clinical Psychology Lecture 2 Reliability, Validity, and Artifact/Bias (Chapters 2-4 in Kazdin)

    2. Internal Validity • To what extent can the intervention, and not other factors, account for study results • History • Maturation • Testing (Practice) • Instrumentation • Statistical Regression • Selection Bias • Attrition • these can occur across all groups or only to select groups • Diffusion/Imitation of treatment • Special treatment/reaction of controls

    3. External Validity • To what extent can the study results be generalized to other samples with different characteristics than the study sample

    4. External Validity: Examples 1 • Sample characteristics • Differences b/n study sample and other samples • Include age, gender, culture, education • Stimulus characteristics and settings • Extension across characteristics of the study • Includes setting, experimenter, materials/apparatus • Ecological validity? • Reactivity of experimental arrangements • Awareness of being in a study may affect behavior • Are responses relevant to those who are not in a study • Multiple-treatment interference • Subjects receives multiple interventions • Relevant to those who did not receive other interventions?

    5. External Validity: Examples 1 • Novelty • Is effect due to newness • Reactivity of assessment • Similar to experimental arrangements, but focuses on awareness of what the measures are tapping • Test sensitization • Does pre-testing or the test itself (in the case of post-test sensitization) alter subject experience and responses • Timing of Measurement • When assessments are given could alter results

    6. Parsimony • Least complex explanations first • Frequently, a threat to internal validity is most parsimonious • When considering limitations in both internal and external validity, however, parsimony suggests that “findings are the best statement of a relationship, unless there are clear reasons to think otherwise.

    7. Why internal validity precedes external validity • Can’t have ExtVal, without IntVal • It would be like asking, “gee can we generalize these results we have no confidence in to a wide variety of individuals?” • One can still have important findings that elucidate basic principles without much ExtVal. • A lack of generalization of a finding across samples may be very important and can span other research

    8. Construct Validity • What is the causal agent and conceptual basis underlying an effect (what is the intervention and why did it lead to change?) • In clinical research methods, CV is different from in test construction where CV is the extent that a measure captures a construct of interest.

    9. Threats to construct validity • Contact time • Placebo effects • Nonblind (single and double) designs • Single operations and narrow stimulus sampling • Is the effect due to the selected IV • Does other aspects of the intervention have an effect beyond the aspect identified by the experimenter • Expectancies • Cues and Demand characteristics • Others exist based on conceptual relevance

    10. Statistical Conclusion ValidityAKA Data Evaluation Validity • Refers to the facets of the quantitative evaluation that influence the conclusions reached about experimental conditions and effects (to what extent are “real” effects demonstrated and interpreted) • Does one understand the stats used • Has one done stats correctly

    11. Rejecting the Null Hypothesis I • Alpha = probability of rejecting the null when you shouldn’t (Type 1 error) • Saying groups are different when in reality are same • Beta = probability of accepting the null when you shouldn’t (Type 2 error) • Saying groups are same when in reality are different • Power = probability of correctly accepting or rejecting the null

    12. Rejecting the Null Hypothesis II • Standard deviation = variability around mean • sqrt [(each observation – mean)2 / (N -1)] • SS/df • Effect size = Beyond significance, it is the magnitude of difference b/n groups • Expressed in terms of SD units • ES = (M1 – M2) / SD

    13. Threats to statistical conclusion validity • Low power • Inability to detect real differences • Variability in the procedures • Subjects in same group get different treatment • Subject heterogeneity • Subjects in same group differ on potentially confounding variables • Unreliability of measures • Multiple comparisons/error rates • Experiment-wise error

    14. Experimental Precision • Controlling vs holding constant • Tradeoffs

    15. Sources of Artifacts and Bias

    16. Rationale, scripts, & procedures • Imprecision in carrying out procedures • Experimenter may not have clear a protocol • Loose protocol effect • Experimenter may ignore protocol • Strategies for overcoming include • developing a clear protocol • actually testing the protocol out ahead of time • Get feedback from participants • Train experimenters together • Document all deviations

    17. Experimenter Expectancy • Experimenters’ knowledge of study hypotheses may bias study implementation • Strategies for overcoming include • Keep experimenters naïve to purpose • Double blind procedures for group assignment • Success of these strategies can be empirically reviewed

    18. Sample selection • Convenience samples and Volunteers • Do these sample differ in ways that affect the results or their generalizability • May select certain types of convenience or volunteer samples to answer particular questions • Attrition • Difference in # of dropouts across groups could have huge effect on results • Reminders and Commitment tactics (backloading $$$) • Select those unlikely to drop-out? • Plan ahead from past research

    19. Other sources • Experimenter characteristics • Situational and context cues • Demand characteristics – clues on how to respond • Subject roles (pg 95 in book) • Data recording and analysis