Reliability • Is the measure able to give consistent measures over time?
Reliability checks? • Test-retest • Split-half • Inter-observer • Convergence…replications
VALIDITY- Does your study reveal accurate and representative results? • Internal Validity and External Validity • External Validity- generalizability • Issues • Sampling • Return rates • …..
…External Validity • Generalizeability • Across population • Across environments • Across time
Convergent validity • Different manipulations • Different measuring devices • Different populations…etc • Replication
COMMON THREATS TO VALIDITY • Selection bias • random selection and assignment • Pretesting and matching • Recruitment technique
History and Maturation Effects • Changes in the DV over time • Between pretest and post-test • Or between test sessions • constancy
Mortality • Subject loss Over time • May reflect unintended effect of IV or DV • Subject loss may result in selection bias
STOP FOR FUN! • ESP in PSYCHOLOGY?? • Somewhat critical • Lacking empirical validation • Should we remain open-minded? • Let’s give it a shot
What is Chance Performance? -1 out of 4 for each trial… -20 trials Chance performance = 5
Statistical Regression • Regression to the mean with repeated testing • Watch out for extreme scores..they may be due to chance
Instrumentation • Changes in DV due to changes in measurement, not due to IV
Sequence Effects-repeated testing effects • Practice effects • Test sensitization • Boredom/fatigue • Carry-over • Randomization and counterbalancing
Subject Bias • Subjects can have attitude! RESPONSE SETS that produce inaccurate results • Soc desireability • Cooperitivity • Negativity • Volunteerism • -blindin-recruitment -deception or disguise lie detectors
Experimentor Bias • Experimentors like to be correct! CONFIRMATIONAL BIAS • Demand characteristics • constancy • Automation • Double blinding
PLACEBO EFFECTS ARE REAL • Expectation • What might someone expect MJ to do? • Beliefs/expectations can lead to real results….that are not related to the real effects of the IV.
DOUBLE-BLIND PLACEBO CONTROL DESIGNS • Fake MJ?/ Ciggarette/ Pill other • Constancy • Can tease apart placebo effects from real drug effects.
RANGE EFFECTS • Don’t forget the inability to see an affect of an IV on a DV does not necessarily mean there isn’t one. • Ceiling effects • Basement effects
ULTIMATE CONTROL • Control procedures may be taylored for unique experiments • Ultimate control may never be completely achieved
BEING A REAL PSYCHOLOGICAL SCIENTIST?? • The crux- identification of potential confounds • AND • RIVAL HYPOTHESIS
RIVAL HYPOTHESIS • Alternative explanations for experimental results • Alternative to the research hypothesis
Consider the following Hampton Court Maze
Training of rats on the RAM • Initial freezing • Adaptation and exploration • Neophobia to food pellet • Adaptation to food • Rapid acquisition of task
RAM memory errors? • Entrance into previously entered arm • One foot? Two? Half-way? All the way?
Rats become quite good • Rarely make a memory error • Win-shift strategy
Maybe they take an algorhythmic strategy • Right-right-right etc
Using forced-choice procedures • No indication of algorhythm • Performance still very good
SO… • What does rat performance in RAM show us?
Rival Hypothesis? • Memory or sensory guidance? • Intra-maze or extramaze guidance? • Spatial memory or…..?