slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Research Methods Psych 402 PowerPoint Presentation
Download Presentation
Research Methods Psych 402

Loading in 2 Seconds...

play fullscreen
1 / 89

Research Methods Psych 402 - PowerPoint PPT Presentation


  • 117 Views
  • Uploaded on

Research Methods Psych 402. LECTURE 4 COLLECT ASSIGNMENT #1 VHS – Sci Am #1 not cc ( Double blind controlled, placebo). Assignment 1: LIBRARY SEARCH. Number EEG papers on autism (autism, asperger, autistic) Number of sleep deprivation papers on high school kids

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Research Methods Psych 402' - trent


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Research MethodsPsych 402

  • LECTURE 4
  • COLLECT ASSIGNMENT #1
  • VHS – Sci Am #1 not cc
  • (Double blind controlled, placebo)
assignment 1 library search
Assignment 1: LIBRARY SEARCH
  • Number EEG papers on autism (autism, asperger, autistic)
  • Number of sleep deprivation papers on high school kids
  • Number of subjects in the one study on sleep deprivation, naps, and caffeine in high school kids
  • Number of papers published by Carl Sagan in English
  • Number of sleep papers published since January 1, 1980
  • Number of co-authored papers by DA Kaiser & MB Sterman
  • Maximum score on Autism Spectrum Quotient Test (AQ Test)
  • Maximum score on the AQ test a female has ever received
  • Size in grams of Immanuel Kant’s brain
  • Publication year of the one book in Wallace library on a feral/neglected child who was discovered in the 20th century

Use Medline for questions 1-5

PsycInfo for question 6

any source for remainder

slide4

STROOP EXPERIMENT

Independent variable (IV) and levels: INK WORD CONGRUENCE, incongruent (1), neutral (2)

Dependent variable (DV): response time to name all ink colors

Control variables (CV): screen, order of colors, same colors (stimulus properties), three columns, number of words, subject demographic and related variables, room

Random variables (RV): people in room, distractions, motivation/anxiety

Possible Confounding variables: order could have

slide5

STROOP EXPERIMENT

Independent variable (IV): Ink-word congruence

Dependent variable (DV): Time to name entire list

Control variables (CV): Room, experimenter, stimulus properties

Random variables (RV): distractors, motivation

Possible Confounding variables: practice, fatigue

slide6

RESEARCH VARIABLES

Independent variable (IV): Variable manipulated by researcher

Dependent variable (DV): Variable measured by researcher, expected to change as a result of the IV

Control variables (CV): Factors kept constant

Random variables (RV): Factors allowed to vary randomly.

Confounding variable: Factors that vary systematically with the levels of the IV, often present in one level and not another

slide7
Principle of Falsifiability

Idea is not a testable hypothesis unless it can be proven false

Freud’s Oedipal complex may not be falsifiable in most implementations of it (and thus not a theory or hypothesis, but dogma)

– “if you don’t admit fear of father, sexual love of mother you’re repressing it”

slide9

Research MethodsPsych 402

  • LECTURE 5 – BIAS
  • VHS Sci Am #2
slide10

RESEARCH VARIABLES

Independent variable (IV): Variable manipulated by researcher

Dependent variable (DV): Variable measured by researcher, expected to change as a result of researcher’s manipulation (“depends on IV”)

Control variables (CV): Factors kept constant (e.g., confederate, room).

Random variables (RV): Factors allowed to vary randomly.

(e.g. time of day effects, fatigue, motivation)

Confounding variable: Variable’s levels covary with IV’s and may be responsible for changes

source of experimental bias
Source of Experimental Bias
  • Subject Bias
    • Social desirability
    • Expectancy effects like Hawthorne
    • Conform to demand characteristics

Solution: single blind, placebo, clever design, deception (e.g., hostile masculinity design)

source of experimental bias1
Source of Experimental Bias
  • Experimenter Bias
    • Confirmation bias
    • Halo effects
    • Assignment (self selection, nonrandom)

Solution: double blind, random assignment, predetermined administrative procedures (e.g., tape record instructions), use experimenters who don’t know hypothesis or expected outcomes

SciAm video #2

incongruent more difficult
RED

GREEN

BLUE

BLUE

RED

BROWN

GREEN

RED

GREEN

BLUE

BROWN

Incongruent/more difficult

RED

GREEN

BLUE

GREEN

BROWN

RED

BLUE

GREEN

RED

GREEN

BROWN

BLUE

RED

RED

BLUE

BROWN

GREEN

RED

BLUE

GREEN

RED

BROWN

neutral easier
RAW

GRAND

BOLD

BOLD

RAW

BRAIN

GRAND

RAW

GRAND

BOLD

BRAIN

Neutral, easier

RAW

GRAND

BOLD

GRAND

BRAIN

RAW

BOLD

GRAND

RAW

GRAND

BRAIN

BOLD

RAW

RAW

BOLD

BRAIN

GRAND

RAW

BOLD

GRAND

RAW

BRAIN

single blind and double blind designs
Single-Blind and Double-Blind Designs
  • Breaking the blind of placebo research
  • In clinical trials, if treatment any good at all, experimenters figure out who got what
    • in depression study, patients receiving active treatment show mood elevation, increased energy, etc.)
  • AND participants can figure it out as well!
efficacy effectivenss studies
Efficacy & Effectivenss studies
  • Efficacy studies
    • Active vs Placebo condition where there is no change in behavior expected
      • (Placebo effect - spontaneous recovery - is around 30 % for numerous conditions)
  • Effectiveness study:
    • Active vs current best treatment

e.g., new drug vs Prozac in depression

More ethical

source of experimental bias2
Source of Experimental Bias
  • Measurement Bias
    • Ceiling & Floor effects
      • Data truncated at top or bottom, respectively, due to limits on possible scores.
      • E.g., 100% or 0% accuracy - 2 words v 1000
  • Choose realistic and reasonable levels.
    • If manipulation (IV) too strong (sledgehammer effects uninteresting), too weak (miss effect).
    • If measurement (DV) crude (miss effect), insensitive (trivial change).
slide18
Counterbalancing – arranging presentation levels of IV to eliminate confounding
      • Multiple orders, reverse orders
      • (not needed for between subject designs)
  • Control for “Items Effect”
slide19
Experimental research

1. Formulate a testable hypothesis

Translate idea into testable hypothesis.

Often expressed as predictions about how changes in one variable will be related to changes in another.

E.g., Violent media leads to aggressive behavior

Formulate and define predictions & variables precisely, unambiguously.

E.g., BDI for depression

slide20

OPERATIONAL DEFINITION

Specify operations required to manipulate or measure concept.

e.g. Anxiety is not vaguely described but explicitly measured by objectively through subjective report scales, behavior checkboxes, etc

beck depression inventory bdi

Twenty-one dimensions of depression:

1. Sadness                              12. Social withdrawal

2. Pessimism                           13. Indecisiveness

3. Sense of failure                   14 Change in body image

4. Dissatisfaction                     15. Retardation

5. Guilt                                   16. Insomnia

6. Expectation of punishment    17. Fatigability

7. Dislike of self                       18. Loss of appetite

8. Self Accusation                    19. Loss of Weight

9. Suicidal ideation                   20. Somatic preoccupation

10. Episodes of crying              21. Low level of energy

11. Agitation

In past week including today:

0 = I do not feel sad

1= I feel sad

2= I am sad all the time and I can’t snap out of it

3= I am so sad that I can’t stand it

0-63 max score (30+ indicates severe depression)

Beck Depression Inventory (BDI)

Aaron Beck

experiment 1
Experiment 1
  • My idea:
    • Spoken words (or ASL) are encoded better and less likely to be forgotten than written words
  • Precisely: Spoken list will lead to better verbal free recall of short list than only written list
slide23
2. Select research method and design

Decide how to test your hypothesis.

How will data be collected (e.g., survey, observation, experiment)?

Each method has advantages and disadvantages. Select most suitable and practical strategy.

Issue of control – too little (numerous potential confounds) vs too much (artificial)

mini experiment 1
Mini-Experiment 1
  • Experiment
  • Verbal list, 10 items each
  • Two types of encoding: spoken/ASL, written
  • Control for stimuli that could influence memory
mini experiment 1 5 letter nouns high frequency concrete familiar
Mini-Experiment 1: 5 letter nouns, high frequency, concrete, familiar

BOARD BREAK CHAIR CHILD CLOSE COAST COURT CROSS DEATH DRAMA DRINK EARTH EIGHT ESSAY FRONT GHOST GLASS GROUP HEART HOUSE LEVEL LIGHT LUNCH MONEY MONTH MOUTH MUSIC NIGHT NOVEL ORDER PAINT PAPER PARTY PHONE PIECE POINT QUIET RADIO RANGE RIGHT ROUND SCALE SCENE SENSE SHEER SHORT SIGHT SMALL SOUND SOUTH SPELL SPOKE STAFF STAND STATE STICK STILL STORY STYLE TABLE TASTE THEME THING THREE TOUCH TRAIN UNDER VOICE WASTE WATER WHITE WOMAN WORLD WRONG

MRC Psycholinguistic Database

slide26
3. Collect data – on whom?

Samples and populations

Ultimate goals – to generalize results

slide27
4. Analyze data and make inferences

Was hypothesis supported by data?

Descriptive statistics (averages, range) describe performance on your sample

Inferential statistics are used to infer likelihood of obtaining same results in any sample taken from the population.

slide28
5. Report findings

Scientific progress is achieved incrementally.

Share findings with scientists and public via peer-reviewed journals, or conferences, or books.

Peer review takes time (up to 1 year after submission), but allows other experts to evaluate and critique your work, occasionally revealing flaws or limitations of the study.

problems with peer review
Problems with peer review
  • Common denominator
  • Competitors review
  • Cognitive dissonance on reviewer part
    • No changes suggested means no reason for reviewing
mini experiment 11
Mini-Experiment 1
  • Class exercise
  • Left written; Right oral/ASL
  • Left side of room will go first
    • BLANK SCREEN for oral/ASL
left written
CHAIR

CHILD

DRINK

FRONT

GLASS

GROUP

HEART

LUNCH

NIGHT

PAPER

Left: written
right oral asl
WORLD

TRAIN

RADIO

SHORT

WHITE

WATER

SMALL

HOUSE

TASTE

BOARD

Right oral/ASL
mini experiment list
CHAIR

CHILD

DRINK

FRONT

GLASS

GROUP

HEART

LUNCH

NIGHT

PAPER

WORLD

TRAIN

RADIO

SHORT

WHITE

WATER

SMALL

HOUSE

TASTE

BOARD

Mini-Experiment List
slide34

RESEARCH VARIABLES

Independent variable (IV): Variable manipulated by researcher

Dependent variable (DV): Variable measured by researcher, expected to change as a result of researcher’s manipulation (“depends on IV”)

Control variables (CV): Factors kept constant (e.g., confederate, room).

Random variables (RV): Factors allowed to vary randomly.

(e.g. time of day effects, fatigue, motivation)

Confounding variable: Variable’s levels covary with IV’s and may be responsible for changes

mini experiment 12
Mini-Experiment 1
  • IV – presentation type, levels written or oral/ASL
  • DV – free recalled number of words
  • CV – room, experimenter, stimulus properties
  • RV – subject skills/memory differences, subject’s relationship to certain words, motivation differences, distractions
  • Confound: alphabetical in first list, others?
slide36

Research MethodsPsych 402

  • LECTURE 6
  • Measures:
  • Distortion, reliability & validity
  • VHS Zimbardo #19 Milgram
  • VHS 514 F (Framingham Heart & domestic violence)
elements of a research study
Elements of a Research Study

Hypothesis

  • Design
  • Participants
  • Measures – tools to quantify behavior
  • Analysis
issues with measures
Issues with Measures
  • Access: how easily data obtained

(e.g., Studying self-identity and gender – It would be well illuminated by responses by those undergoing sex-change operations)

  • Amount: how much data to collect (last year, decade)
  • Truncation: ceiling & floor effects

(e.g., aggression: violence is rare, displays of anger are not)

  • Distortion: how situation and behavior varies from real world (e.g., Milgram’s experiment)
  • Reliability: how consistent measure is
  • Validity: does tool measure intended phenomenon?
distortion ecological validity may not generalize to real world behavior
Distortion / Ecological Validity(May not generalize to real world behavior)

“Effect of Punishment on Memory”

“Learner” is confederate; mentions untrue mild heart condition.

Teacher (true subject) straps learner, samples 45 volt jolt

Examiner says shocks may be painful but no permanent damage

STANLEY MILGRAM

1933-1984

effect of punishment on memory
“Effect of Punishment on Memory”

Shocks on machine run from 15 (mild) to 450 volts (XXX) in steps of 15 volts

Learner’s script:

120 shout about pain

150 demand experiment stops

180 “no more pain!”

300 pounds on wall, screams

330 single cry then answers no more

Examiners prod subjects when they attempt to end experiment

“Learner” is an actor

milgram s experiment
Milgram’s Experiment

65% gave maximum shock!

reality of the experiment
Reality of the experiment

Subject is 50y unemployed man. After delivering 180-volt shock, he pivots around and addresses experimenter in agitated tones:

  • Subject: I can't stand it I'm not going to kill that man in there. You hear him hollering?
  • Experimenter: As I told you before, the shocks may be painful, but
  • S: But he's hollering. He can't stand it. What's going to happen to him?
  • E: The experiment requires that you continue…
  • S: I refuse to take the responsibility. He's in there hollering
  • E: It's absolutely essential that you continue….
  • S: All right.

(He continues until learner ”passes” out at 330 volts.)

  • S: I think something's happened to that fellow in there. I don't get no answer. He was hollering at less voltage. Can't you check in and see if he's all right, please?
  • E: Not once we've started. Please continue, Teacher.
  • S (sits down): "Cool day, shade, water, paint." Answer please. Are you all right in there? Are you all right,
  • E: Please continue, Teacher. Continue, please. (Subject pushes lever)
  • S: You accept all responsibility?
  • E: The responsibility is mine. Please go on. (Subject continues)
why so many people acted lethally
Why so many people acted lethally?

Coercive factors:

Yale grounds

Assured not dangerous by experimenters

…many others

Results may not generalize to other settings and circumstances

would milgram s effects occur in real life
Would Milgram’s effects occur in "real life"?

Hofling et al. (1966)

  • 22 nurses on night duty in psychiatric hospital in US
  • Unknown “doctor” phones nurse on duty and instructs her to give medication to a patient at twice maximum dosage. Maximum dosage of 10 mg shown on bottle label; he instructs her to give 20 mg. He says he’ll sign relevant authorization papers when he arrives in a few minutes.
  • Findings: 21/22 of the nurses obeyed telephone instruction and began to prepare dosage before they were stopped
  • However: Drug was unknown to nurses. When experiment repeated with a drug that the nurses had heard of, NONE obeyed.
further testing
Further testing
  • Rank and Jacobson (1977) carried out a similar study, but gave nurses opportunity to seek advice.
    • Only 2 out of 18 proceeded.
  • Meeus and Raaijmakers (1986)
    • Similar procedure to Milgram but in Holland with job applicants.
    • Had to deliver 15 ‘stress remarks’ to try and put off ‘interviewee’.
    • 22 out of 24 delivered all 15 despite the fact that at the time there was high unemployment in Holland.
issues with measures1
Issues with Measures
  • Access
  • Amount

Distortion

  • Truncation (ceiling or floor)
  • Reliability: how consistent measure is
  • Validity: does tool measure intended phenomenon?
validity of measure
Validity of measure
  • Does IQ measure Intelligence?
  • Does respond to movie violence reflect true response to real violence
  • Fear Factor Index of Courage

(e.g., time spent in coffin filled with worms)

reliability
Reliability
  • Does measurement instrument yield consistent results when the characteristic being measured hasn’t changed?
different types of reliability
Different types of reliability
  • Inter-rater reliability
  • Test-retest reliability
  • Internal reliability (Split-half)
  • Alternative forms reliability
inter rater reliability
Inter-rater reliability
  • Extent to which two or more individuals evaluate same product or performance give identical judgments.
internal reliability
Internal reliability
  • Extent to which all the items within a single instrument yield similar results.
  • More items  more consistency (generally)
alternative forms reliability
Alternative forms reliability
  • Extent to which two different versions of the same instrument yield similar results.
test retest reliability
Test-retest reliability
  • Extent to which the same instrument yields the same result on two different occasions.
correlation
Correlation
  • Correlation is a measure of the relation between two or more variables.
  • How well values for one variable predicts values for another variable
  • Ranges from -1 to +1

0: no predictability

+1 or -1 : perfect predictability

  • Test-retest should show high correlation if test is reliable
no predictability from one variable to the other
No predictability from one variable to the other
  • More about correlation when we go over surveys & observational research
reliability validity
Reliability & Validity
  • Reliability = consistency
  • Validity = measure what is intended
vhs 514 f outcome
VHS 514 F - Outcome
  • Sherman, L.W., Smith, D.A., Janell, D., & Rogan, D.P. (1992).Crime, punishment, and stake in conformity: Legal and informal control of domestic violence. American Sociological Review, 57, 680-690.
  • ABSTRACT:
  • Using police contacts for domestic violence offenses in Milwaukee, the authors tested whether the effect of arrest on recidivism is conditional on key individual characteristics indicating a "stake in conformity." Ss were 1,200 cases of misdemeanor domestic battery. 91% were male, 79% were Black, and 56% were unemployed. Contrary to deterrence theories, arrest had no overall crime reduction effect in either the official or victim interview measures of repeat domestic violence. Consistent with labeling theories, arrest increased recidivism among those with a low stake in conformity: the unemployed and the unmarried. Neither race nor a record of prior offenses conditioned the effect of arrest on subsequent domestic violence.
validity
Validity
  • Extent to which the instrument measures what it is supposed to measure.

scale->weight, clock->time,

thermometer->temperature,

barometer->air pressure

IQ test -> intelligence?

Time in coffin -> courage

validity is it true accurate
Validity = Is it true & accurate?
  • face validity (intuitively appears to measure intended phenomenon)
  • convergent validity (correlates with similar measures)
  • discriminant validity (doesn’t measure what it isn’t meant to)
  • predictive validity (ability to predict something it should be able to predict)
face validity
Face validity
  • On the surface, does instrument appear to measure a particular characteristic.
  • IQ test – ask vocabulary questions vs questions about favorite sport team
convergent validity
Convergent validity
  • Does instrument correlate with another, presumably related measure?
  • Intelligence reflected in GPA, trivia contest outcomes, income, graduation rate – does IQ correlate with these?
discriminant validity
Discriminant validity
  • Does instrument not correlate with unrelated measures?
  • IQ correlates weakly or not at all with language proficiency, test smarts, shoe size, etc
predictive validity
Predictive validity
  • Ability to predict something it should theoretically be able to predict.
  • IQ test correlates with future GPA?
internal validity
Internal validity
  • Study has sufficient contents (design, data) to ensure that our conclusions are truly warranted
    • Or defending against sources of bias which would affect the cause-effect process being studied by introducing covert variables
threat to internal validity or possible source of confounds
All studies

Experimenter bias

Subject bias

All studies with control groups

Selection bias

Mortality bias

Social interaction

Unintended treatment (Hawthorne effect)

Compensatory rivalry or resentful demoralization

Treatment imitation or diffusion

benefit from info given to the treatment group

Before/After studies

History: External events take place between initial and latter measurements that has an effect on participants

Maturation: Changes within participants during study

Test experience: Exposure alters future performance

Instrumentation: Unreliable measures or measurers

Regression to the Mean

Threat to Internal Validity, or possible source of confounds
subject mortality different rates of leaving each group possibly treatment dependent
Subject “mortality”(different rates of leaving each group, possibly treatment dependent)

% in

Treatment

Weeks

to ensure internal validity
To ensure internal validity

Eliminate other possible explanations for the results we observed

  • Controlled laboratory study - Regulate environmental conditions in a laboratory setting.
  • Double-blind experiment - Both subject and test-conductor are not aware of research hypothesis, assignment.
  • Unobtrusive measures - People do not know their performance is being monitored.
  • Triangulation - Multiple sources of data are collected with the hope that they all converge to support one hypothesis.
external validity
External validity
  • Conclusions drawn from data generalize to other contexts, real-world settings
to enhance external validity
To enhance external validity
  • Real-life setting

Tight controls of a laboratory setting may make results ungeneralizable – study phenomenon in the field.

  • Representative sample

Tested sample must be representative.

  • Replication in different context

Similar data or conclusions in dissimilar contexts suggests external validity

(e.g., Milgram’s replications).

slide79

Research MethodsPsych 402

  • LECTURE 7
  • Longitudinal Research
  • Or DVD 387 “42 up”
  • or VH 2729 “35 up”
pilot studies
Pilot studies
  • Pilot studies are initial attempts to study a phenomenon without sufficient numbers and/or controls to make firm conclusions about results
    • e.g., n=10, n=4, impure groups
    • Promising or unpromising only

Results from true (full) studies can confirmed or disconfirmed one’s hypothesis

outliers
Outliers
  • Eliminating subjects
    • Predetermined criteria of exclusion based on performance
slide83

DESIGN

Within-subject design

- each participant exposed to every level of IV (person acts as own control)

Between-subject design

- each participants exposed to one level of IV (and unaware of what other participants are exposed to)

within subject design
Advantages

Fewer subjects needed

Experimental time shorter

Subject acts as own control

Variability between conditions smaller

Disadvantages

Carryover effects

Counterbalancing corrections presume linear transfer between conditions

Stroop example

Incongruent/neutral

Neutral/Incongruent

Within-Subject Design
list 1
RED

GREEN

BLUE

BLUE

RED

BROWN

GREEN

RED

GREEN

BLUE

BROWN

List 1

RED

GREEN

BLUE

GREEN

BROWN

RED

BLUE

GREEN

RED

GREEN

BROWN

BLUE

RED

RED

BLUE

BROWN

GREEN

RED

BLUE

GREEN

RED

BROWN

slower when exposed to incongruent list first
RAW

GRAND

BOLD

BOLD

RAW

BRAIN

GRAND

RAW

GRAND

BOLD

BRAIN

Slower when exposed to incongruent list first

RAW

GRAND

BOLD

GRAND

BRAIN

RAW

BOLD

GRAND

RAW

GRAND

BRAIN

BOLD

RAW

RAW

BOLD

BRAIN

GRAND

RAW

BOLD

GRAND

RAW

BRAIN

between subjects design
Advantages

No carryover effects

No counterbalancing needed

Disadvantages

More subjects needed

More experimental time

Group differences on skills, motivation, and numerous other factors are possible

Matching subjects can reduce variability between conditions

Random assignment eliminates bias

Between-Subjects Design
longitudinal v cross sectional
Longitudinal v Cross Sectional
  • Longitudinal : same group compared at different times
    • (e.g., age 5 and at 10 and at 25 and at 40 years of old)
  • Cross sectional: different age groups compared at one time
    • (e.g., Jan 2004)
slide89
Cross-sectional study

Easier to do, collect data at one time.

Group differences

Cannot track how attitudes, etc., evolve over time (free of culture)

Longitudinal study

Harder to do

Practice effects

Longitudinal v Cross Sectional