revised 9 11 13
Download
Skip this Video
Download Presentation
Revised 9/11/13

Loading in 2 Seconds...

play fullscreen
1 / 76

Revised 9/11/13 - PowerPoint PPT Presentation


  • 240 Views
  • Uploaded on

Psychology 242, Dr. McKirnan. Revised 9/11/13. Right click for “full Screen” or “end show”. Left click to proceed, . Lectures 3: Developing research questions and hypotheses. Basic experimental design & internal validity . Research questions, hypotheses & designs.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Revised 9/11/13' - doyle


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
revised 9 11 13

Psychology 242, Dr. McKirnan

Revised 9/11/13

Right click for “full Screen” or “end show”. Left click to proceed,

Lectures 3:

Developing research questions and hypotheses.

Basic experimental design & internal validity.

Week 3; Experimental designs

research questions hypotheses designs
Research questions, hypotheses & designs
  • How do social values affect science?
  • Where do research questions & hypotheses come from.
  • Variables in research
  • Basic experimental designs
  • True vs. Quasi- Experiments

Phenomenon

Theory

Hypothesis

Methods

& data

Results

Discussion &

Conclusions

Week 3; Experimental designs

the big picture introductory lectures
The big picture: Introductory lectures

  • How do social values affect science?
  • Where do research questions & hypotheses come from.
  • Variables in research
  • Basic experimental designs
  • True vs. Quasi- Experiments

Phenomenon

Theory

Hypothesis

Methods

& data

Results

Discussion &

Conclusions

Week 3; Experimental designs

social values
Social Values…

Phenomenon

…affect what we choose as our research question…

Theory

Hypothesis

Methods

& data

Specific methods are more standard and objective

Results

Discussion &

Conclusions

…and our conclusions

Week 3; Experimental designs

values theory and data in the scientific process
Values, theory and data in the scientific process.

Social values help define a scientific “problem” or question.

Norms, values (& data) determine what is credible / fundable.

Phenomenon

Theory

Theory is influenced by norms + empirical background of field.

Hypothesis

Science hinges on clear, objectively stated hypotheses.

Clear hypotheses lessen bias in interpreting results.

Methods

& data

Methods & analyses are most objective, but fields vary in methodological rigor.

Results

Discussion &

Conclusions

  • The “meaning” of a finding is influenced by cultural & social values or concerns.

…for science and, particularly, for society.

Week 3; Experimental designs

values and science the internet and sexual risk
Values and science: The internet and sexual risk.

Social values help define a scientific “problem” or question.

Norms, values (& data) determine what is credible / fundable.

Phenomenon

Theory

Sexual internet use became a large issue for gay/bisexual men in the 2000s

  • Internet use exploded, and became a sexual venue.
  • Men who have sex with Men [MSM] seemed particularly likely to be risky via internet sex.
  • The HIV prevention field needed to understand how this worked.
  • HIV made research on MSM a legitimate and important topic

Hypothesis

Methods

& data

Results

Discussion &

Conclusions

Week 3; Experimental designs

values theory and data in the scientific process1
Values, theory and data in the scientific process.

Phenomenon

Theory

Theory is influenced by values & empirical background of field.

Hypothesis

Several theories may help us explain internet  sexual risk:

  • Theories of depression: a Ψ factor within the person may interfere with his decision making.
  • Social structure: The social environment of MSM may influence internet use & risk.

Taking an individual v. social perspective can be a value choice.

Methods

& data

Results

Discussion &

Conclusions

Week 3; Experimental designs

values and science climate change 1
Values and science: climate change, 1.

Phenomenon

Theory

Hypothesis

Science hinges on clear, objectively stated hypotheses.

Clear hypotheses lessen bias in interpreting results.

Methods

& data

  • Statistical tests of “mediation” can test relatively clear tests of hypotheses about depression and sexual risk

Results

Internet +sex

Depression

Risk

Discussion &

Conclusions

Week 3; Experimental designs

values theory and data in the scientific process2
Values, theory and data in the scientific process.

Widely shared conventions make value judgments less important. However:

  • Behavioral sciences vary considerably in their rigor.
  • Choosing, e.g., quantitative vs. qualitative research can be a value choice.
  • Fields such as literary criticism, history or feminist studies may use substantially different methods.

Phenomenon

Theory

Hypothesis

Methods

& data

Methods & analyses are most objective, but fields vary in methodological rigor.

Results

Discussion &

Conclusions

Week 3; Experimental designs

values theory and data in the scientific process3
Values, theory and data in the scientific process.

No scientific finding can completely answer a research question;

  • Its meaning is affected by existing theory and empirical findings.
  • Society may not be “ready” for certain findings: e.g., data showing that GLBT people are not “mentally ill” were rejected for many years.
  • Many important findings – e.g., from economics – have little affect on social policy if they contradict a widely shared ideology.

Phenomenon

Theory

Hypothesis

Methods

& data

Results

Discussion &

Conclusions

  • The “meaning” of a finding is influenced by cultural & social values or concerns.

…for science and, particularly, for society.

Week 3; Experimental designs

the big picture introductory lectures1
The big picture: Introductory lectures
  • How do social values affect science?
  • Where do research questions & hypotheses come from.
  • Variables in research
  • Basic experimental designs
  • True vs. Quasi- Experiments

Phenomenon

Theory

Hypothesis

Methods

& data

Results

Discussion &

Conclusions

Week 3; Experimental designs

r esearch questions
Research questions

Where do research questions come from?

  • Practical questions
  • Unanswered questions from previous research
  • Testing theories.

Week 3; Experimental designs

sources of research questions
Sources of research questions

Practical / applied questions

  • Describe or explain an important social process
  • Evaluate an intervention or policy change

Chicago school closings and reassignment are predicted to result in better scores.

Do they over 5 years of follow-up?

E X A MP L E

Week 3; Experimental designs

sources of research questions1
Sources of research questions

Unanswered questions from previous research

  • Clarify conflicting / unclear findings
      • Who actually uses the internet to meet people…
  • Do previous findings generalize to…
    • …different groups
      • Many Social Psychology studies enroll middle(+) classWhite female undergraduates in research labs.
    • …different research areas
      • Can interventions to increase healthy behavior generalize to recycling and energy conservation?
    • …different research approaches
      • Do controlled lab studies generalize to less controlled field research?

Practical / applied questions

Week 3; Experimental designs

theories
Theories

Practical / applied questions

  • Use existing theory to explain a new phenomenon
    • “Sensation seeking” personality is associated with drug use. Might it also explain unsafe sex in adolescents or gay men?
  • Test contrasting theories of a phenomenon
    • How much is adolescent problem behavior controlled by psychological variables (depression) vs. peer influence?
  • Develop new / expanded theory
    • We discriminate among very subtle differences in smell. Might olfactory cues affect who we are attracted to?

Unanswered questions from previous research

Testing theories

Week 3; Experimental designs

slide16

Theories

How do we go from a research question to an actual study?

  • Phenomenon
  • Theory
  • Hypothesis
  • Methods

Key Terms:

  • Hypothetical Construct
  • Operational Definition

Week 3; Experimental designs

the research process
The research process

Phenomenon

Overall issue or question;

What controls emotional states?

Why are some people vulnerable to depression?

Theory

Possible explanation: “How it works” statement

Several theories may help explain the phenomenon

Theory 1

Emotional stability requires secure emotional attachments.

Theory 2

Some brains are genetically disposed to serotonin depletion during stress

Week 3; Experimental designs

research process 2
Research process, 2

Theory 1

Emotional attachment  emotional stability.

A theory can lead to several hypotheses

Hypothesis 1

Fewer parent – child interactions  vulnerability to depression.

Hypothesis 2

Emotional support during stress  less depression

A given hypotheses can be tested in several ways

Methods 1

Survey measurement: assess # of “family meals” per week, correlate it with self-reported depression.

Methods 2

Experiment: ½ have structured parent / child interactions, ½ do not, induce stress to both groups, assess depression

Week 3; Experimental designs

research process 3
Research process, 3

Theory 1

Emotional attachment  emotional stability.

Hypothesis 1

Family interactions  depression.

Hypothesis 2

(Non)Support + stress  depression

Some hypotheses are best tested in a measurement approach, and some with experimental designs

Best tested by a measurementstudy

  • Family interactions are difficult to bring into the lab,
  • Possible ethical problems.

Can be tested in an experiment:

Both support and stress can be controlled or manipulated in the lab.

Week 3; Experimental designs

research process the big picture
Research process: The Big Picture

Phenomenon

Big picture question.

Theory 1

Possible explanation, invoking one set of hypothetical constructs.

Theory 2

Alternate explanation, invoking other hypothetical constructs.

Hypothesis 1

A prediction that logically flows from – and tests – the theory.

Hypothesis 2

Another prediction that tests the same theory.

Methods 1

Operationally define the variables & test the hypothesis.

Methods 2

An alternate operational definition & way of testing the hypothesis.

Week 3; Experimental designs

question 1
Question 1

A hypothetical construct is…

A = A specific prediction about the outcome of an experiment

B = A little known band from Muncie Indiana

C = A general ψ process that underlies our observations

D = A central element in a theory

Week 3; Experimental designs

question 2
Question 2

To be testable, a hypothesis…

Must rest on operational definitions.

A = true

B = False

C = I don’t know

Week 3; Experimental designs

question 21
Question 2

An operational definition is

A = The procedure(s) we use to measure a study variable

B = The way we define our theory

C = The procedure(s) we use to manipulate a study variable

D = What we use to derive our hypothesis

Week 3; Experimental designs

question 3
Question 3

To be testable, a hypothesis…

Must potentially be found to be false.

A = true

B = false

C = I don’t know

Week 3; Experimental designs

question 11
Question 1

A theory…

A = Leads to one specific hypothesis

B = May be one of several ways to explain something

C = Is not as important as simply collecting data

D = Is what you make up to explain why you forgot your boy/girl friend’s birthday

E = Is not really affected by social or personal values

Week 3; Experimental designs

the big picture introductory lectures2
The big picture: Introductory lectures
  • How do social values affect science?
  • Where do research questions & hypotheses come from.
  • Variables in research
  • Basic experimental designs
  • True vs. Quasi- Experiments

Phenomenon

Theory

Hypothesis

Methods

& data

Results

Discussion &

Conclusions

Week 3; Experimental designs

slide27

Theories

Types and uses of variables in research:

  • Experiments
    • Independent vs. Dependent variables
  • Measurement / field studies
    • “Predictor” vs. “Criterion”
  • Forming variables:
    • Direct manipulation
    • Indirect manipulation
    • Measurement

Week 3; Experimental designs

types of variables experimental designs
Types of Variables:Experimental designs

Independent VariableDependent Variable

Defines the “contrast space”

Models the phenomenon

What is compared to what

e.g., drug v. placebo

  • What is being explained;

e.g., task performance

Hypothetical “cause”

  • “Effect”

Imposed by researcher as experimental conditions

  • Measured

as the outcome

Categorical

  • Continuous

Week 3; Experimental designs

types of variables measurement studies
Types of Variables:Measurement Studies

PredictorCriterionorOutcome

Defines the “contrast space”

Models the phenomenon

What predicts the outcome?

e.g., age, education, gender

  • What is being explained;

e.g., political attitude

Hypothetical “Cause”

  • “Effect”

Measured

not imposed by researcher

  • Measured

as the outcome

Continuous

  • Continuous

Week 3; Experimental designs

slide30

Experimental v. Measurement designs

Experimental designs

  • Manipulating the Independent Variable:
    • Enhances internal validity
    • May lessen external validity
  • Participants randomly assigned to experimental v. control groups

Measurement (or correlational)designs

  • Measurement “in the field”
    • May enhance external validity
    • Typically lessens internal validity
  • Sampling very important

Week 3; Experimental designs

creating independent variables ivs
Creating independent variables [IVs]

1. Direct experimental manipulation

  • Most typical of “true” experiments
  • Maximum control over IV

2. Indirect manipulation via experimental or research conditions

  • Less direct control over IV

3. Quasi-Independent variables: forming groups using a measured variable.

  • Experiments without complete control over variables
  • Used in measurement studies

Week 3; Experimental designs

forming variables
Forming Variables

1.Direct experimental manipulations

  • Drug or biomedical intervention
  • Behavioral intervention or treatment
  • Focused experimental study of, e.g., specific stimuli
  • System-wide “treatment” (e.g., policy change, school-based)
    • Simple presence v. absence of the treatment or stimulus
  • Single v. multiple treatment doses
  • Type of treatment or stimulus
  • Structurethe IV

vis-à-vis:

Week 3; Experimental designs

direct experimental manipulation
Direct experimental manipulation
  • Hypothesis: words presented in a semantic context are recalled better than when presented randomly.

Independent Variable

Dependent Variable

Experimental group

Target words presented within complete sentences

Word recognition task

Control group

Target words presented randomly

Word recognition task

Completely controlled by the experimenter

Experimental manipulation same as Independent Var.

Week 3; Experimental designs

forming variables1
Forming Variables

2.Indirect experimental manipulations

  • Experimental “induction” of a mood or state…
    • “Stage manage” a social event 
    • Induce mood via description of the experiment

Stress require presentation in front of peers

Depression  Write about worst mistake you ever made

Stereotype threat  “This test reflects on your group”

Anxiety  Stage a robbery or fight

Happiness  Lottery winnings?

Relaxation  Meditation

E X A MP L E

Week 3; Experimental designs

forming variables2
Forming Variables

2.Indirect experimental manipulations

  • Experimental “induction” of a mood or state…
    • “Stage manage” a social event 
    • Induce mood via description of the experiment
  • Do a manipulation check to see if you actually manipulated your Independent Variable
    • Self-report, standard assessment (e.g., of stress)
    • Observer rating

Week 3; Experimental designs

indirect experimental manipulation
Indirect experimental manipulation
  • Hypothesis: happiness enhances pain resistance.

Independent Variable

Experimental Manipulation

Dependent Variable

Imagine you won the lottery – what will you buy first?

Happy state

Cold Presser Task (ice bucket)

Experimental group

Control group

…what will you need to buy this month?

Normal / baseline state

Cold Presser Task

Directly controlled by experimenter

Notdirectlycontrolled by experimenter

Our induction of the Independent Variable (happiness) is indirect.

Week 3; Experimental designs

indirect experimental manipulation1
Indirect experimental manipulation
  • Hypothesis: happiness enhances pain resistance.

Independent Variable

Experimental Manipulation

Dependent Variable

Imagine you won the lottery – what will you buy first?

Happy state

Cold Presser Task (ice bucket)

Experimental group

Control group

…what will you need to buy this month?

Normal / baseline state

Cold Presser Task

AManipulation Checktests whether the experimental manipulation actually induced the Independent Variable

Week 3; Experimental designs

quasi independent variables
Quasi-independent variables

3.Create a quasi-Independent variable using a measured variable.

  • Categorize participants by measuring (not manipulating) something:
    • Scores over / under an established “cut point”,
      • e.g., over 4 depression symptoms on a standard scale.
    • Scores based on a frequency a distribution:
      • Median split: top v. bottom half.
      • Extreme scores: top v. bottom 10% of scores.
  • Simple self-identification:
      • e.g., “Republican” v. “Democrat”.
  • Behavioral index:
      • Used any drug in previous year v. not.
      • Voted in 2012 v. not.

Not a “True” IV: Participants not randomly assigned to groups.

Week 3; Experimental designs

using a measured variable to create groups
Using a measured variable to create groups

# of symptoms rated 2 or 3

  • Administer depression scale, count the # symptoms rated 2 or 3.

Form groups based on a cut point; e.g., > 4 symptoms = quasi-clinical depression.

Participants are assigned to groups based on their ratings, not random assignment.

Below is a list of different feelings. Circle the number that shows howmanydaysyou felt each of these over the PAST WEEK.

Rarely or A Little A moderate Most or all of

none of of the Time amount of the time

the time the time

(less than 1 day) (1 or 2 days) (3 - 4 days) (5 - 7 days)

I was bothered by things that usually do 0 1 2 3

not bother me.

I felt I could not shake off the blues even 0 1 2 3

with help from my friends or family.

I had trouble keeping my mind on what 0 1 2 3

I was doing.

I felt depressed. 0 1 2 3

I felt that everything I did was an effort. 0 1 2 3

My sleep was restless. 0 1 2 3

I was happy. 0 1 2 3

I enjoyed life. 0 1 2 3

I felt sad. 0 1 2 3

Week 3; Experimental designs

question 4
Question 4

An independent variable…

A = Is measured on a continuous scale

B = Is manipulated by the researcher

C = Is the outcome of the experiment

D = Is the “phenomenon” you are trying to explain.

E = Does not care about other people

Week 3; Experimental designs

question 5
Question 5

An dependent variable…

A = Is typically measured on a binary scale

B = Is manipulated by the researcher

C = Is the putative cause in the theory

D = Is the “phenomenon” you are trying to explain.

E = Is over-concerned about other people

Week 3; Experimental designs

the big picture introductory lectures3
The big picture: Introductory lectures
  • How do social values affect science?
  • Where do research questions & hypotheses come from.
  • Variables in research
  • Basic experimental designs
  • True vs. Quasi- Experiments

Phenomenon

Theory

Hypothesis

Methods

& data

Results

Discussion &

Conclusions

Week 3; Experimental designs

overview basic designs
Overview: Basic Designs
  • “Pre-experimental” designs: no control group

Experimental

Observe1

Treatment

Observe2

Post-Test Only Design

Pre- Post- Test Design

Group assignment

Pre-test

Experimental manipulation

Outcome

Week 3; Experimental designs

basic designs
Basic Designs
  • “Pre-experimental” designs: no control group

Experimental

Observe1

Observe1

Treatment

Observe2

Post-Test Only Design

Pre- Post- Test Design

True (or Quasi-)experimental designs with a control group

“After only” Control group design

Pre- Post- Group Comparisons

Group assignment

Pre-test

Experimental manipulation

Outcome

Control

Control

Observe2

basic designs1
Basic Designs
  • “Pre-experimental” designs: no control group

Post-Test Only Design

Experimental

Observe1

Treatment 1

Observe2

Pre- Post- Test Design

Control

Experimental

Observe1

Treatment 2

Observe2

True (or Quasi-)experimental designs with a control group

Control

Observe1

Observe2

“After only” Control group design

Pre- Post- Group Comparisons

Multiple group comparison

Group assignment

Pre-test

Experimental manipulation

Outcome

pre experimental designs
“Pre-experimental” designs

Post-Test Only Design

Group

Treatment

Measure

Only 1 group - typically an existing group: no selection or assignment occurs.

Experimental intervention (“Treatment”) may or may not be controlled by the researcher.

Use for naturally occurring or system-wide events (e.g., group trauma, government policy change, etc.).

Measurement may or may not be controlled by the researcher.

Pre- Post- Test Design

Group

Measure1

Treatment

Measure1

Only one group;

  • only group available?
  • naturally occurring intervention?

Measurements given to all participants at baseline & follow-up

All participants get the same treatment, which may or may not be controlled by the researcher.

Week 3; Experimental designs

pre experimental designs 2
“Pre-experimental” Designs (2)

Advantage of “Post-” & “Pre- Post-” Designs:

Allow us to study naturally occurring interventions.

    • e.g., test scores before and after some school change,
    • Crime rates after a policy change, etc.
  • Having both Pre- and Post measures allows us to examine change.

Week 3; Experimental designs

pre experimental designs 21
“Pre-experimental” Designs (2)

Disadvantage of “Post-” & “Pre- Post-” Designs:

No control group = many threats to internal validity.

Maturation: Participants may be older / wiser by the post-test

History; Cultural or historical events may occur between pre- and post-test that change the participants

Mortality: Participants may non-randomly drop out of the study

Regression to baseline: Participants who are more extreme at baseline look less extreme over time as a statistical confound.

Reactive Measurement:Scores may change simply due to being measured twice, not the experimental manipulation.

Week 3; Experimental designs

experiments
Experiments

“After only” Control group design

Control

Experimental

Treatment 2

Observe2

Adds a control group. Either…

Observed Groups:

  • Naturally occurring (e.g., Class 1. v. Class 2) or
  • Self-selected (sought therapy v. did not…).

Assigned Groups:

  • Randomlyassign participants to experimental v. control group, or
  • Matchparticipants to create equivalent groups.

Measure Dependent Variable(s) only at follow-up.

Use experimental or standard measures (e.g., grades, census data, crime reports).

Control

Observe2

Week 3; Experimental designs

advantages of experimental design
Advantages of experimental design

“After only” Control group design

Control

Experimental

Treatment 2

Observe2

Control

Observe2

Advantage: Lessens the likelihood of confounds or threats to internal validity.

  • Control group
  • Random assignment

Disadvantage: Existing or self-selected groups may have confounds.

No baseline or pre- measure available:

  • We cannot assess change over time.
  • We cannot assess whether the groups are equivalent at baseline.

Week 3; Experimental designs

basic designs true experiments 2
Basic Designs: True experiments (2)

Pre- Post- Group Comparisons (most common study design)

Group 1

Measure 1

Group 2

Measure 1

Two groups:

Observed

(quasi-experiment)

or

Assigned

(true experiment).

Baseline (“pre-test”) measure of study variables and possible confounds.

Week 3; Experimental designs

basic designs true experiments 21
Basic Designs: True experiments (2)

Pre- Post- Group Comparisons (most common study design)

Group 1

Measure 1

Treatment

Measure 2

Control

Group 2

Measure 1

Measure2

The group getting the experimental condition is contrasted with a control group..

“Post-test” follow-up of dependent variable(s);

  • Simple outcome
  • Change from baseline.

Advantages: Pre-measure assesses baseline level of Dependent Variable

-- allows researcher to assess change

-- can detect ceiling (or floor) effects

-- can use to assign participants to groups via matching

-- can assess baseline equivalence of groups

Disadvantage: Highly susceptible to confounds if using observed or self-selected groups.

Week 3; Experimental designs

basic designs true experiments 22
Basic Designs: True experiments (2)

Pre- Post- Group Comparisons (most common study design)

Group 1

Measure 1

Treatment

Measure 2

Control

Group 2

Measure 1

Measure2

Advantages: Pre-measure assesses baseline level of Dependent Variable

  • Allows researcher to assess change
  • Can find matched pairs of participants and assign each to different groups (rather than random assignment).
  • Can assess whether groups are equivalent at baseline.

Disadvantage: Highly susceptible to confounds if using observed or self-selected groups.

Week 3; Experimental designs

more complex experimental designs
More Complex Experimental Designs

Multiple group comparison

Group 1

Measure1

Treatment #1

Measure1

Group 2

Treatment #2

Control

Measure1

Group 3

  • 3 (or more) groups
  • Typically formed by Random assignment.

Multiple experimental groups, e.g.

  • Low drug dose,
  • High drug dose,
  • Placebo.

or

  • Male therapist,
  • Female therapist,
  • Wait list control.

Week 3; Experimental designs

more complex experimental designs1
More Complex Experimental Designs

Multiple group comparison

Group 1

Measure1

Treatment #1

Measure2

Measure1

Group 2

Treatment #2

Measure2

Control

Measure1

Group 3

Measure2

Compare:

  • Level 1 of independent variable from Level 2
  • Either / both experimental groups from control grp.

Week 3; Experimental designs

more complex experimental designs2
More Complex Experimental Designs

Multiple group comparison

Group 1

Measure1

Treatment #1

Measure2

Measure1

Group 2

Treatment #2

Measure2

Control

Measure1

Group 3

Measure2

Advantages: Test dose or context effects:

  • Drug doses, amounts of psychotherapy, levels of anxiety, etc. Increasing dose effect can be tested against no dose.
  • Diverse conditions to test 2nd hypotheses or confounds, e.g., therapy delivered by same sex v. opposite sex therapist.

Disadvantage:

  • More costly and complex.
  • Potential ethical problem with a “no dose” (or very high dose) condition.

Week 3; Experimental designs

experimental design overview
Experimental design overview

We recruit a sample of participants from the larger population

…then randomly assign them to groups.

Procedures for all groups should be exactly the same…

…except the experimental manipulation, (Independent variable).

Hypothesis: The outcome (Dependent Variable) varies by group only.

Week 3; Experimental designs

overview of true experimental designs
Overview of true experimental designs

Experimental group

Control group

Week 3; Experimental designs

overview experimental designs
Overview: experimental designs

Does the sample well represent the population?

  • Is recruitment biased?
  • Is the sample size large enough?

External validity

Random selection

overview experimental designs1
Overview: experimental designs

Does the sample well represent the population?

Are the groups equal at baseline?

  • Self-selection (in or out)
  • Existing groups?

External validity

Random selection

Internal validity

Random Assignment

overview experimental designs2
Overview: experimental designs
  • Do both groups have the same expectations?
  • Are participants (and researchers) really blind?
  • Do we treat both groups the same?

Does the sample well represent the population?

Are the groups equal at baseline?

Procedures the same for all groups?

External validity

Random selection

Internal validity

Random Assignment

Internal validity:

Lack of confounds

overview experimental designs3
Overview: experimental designs

Procedures the same for all groups?

Does the sample well represent the population?

Are the groups equal at baseline?

Independent variable faithfully reflects the construct?

  • Does the operational definition really express the construct we are interested in?
  • Have we given the correct dose of the IV?

External validity

Random selection

Internal validity

Random Assignment

Internal validity:

Lack of confounds

External Validity

Correct IV?

overview experimental designs4
Overview: experimental designs

Procedures the same for all groups?

Does the sample well represent the population?

Are the groups equal at baseline?

Independent variable faithfully reflects the construct?

Groups really different at outcome?

  • Is any difference we see actually statistically significant (reliable & meaningful)?

External validity

Random selection

Internal validity

Random Assignment

Internal validity:

Lack of confounds

External Validity

Correct IV?

Internal Validity:

Likelihood of chance results

overview experimental designs5
Overview: experimental designs

Does the sample well represent the population?

Are the groups equal at baseline?

Procedures the same for all groups?

Independent variable faithfully reflects the construct?

Groups really different at outcome?

External validity

Random selection

Internal validity

Random Assignment

Internal validity:

Lack of confounds

External Validity

Correct IV?

Internal Validity:

Likelihood of chance results

internal external validity
Internal & external validity
  • How do social values affect science?
  • Where do research questions & hypotheses come from.
  • Variables in research
  • Basic experimental designs
  • True vs. Quasi- Experiments

Experimental Design & sampling

true experiments
True experiments

How do True experiments differ from Quasi-experiments?

Group 1

Observe1

Treatment

Observe2

Observe2

Group 2

Observe1

Control

Group Assignment

Baseline (pre-test)

Experimental condition

Follow-up (post-test)

Week 3; Experimental designs

true experiments1
True experiments

Key elements of true experiments

  • Manipulate the Independent Variable
  • Randomly assign participants to groups
  • Groups are Equivalent:
    • Participants
    • Researcher
    • Common procedures

Group 1

Observe1

Treatment

Observe2

are blind

Observe2

Group 2

Observe1

Control

Group Assignment

Baseline (pre-test)

Experimental condition

Follow-up (post-test)

Week 3; Experimental designs

quasi experiments
Quasi-experiments

Quasi-experiments

  • No control over the Independent Variable;
    • Naturally occurring event
    • Existing data
  • Non-equivalent groups:
    • Non-random assignment
    • Participants not blind
    • Naturally occurring groups
    • Self-selection into groups

Group 1

Observe1

Treatment

Observe2

Observe2

Group 2

Observe1

Control

Group Assignment

Baseline (pre-test)

Experimental condition

Follow-up (post-test)

Week 3; Experimental designs

slide69

Experimental Designs, example 1

  • True Experiment
  • I randomly assign participants to groups
  • I manipulate the independent variable
    • Group 1: Drug dose
    • Group 2: Placeboor alternate drug
  • I keep all procedures = across groups
    • E.g., keeping researcher & participants blind.
  • I am (more) assured that the outcome is due only to the independent variable

E X A MP LE

Random Assignment to Groups

Procedures = for groups

Baseline Assessment

Experimental condition

Follow-up (post-test)

Assess all participants’

blood pres.

Group 1

New drug

Assess Bp for 1 year for all Parts.

Group 2

Placebo

Experimental Design & sampling

slide70

Experimental Designs, example 1

  • Quasi-experiment
  • Random assignment may not be possible.
    • Existing groups
    • Self-select in or out of experiment
  • Researcher & participants not blind?
    • Psychotherapy or other exp. conditions
    • Researcher cannot be blind?
  • May not controlall of the experiment.
    • Cannot treat gropsexactly equivalent?

E X A MP LE

Random assignment possible?

Random Assignment to Groups

Groups may need different procedures

Baseline Assessment

Experimental condition

Follow-up (post-test)

Assess fear & loathing of statistics

Group 1

Induce stress

More may drop out of stress grp.

Group 2

Relax

Experimental Design & sampling

overview key terms
Overview: key terms
  • Theory
  • Hypothetical construct
  • Hypothesis
  • Variable
  • Operational definition
  • Internal & external validity
  • Independent v. Dependent variables
  • Measurement v. experimental studies

Weeks 1 & 2; Introduction to science.

slide72

Research flow

Week 3; Experimental designs

basics of major forms of research
Basics of major forms of research.

External validity Internal validity

Weeks 1 & 2; Introduction to science.

key terms concepts
Key terms & concepts
  • Role of values & social judgments in the research process
  • Basic elements of science
    • Hypothetical constructs
    • Operational definitions
    • Statement of testable hypothesis
      • Predictive, potentially refutable
      • Specify Variables in functional relationship
    • Replication

The hierarchy of phenomena, theory, hypotheses, & methods:

Week 3; Experimental designs

key terms concepts 2
Key terms & concepts, 2
  • Measurement v. experimental methods
    • Types of variables used
    • Cause & effect assumptions
  • Creating variables
    • Direct treatment dose or manipulation
    • Indirect use of context (manipulation check)
    • Using a measured variable (self-reports or “status” variable”) to assign to groups

Week 3; Experimental designs

overview 3
Overview, 3
  • Experimental design key elements
      • Control group v. non-controlled designs
      • Threats to internal validity:
  • Maturation
  • History
  • Mortality
  • Regression to baseline
  • ReactiveMeasurement
  • “Pre-experimental” designs
  • Pre-post designs
  • Multiple group comparisons.

Week 3; Experimental designs

ad