Types of experiments research designs
Download
1 / 30

Types of Experiments & Research Designs - PowerPoint PPT Presentation


  • 209 Views
  • Uploaded on

Types of Experiments & Research Designs. UAPP 702: Research Design for Urban & Public Policy Based on notes from Steven W. Peuquet. Ph.D. Types of Experiments. There are four kinds of experiments: True experiments Quasi-experiments Natural experiments Naturalistic experiments.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Types of Experiments & Research Designs' - akina


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Types of experiments research designs

Types of Experiments & Research Designs

UAPP 702: Research Design for Urban & Public Policy

Based on notes from Steven W. Peuquet. Ph.D.


Types of experiments
Types of Experiments

There are four kinds of experiments:

  • True experiments

  • Quasi-experiments

  • Natural experiments

  • Naturalistic experiments


True experiments
True Experiments

  • Some true experiments are done in the lab, others are done in the field.

  • There are five steps to follow in conducting true experiments:


True experiments1
True Experiments

1. Need at least two groups: anexperimental group and a control group. One groups gets the intervention, the other group does not.

2. Random assignment of individuals to the groups. (The degree to which random assignment ensures equivalence of the groups is dependant upon the size of the groups.)


True experiments2
True Experiments

3. The groups are measured on one or more dependent variables. This is called the pretest.

4. The intervention (independent variable) is introduced.

5. The dependent variables are measured again. This is the post test.


Quasi experiments
Quasi-Experiments

  • Are experiments where the ability to randomly assign individuals to the experimental and control groups is limited or nonexistent.

  • Quasi-experiments are more commonly used in evaluating social programs.


Quasi experiments1
Quasi-Experiments

Example

In a study of the effectiveness of a new math curriculum, a 6th grade class in Porter Middle School was not given the new curriculum (the “control group”), while the 6th grade class in Farley Middle School was given the new curriculum (experimental group).


Quasi experiments2
Quasi-Experiments

Example (cont.)

At the end of the school year the 6th graders in the Farley MS class had higher average math achievement test scores than the kids in the Porter MS 6th grade class.

Is this a “quasi-experiment”? Why?


Quasi experiments3
Quasi-Experiments

Example (cont.)

Answer: Most likely -- YES

Random assignment to the two classes was probably not made, hence, groups may not be equivalent. Possibility of results being confounded by socio-economic status (SES) or other factors.


Natural experiments
Natural Experiments

Natural experiments are happening around us all the time. They are not conducted by researchers, but simply evaluated by researchers. In other words, the researcher does not have control over the application of the treatment. This also means that there is no control over what groups receive the treatment and the composition of those groups.


Natural experiments1
Natural Experiments

Examples:

  • Migration patterns of people — some people migrate, some don’t.

  • Some middle class Latino students go to college, some don’t.

    Note: Each of these situations tests something

    about human behavior, the question is: “What

    is being tested by what is going on?”


Naturalistic experiments
Naturalistic Experiments

In naturalistic experiments, one contrives to collect experimental data under natural conditions. You make the data happen out in the natural world (not in the lab), and you evaluate the results.


Naturalistic experiments1
Naturalistic Experiments

The difference between a natural experiment and a naturalistic experiment is that the first just happens, the second must be contrived to happen.


Naturalistic experiments2
Naturalistic Experiments

Naturalistic experiments deviate from true experiments because group membership is not randomly assigned, and exogenous factors (confounding variables) are not controlled.


Naturalistic experiments3
Naturalistic Experiments

Examples:

  • Links needed to connect any two randomly selected people (passage of folders)

  • Patience of drivers behind a car that did not move soon after a red light turned green (independent variables: newness of car and dress of driver)

  • Matched pair testing to uncover discrimination


Internal and ExternalValidity of an Experiment


Internal validity
Internal Validity

  • High internal validity means that changes in the dependent variable were caused by—not merely related to or correlated with—the treatment.

  • High internal validity can be attained only in an ideal true experimental design, or when steps have been taken to eliminate confounding factors or influences.


Internal validity1
Internal Validity

Internal validity can be increased by:

  • Repeating an experiment many times, and by changing or adding independent variables;

  • Switching the control and experimental groups

  • Controlling confounding influences.


Internal validity2
Internal Validity

Controlled experiments have the virtue of producing results with high internal validity, but they have the liability of low external validity


External validity
External Validity

External validity is the extent to which the cause and effect relationship can be generalized to other groups that were not part of the experiment.


External validity1
External Validity

External validity is critically important in policy and program evaluation research because we would like our research findings to be generalizable. If external validity is low, then the results are not generalizable.


Threats to validity
Threats to Validity

History confound —

Any independent variable other than the treatment variable, that occurs between the pre-test and the post-test.


Threats to validity1
Threats to Validity

Maturation confound —

Refers to the fact that as the experiment is conducted, people are growing older, and are getting more experienced. (Problem in longitudinal studies in particular)


Threats to validity2
Threats to Validity

Testing confound —

Refers to different responses that an experimental subject may have to questions they are asked over and over again.


Threats to validity3
Threats to Validity

Instrumentation confound —

Results from changing measurement instruments. (Example: use of different people to collect data, if there is no inter-rate reliability, you have changed the measurement instrument.)


Threats to validity4
Threats to Validity

Regression to the mean —

Groups tend to increasingly resemble the mean over time.


Threats to validity5
Threats to Validity

Selection bias confound —

When self-selection by individuals into groups is possible, a selection bias can result.


Threats to validity6
Threats to Validity

Diffusion of treatment

confound —

Occurs when a control group cannot be prevented from receiving the treatment in an experiment.


Examples
Examples

Let’s look at two related but different research questions, and see what types of experiments would be appropriate for each.


Final important principle
Final Important Principle!

NEVER USE A RESEARCH DESIGN OF LESS LOGICAL POWER WHEN ONE OF GREATER LOGICAL POWER IS FEASIBLE !


ad