eval 6970 experimental and quasi experimental designs n.
Skip this Video
Loading SlideShow in 5 Seconds..
EVAL 6970: Experimental and Quasi-Experimental Designs PowerPoint Presentation
Download Presentation
EVAL 6970: Experimental and Quasi-Experimental Designs

Loading in 2 Seconds...

play fullscreen
1 / 21

EVAL 6970: Experimental and Quasi-Experimental Designs - PowerPoint PPT Presentation

  • Uploaded on

EVAL 6970: Experimental and Quasi-Experimental Designs. Dr. Chris L. S. Coryn Dr. Anne Cullen Spring 2012. Agenda. Regression discontinuity designs. Questions to Consider. Throughout today’s discussion, consider the following questions

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'EVAL 6970: Experimental and Quasi-Experimental Designs' - kyle-dennis

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
eval 6970 experimental and quasi experimental designs

EVAL 6970:Experimental and Quasi-Experimental Designs

Dr. Chris L. S. Coryn

Dr. Anne Cullen

Spring 2012


Regression discontinuity designs

questions to consider
Questions to Consider
  • Throughout today’s discussion, consider the following questions
    • What characteristics of regression discontinuity designs make them more amenable to causal interpretation versus those discussed previously (i.e., quasi-experimental designs with and without pretests and control groups and interrupted time-series)?
    • What are the associated limitations or drawbacks of regression discontinuity designs, if any?
    • How could you apply regression discontinuity designs to your own work?
  • The principles of regression discontinuity can be confusing, so…
some assumptions
Some Assumptions

Most randomized experiments compare posttest means

Regression discontinuity designs compare regression lines for treatment and control groups

Rather than making the assumption that pretest means are equivalent for both groups, regression discontinuity looks for a change between the functional form of the regression line (e.g. slope or intercept) for the treatment group and control

some assumptions1
Some Assumptions

In (most) other designs for quasi-experiments, the selection process is never fully known (selection is determined by a large system of variables beyond the researcher’s control)

In sharp contrast, there are no unknowns in the assignment process for regression discontinuity

In fact, the selection process is completely known and perfectly measured (even though there will be some error associated with the assignment variable)

Assignment variables only measure how participants got into conditions, and when assignment is based only on that score, error is effectively zero

some assumptions2
Some Assumptions
  • Additionally, most experiments and quasi-experiments try to equate treatment and control
  • Regression discontinuity designs explicitly acknowledge—and, in fact, base assignment on—pre-existing differences between treatment and control groups
  • Units are assigned to conditions based on a cutoff score (i.e., cut score) on an assignment variable
  • The assignment variable must occur prior to treatment
    • Units on one side of the cut score are assigned to one condition, and units on the other side to another condition
  • Like randomized experiments, regression discontinuity designs yield unbiased estimates of treatment effects (as long as assumptions are met)
  • Assignment to treatment must be based only on the cutoff score
  • The assignment variable cannot be caused by the treatment, and must be continuous
    • Dichotomous variables should not be used as an assignment variable, because this makes it impossible to estimate a regression function
  • The assignment variable is often a pretest score, but it can be almost anything
  • The cut score should be near the mean of the assignment variable, because extreme values can cause problems
    • Modeling the regression function can become more difficult and/or error-prone in smaller samples
    • Statistical power depends on sample size, and also “prefers” samples with equal or nearly equal sizes
  • Researchers can use a composite variable for assignment, to include the effect of multiple influences
  • Avoiding selection bias requires that
    • Assignment to conditions is strictly controlled
    • All units could have received treatment
types of effects
Types of Effects
  • First, note that “discontinuity” means
    • A treatment effect (if present) causes an upward or downward displacement in the regression function
    • A discontinuity can be a change in either the intercept or the slope
    • The discontinuity should occur at exactly the cut score
validity threats
Validity Threats
  • Few internal validity threats are plausible with a well planned and correctly executed regression discontinuity design
  • A plausible threat would have to cause a discontinuity in the regression line that corresponds precisely with the cut score
    • Except in rare cases, selection, history, and maturation threats are drastically reduced
    • However, attrition can be a major concern, especially if attrition rates are correlated with the assignment variable
  • Therefore, validity concerns mainly focus on statistical conclusion validity
    • In particular, the regression lines must be good models of functional form (e.g., nonlinear functions, interaction terms)