1 / 10

UP504 Winter 2008 Prof. Campbell University of Michigan

UP504 Winter 2008 Prof. Campbell University of Michigan EXAMPLE OF USING REGRESSION TO TEST A POLICY IMPACT. Example: A city has 30 low-income housing projects .

sulwyn
Download Presentation

UP504 Winter 2008 Prof. Campbell University of Michigan

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. UP504 Winter 2008 Prof. Campbell University of Michigan EXAMPLE OF USING REGRESSION TO TEST A POLICY IMPACT

  2. Example: A city has 30 low-income housing projects. A large number of vacant units in these projects creates a wide variety of problems (reduced revenues, vandalism, lower morale of existing tenants, etc.) There is a wide range of vacancy rates, from less than 10 percent to over 30 percent. The city officials believe that drug trafficking in the housing projects is discouraging people from either moving into or staying in the projects. http://accuweather.ap.org/cgi-bin/aplaunch.pl Passersby walk past the Bromley-Heath Housing Complex in the Jamica Plain district of Boston, Monday, Nov. 2, 1998. What began as a 1960's experiment in self-government ended this past weekend when management of Bromley-Heath Housing Complex, Boston's only tenant-run public housing, was seized by the city after a number of drug related arrests. (AP Photo/Steven Senne)

  3. To prove the key role of drug dealing in shaping housing project vacancy rates, the city releases data showing that vacancy rates in projects with anti-drug programs (run by the police department) have a lower vacancy rate: Vacancy rate (projects with anti-drug programs): 19 percent Vacancy rate (projects without anti-drug programs): 25 percent And just to be sure, they ran a difference of means test to demonstrate that the results were statistically significant at the 0.05 level.

  4. To further demonstrate the significant role that this policy anti-drug program plays, the city also collects data on housing expenses family structure (since these two variables also affect vacancy rates). An unidentified Long Beach Police officer questions a suspect during a drug raid on an apartment complex as a child watches, Tuesday July 15, 1997 in Long Beach, Calif. An 80-member police task force arrested nearly a dozen people in a four-block area targeted for rock cocaine sales and drug safe houses. The operation culminated six weeks of surveillance and was called, "Operation Clean Streets."(AP Photo/MIchael Caulfield http://accuweather.ap.org/cgi-bin/aplaunch.pl

  5. The city then releases the results of their own in-house multiple regression analysis, controlling for these two variables. Even controlling for the other two variables, the police anti-drug program is statistically significant, and seems to reduce the vacancy rate by 6 percentage points -- and then asks for more money for the program.

  6. NOT SO FAST, cries a tenants organization, which has been skeptical of the police anti-drug program’s effectiveness, and instead argues that the strong presence of organized tenants groups makes the difference in keeping vacancy rates lower. http://www.nhi.org/online/issues/95/phorg.html

  7. Controlling also for this new variable, the police anti-drug program is no longer statistically significant, an instead the presence of the active tenants group makes the dramatic difference. (and look at that great R square!). However, we are no quite done…

  8. Write out the equation: • Predicted vacancy rate = • + 0.366 • - 0.256 [percent 2-parent families] • 0.125 [active tenants group] • Example (50% 2-parent & active tenants group]: • Predicted vacancy rate = • + 0.366 • - 0.256 [.50] • 0.125 [1] = .113 or 11.3 percent Since the police variable now has a statistically insignificant t-score, we remove it from the model. (We also remove the income variable, since it also becomes insignificant after we remove the police variable.) We are left with two independent variables: percent of 2-parent families and active tenants group.

  9. Looking at a correlation matrix can help understand the interrelationships between variables.

  10. Moral of the story? • Multiple regression can be a powerful tool in evaluation research • One should be careful of generalizing from under-specified models (with omitted variables). This is especially true when the R-square is dramatically less than 1.00. • Evaluation research is the effort to isolate the specific (or unique) impact of a specific policy/program/influence on a dependent variable (an outcome). • A great challenge is to estimate the “counter-factual”, in this case, what would the vacancy rate have been without the police anti-drug program. (Here we used regression to make a prediction -- one can never know for sure.) • Why might the police program initially seemed to have a strong positive effect? Perhaps because of self-selection: the police -- either intentionally or by accident -- may have targeted their program on housing projects that already had the characteristics of projects with lower vacancy rates. • How do you get around this self-selection bias: use random assignment (a sometimes difficult but powerful approach in evaluation research).

More Related