evaluating osha inspections for intended and unintended outcomes n.
Skip this Video
Loading SlideShow in 5 Seconds..
Evaluating OSHA inspections for intended and unintended outcomes PowerPoint Presentation
Download Presentation
Evaluating OSHA inspections for intended and unintended outcomes

Loading in 2 Seconds...

play fullscreen
1 / 30

Evaluating OSHA inspections for intended and unintended outcomes - PowerPoint PPT Presentation

  • Uploaded on

Evaluating OSHA inspections for intended and unintended outcomes. Mike Toffel Associate Professor Harvard Business School Liberty Mutual Research Institute for Safety August 2012. May 18, 2012 . OSHA, a much-criticized agency. Too lenient! (?) Inspections too seldom

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Evaluating OSHA inspections for intended and unintended outcomes' - freja

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
evaluating osha inspections for intended and unintended outcomes
Evaluating OSHA inspections for intended and unintended outcomes

Mike Toffel

Associate Professor

Harvard Business School

Liberty Mutual Research Institute for Safety

August 2012

osha a much criticized agency
OSHA, a much-criticized agency
  • Too lenient! (?)
  • Inspections too seldom
  • Penalties too small
  • Lengthy process to adopt new regulations compromises worker safety
  • Too costly! (?)
  • Stifling job creation / job killer
  • Increasing labor costs
  • Eroding America's competitiveness
challenges evaluating impact of osha inspections causality
Challenges evaluating impact of OSHA inspections:Causality
  • Most OSHA inspections are not random:
    • After accidents and deaths
    • When employees complain
  • If accidents/deaths are rare events, outcomes will feature mean reversion:
  • Problems likely decline after inspections… but even without inspections
  • Our approach
  • Examine random inspections and compare to a control group.
challenges evaluating impact of osha inspections measuring outcomes
Challenges evaluating impact of OSHA inspections:Measuring outcomes
  • Several studies have relied on company logs
  • But inspections can lead companies to improve logs’ comprehensiveness, increasing reported injuries
  • This cloaks changes in actual injury rates
  • Our approach
  • Rely on workers’ compensation claims
    • Annual number of claims
    • Annual cost of all claims
3 data sources
3 data sources



Dun & Bradstreet





  • Data restrictions
  • California
  • Cal/OSHA
  • Some high hazard industries (randomization targets)
  • Single-plant firms


Number of claims

Cost of claims

Occupation classes


data issues
Data issues
  • WCIRB data
  • MOU between California Department of Insurance (CDI) and Commission on Health and Safety and Worker’s Compensation listing us as subcontractors
  • Required to keep data anonymous
  • OSHA data
  • Complex data structure
  • Limited documentation
  • Many conversations with Cal/OSHA to understand implementation of randomization process
developing a matched sample
Developing a matched sample
  • Treatments
  • Single-plant establishments randomly selected for a programmed inspection
    • High-hazard industries Cal/OSHA targeted for random inspection each year
  • Matched controls
  • Find population of single-plant establishments at risk of random inspection, but not selected
    • Exclude if < 10 employees or recently inspected
  • For each treatment, select one control:
    • Same industry and region
    • Closest size

Result: 409 matched pairs

matching led to a balanced sample of very similar treatments and controls
Matching led to a balanced sample of very similar treatments and controls
  • In the two years before the match year, the 409 matched pairs had very similar:
  • Sales
  • Employment
  • Payroll
  • Credit scores (PAYDEX, Comprehensive Credit Appraisal)
  • Annual number of WC claim
  • Annual total cost of WC claims
evaluation model
Evaluation model
  • Did treatments experience a greater decline in annual injuries (or injury-related costs) after inspections than the controls, examined over the same time period?
  • Fixed effects regression
  • Control for establishment characteristics
  • Difference-in-differences approach
    • Compares two groups over time
unanticipated consequences of inspections
Unanticipated consequences of inspections?
  • Inspections (and consequences) cause interruptions, but are they substantial?
    • Sales impact?
    • Credit worthiness?
    • Employment, payroll?
    • Firm survival?
unanticipated consequences of inspections2
Unanticipated consequences of inspections?

No difference in employment, payroll, sales (Table 2)

    • Tight confidence intervals enable us to rule out that inspections caused big declines of employment or payroll
  • No difference in credit ratings (Table S8)
  • Late bills, etc. more sensitive to financial burden than firm death
    • Two D&B metrics of financial distress: PAYDEX & CCA
  • No difference in firm survival (Table S7)
    • Approx. 5% of treatments and of controls died
      • Difference not statistically significant
      • Result robust to survival regressions
summary of results
Summary of results
  • Evidence of intended results
  • Annual injuries reduced by 9.4%
  • Annual injury costs reduced by 26%
  • No evidence of unintended consequences
    • Sales impact
    • Credit worthiness
    • Employment, payroll
    • Firm survival
  • Cal/OSHA is not entirely toothless
    • Injuries and injury costs decline a lot
    • Social value of very approximately $350,000 per inspection
      • Includes medical costs and lost wages, but not productivity loss, pain and suffering, etc.
      • With many assumptions (similar effects across all states and inspection type, etc.) translates to $22 B per year nationwide
  • OSHA is not detectably a job killer
    • But we cannot rule out some costs

Levine, David I., Michael W. Toffel, and Matthew S. Johnson. "Randomized Government Safety Inspections Reduce Worker Injuries with No Detectable Job Loss." Science 336, no. 6083 (May 18, 2012)

future research in this domain
Future research in this domain

1. When do random inspections bolster regulatory compliance?

  • High vs. low hazardous industries
  • Strong vs. weak compliance history
  • VPP participants versus others
  • Prior OSHA consultation versus not

2. Do random inspections have spillover effects?

  • Organizational: bolster compliance at corporate siblings?
  • Geographic: bolster compliance at neighboring facilities?
  • Regulated domains: bolster compliance with EPA regulations?

3. Your ideas?

  • Loss prevention vs. OSHA inspections?
other research
Other research…
  • Environmental reporting & carbon disclosure
    • Responding to public and private politics: Corporate disclosure of climate change strategies, Strategic Management Journal (2009)
    • Engaging supply chains in climate change. Working Paper (2012)
    • When do firms greenwash? Corporate visibility, civil society scrutiny, and environmental disclosure, Working Paper (2012)
  • Environmental ratings
    • How well do social ratings actually measure corporate social responsibility? Journal of Economics & Management Strategy (2009)
    • How firms respond to being rated, Strategic Management Journal (2010)
environmental policy evaluation
Environmental policy evaluation
  • Voluntary violations disclosure (EPA Audit Policy)
    • Coerced confessions: Self-policing in the shadow of the regulator, Journal of Law, Economics and Organization (2008)
    • Coming clean and cleaning up: Does voluntary self-reporting indicate effective self-policing, Journal of Law and Economics (2011)
    • Making self-regulation more than merely symbolic: The critical role of the legal environment, Administrative Science Quarterly (2010)
  • Mandatory performance disclosure
    • How firms respond to mandatory information disclosure, Strategic Management Journal (forthcoming)
  • Tailpipe emissions fraud
    • Competition and illicit quality, Working Paper (2012)
    • The role of organizational scope and governance in strengthening private monitoring, Working Paper (2012)

Quality management: ISO 9001 evaluation

    • Quality management and job quality: How the ISO 9001 standard for quality management systems affects employees and employers, Management Science (2010).
  • Occupational health and safety
  • Randomized government safety inspections reduce worker injuries with no detectable job loss, Science (2012).
thank you
Thank you!
  • Prof. Michael Toffel
  • Harvard Business School
  • Morgan Hall 497
  • Boston, MA 02163
  • mtoffel@hbs.edu
  • www.people.hbs.edu/mtoffel/