1 / 19

Review of Sampling Methods in Base-Year Generation Studies: Key Findings and Recommendations

This report evaluates extrapolated vs. nonextrapolated new base-year generation studies, highlighting concerns, impacts, and proposed improvements. Key findings address study design, sampling errors, and technical data components.Responding to concerns, the report suggests maximizing efficiency by targeting largest diverters and enhancing documentation for future studies. The importance of thorough documentation and targeting top businesses for diversion studies is emphasized for optimal outcomes.

kelly-jarvis
Download Presentation

Review of Sampling Methods in Base-Year Generation Studies: Key Findings and Recommendations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Overview of the Final Report and Findings from the Review of Sampling Methods in Extrapolated New Base-Year Generation Studies May 11-12, 2004

  2. Types of New Base Year Generation Studies • Nonextrapolated • “Actual” diversion data collected from diversion programs implemented within the jurisdiction • Extrapolated • Includes diversion data collected from a sample of the non-residential businesses and extrapolated to the entire non-residential business sector

  3. Number of New Base-Year Generation Studies • Since April 1998, Board staff have reviewed 184 submitted new base year studies for Board consideration. • 143 Nonextrapolated studies • 41 Extrapolated studies

  4. Background • Board expressed concerns regarding validity of extrapolated studies • Working group convened • Board staff performed analyses on extrapolation vs. nonextrapolation study methodologies

  5. Extrapolated vs. Nonextrapolated Study Data Comparisons • Generation pounds per person per day was 79% higher for extrapolated methodologies • Diversion rates for extrapolated studies were significantly higher (53%) than nonextrapolated studies (41%). • Average source reduction tonnage in extrapolated studies exceeded nonextrapolated studies by an average of 300%.

  6. Response to Concerns regarding New Base-Year Study Data • Base-Year Modification Certification Request form • Board staff began on-site verification of new base-year generation study data • Board pursued independent, third-party review of extrapolated study methodologies

  7. Scope of Review • Review 20 extrapolated studies. • Only 2 of the 20 studies reviewed submitted sufficient data. • Provide observations and recommendations for improving extrapolated methodologies used. • Impact of poor quality or lack of study data.

  8. Findings • The findings in the final report address the impact of errors relating to: • Study design; • Sampling errors; and • Technical data components of nonresidential diversion surveys.

  9. Study Design • Jurisdictions should list assumptions made in the study design and use sample data to assess their validity and appropriateness. • Studies did not include listing of such assumptions. • Jurisdictions did not use the study data to test the validity of assumptions.

  10. Sampling Errors • Accurate data on the selection of samples is needed for Board staff to fully evaluate the sampling methodology. • Lack of documentation regarding sample selection. • Studies employed nonrandom sampling methods.

  11. Technical Data • Lack of documentation regarding nonresponse. • Most jurisdictions provided outlier analysis, but did not address all study design variables. • Lack of access to study data.

  12. Additional Analyses • Case Study of One Extrapolated New Base-Year Generation Study • Review of 35 extrapolated new base-year generation study data • Diversion of Top 10 Businesses Compared to All Surveyed Businesses

  13. Commercial Business diversion Represents Approximately 38% of Total Diversion 25% Residential NonResidential 37% Business Audits Res/ADC/Landfill Salvage 38%

  14. New Base Year Case Study: Impact of the Largest Businesses (333) Audited Relative to the Total Business Audits (approx. 7,600) 90 80 70 60 50 % of Diversion tonnage from Businesses Audits 40 30 20 10 0 top 10 top 20 top 30 Business Generators

  15. Diversion of Top 10 Businesses Compared to All Surveyed Businesses • By surveying 10 top businesses: • 94% of the jurisdictions captured 50% or more of the total diversion; • 60% of the jurisdictions captured 80% or more of the total diversion; and • 37% of the jurisdictions captured 90% or more of the total diversion.

  16. Maximizing Efficiencies in New Base-Year Generation Studies • Many of the jurisdictions would have been better off targeting their largest diverters and submitting a nonextrapolated study. • These jurisdictions could have captured most of the industrial/commercial diversion without having to address problems associated with sample selection, nonresponse, outliers, or choosing an estimator.

  17. Future Efforts • Board staff will: • Update the Board’s Base-Year Modification Request Certification form for extrapolated studies. • Update the Board’s Diversion Study Guide to explicitly underscore the importance of documenting all critical aspects of study design, implementation and diversion rate calculation. • Develop a Web-based reporting format.

  18. Future Efforts (continued) • Board staff will continue to • Assist jurisdictions regarding their future new base-year generation studies; and. • Stress the benefits of targeting the largest businesses first to measure the non-residential sector.

  19. Summary • Board staff were unable to perform a complete statistical review for 18 of the 20 studies due to insufficient study data. • Findings support the value of targeting the top 10 businesses.

More Related