1 / 31

Baseline Analysis CBP, AMP, and DBP

Baseline Analysis CBP, AMP, and DBP. Steve Braithwait, Dan Hansen, and Dave Armstrong Christensen Associates Energy Consulting DRMEC Spring Workshop May 7, 2014. Presentation Outline. Objectives Methodology Data Performance measures Aggregator program (CBP and AMP) results

hadar
Download Presentation

Baseline Analysis CBP, AMP, and DBP

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Baseline AnalysisCBP, AMP, and DBP Steve Braithwait, Dan Hansen, and Dave Armstrong Christensen Associates Energy Consulting DRMEC Spring Workshop May 7, 2014

  2. Presentation Outline • Objectives • Methodology • Data • Performance measures • Aggregator program (CBP and AMP) results • Demand Bidding Program (DBP) results

  3. Objective: Assess Performance of Alternative Baseline Types • For each Utility and Notice type: • All customers, with BL adjustment as chosen • All customers, simulated with universal selection of the BL adjustment • Sum of individual BL vs. portfolio BL (constructed from aggregated customer loads), for AMP and CBP only • Examine unadjusted and day-of adjustments with 20%, 30%, 40%, 50% caps, and uncapped

  4. Analysis Details • For actual program event days • The “true” baseline is the estimated reference load from the ex post evaluation • For event-like non-event days • The “true” baseline is the observed load

  5. Performance Measures (1)Percentage Baseline Error • Percentage BL error for each customer/portfolio-event day is: • Percentage error = (LPd – LAd) / LAd • LAd = actual, or “true” baseline load on day d • LPd = “predicted” baseline to be evaluated • Positive value = over-estimated baseline (implies over-stated program load impact) • Negative value = under-estimatedbaseline (implies under-stated program load impact)

  6. Performance Measures (2) Accuracy • Accuracy is measured as the median absolute percentage error (MAPE) • Calculate the absolute value of the percentage error for each customer/event-day • Calculate the median of values across customer/event-days (mean can be misleading due to extreme values) • Higher values correspond to larger baseline errors

  7. Performance Measures (3)Bias • Bias is measured by the median percentage error, without taking the absolute value • Positive values indicate upward bias (i.e., the program baseline tends to over-state the “true” baseline) • Negative values indicate downward bias (i.e., the program baseline tends to under-state the “true” baseline)

  8. Nominated Customers by Choice of BL Adjustment – CBP and AMP

  9. Accuracy (Median Abs. % Error)PG&E CBP-DO

  10. Bias (Median % Error)PG&E CBP-DO

  11. Percentiles of % Errors – PG&E CBP-DOActual Events, by Adjustment Cap

  12. Percentiles of % Errors – PG&E CBP-DOSimulated Events, by Adjustment Cap

  13. Summary: Accuracy & Bias (Aggregated Indiv.; Universal Adj.; 40% cap)

  14. Summary: Percentiles of % Errors(Aggregated Indiv.; Universal Adj.; 40% cap)

  15. Summary of Findings • Accuracy and bias measures vary by utility, program and notice type • Suggests that factors other than baseline type and adjustment caps may be most important, such as types of customers (e.g., highly variable load) and event-day characteristics (e.g., event on isolated hot day) • Day-of adjustment often improves accuracy and reduces bias, but level of cap is less important • Largest errors typically occur for Unadjusted BL and Unlimited cap • BL with small median error (e.g., 1%) can have >10% errors in 20 percent of cases

  16. DBP Results:PG&E Distribution of % Errors

  17. DBP Results:SCE Distribution of % Errors

  18. Summary • Day-of adjustments tend to improve baseline accuracy and reduce bias • The analysis provides support for making the day-of adjustment the default option • The effectiveness of the day-of adjustment is not very sensitive to the level of the cap

  19. Questions? • Contact – Steve Braithwait or Dan Hansen, Christensen Associates Energy ConsultingMadison, Wisconsin • Steve@CAEnergy.com • Danh@CAEnergy.com • 608-231-2266

  20. Appendix • SCE – CBP DO • SDG&E – CBP DO • PG&E – AMP DO • SCE – AMP DO

  21. Accuracy (Median Abs. % Error)SCE CBP-DO

  22. Bias (Median % Error)SCE CBP-DO

  23. Percentiles of % Errors – SCE CBP-DOActual Events, by Adjustment Cap

  24. Percentiles of % Errors – SCE CBP-DOSimulated Events, by Adjustment Cap

  25. Accuracy (Median Abs. % Error)SDG&E CBP-DO

  26. Accuracy – Med. Abs. Err. (MW)SDG&E CBP DO

  27. Bias (Median % Error)SDG&E CBP-DO

  28. Accuracy (Median Abs. % Error)PG&E AMP-DO

  29. Bias (Median % Error)PG&E AMP-DO

  30. Accuracy (Median Abs. % Error)SCE AMP-DO

  31. Bias (Median % Error)SCE AMP-DO

More Related