NTG Estimation for California IOUs’ 2004-2005 Upstream Lighting Program

1. NTG Estimation for California IOUs’ 2004-2005 Upstream Lighting Program CALMAC Meeting Pacific Energy Center July 18, 2007 Tami Rasmussen, KEMA Inc.

2. Organization of Presentation • Program characteristics • NTG assessment objectives • Methods considered • Overview of each method • Caveats • Whether method was used/if not, under what contexts method may be useful • Keys to success for method used

3. Program Characteristics • Upstream program • lighting retailers and manufacturers receive the incentives (point-of-sale or buydown) • Consumers may or may not be aware that the price of the product they are buying has been reduced by the program • Prior California IOU lighting programs have created broad and cumulative market effects • Prior and current programs have influenced the market beyond the specific products rebated

4. NTG Assessment Objectives • Prior Upstream Lighting program evaluations focused on characterizing the market and establishing linkages between program interventions and market change • Attribution was assessed qualitatively • 2004-2005 study required an explicit estimate of NTG • Estimate must include only free-ridership – spillover may be estimated separately but must be excluded from final NTG estimate

5. NTG Methods Considered – Net Effects • Overview of method • Net effects = Difference between baseline sales and total CFL sales • Baseline sales = sales that would have occurred in absence of the program (typically estimated on a per capita basis for a similar state or region that does not have a program) • Total CFL sales – often difficult to estimate • NTGR = ratio of program sales to net effects • This method has been used recently in the Northeast and Wisconsin

6. Net Effects Method • Caveats: • Difficult if not impossible to reliably estimate total CFL sales for California • Baseline sales estimates for CA will be overstated (net effects/NTGR will be understated) because any region or state without a program will have been influenced by CA’s programs • Method will implicitly capture cumulative effects of prior CA programs (reflected in total CFL sales) leading to overstatement of net effects/NTGR • Results include both free-ridership and spillover

7. Net Effects Method • Method may be useful: • In regions where programs are small-scale, such that the national market is not effected (so baseline estimates are not biased) • Where total CFL sales may be reliably estimated • During the first year or two of a program, where cumulative effects are not a significant issue • In contexts where it is acceptable to use a NTGR that includes both free-ridership and spillover

8. NTG Methods Considered – Regional Regressions • Overview of method: • Model CFL sales (or market share/penetration) for various states/regions across the U.S. including California • Use the parameter estimates to estimate CFL sales in California in absence of the program • Used in the Northeast to estimate program attribution of ENERGY STAR appliance rebates • Incorporates both free-ridership and spillover

9. Regional Regressions • Caveats: • Significant CFL data quality and availability issues across the U.S. • California programs have influenced CFL sales/market shares across the U.S. leading to an understatement of the program’s influence in California (like the net effects method) • Not enough observations (obs=states/regions) since few states/regions collect and/or estimate CFL sales

10. Regional Regressions • Method may be useful: • For CFLs if CFL sales data become more available across the U.S. • For other products

11. NTG Methods Considered – Participant Self-Report • Overview of method: • Surveying participants (retailers and manufacturers in this case) regarding the influence of the program on their behavior • May address free-ridership and spillover separately • Used for NYSERDA Upstream Motors program

12. Participant Self-Report • Caveats: • A handful of lighting suppliers handle the majority of CFL sales in the state • Key decision-makers are often hard to contact and may be unwilling to provide information they consider proprietary

13. Participant Self-Report – Selected Method • Algorithm used for 1 of 3 CF product categories

14. Participant Self-Report – Selected Method • Algorithm used to combine results

15. Participant Self-Report • Keys to success: • To set up analysis framework: • Anticipate how program influence will likely vary by CF product and market actor type • Obtain detailed program tracking data • To ensure representative sample: • Obtain current contact information for participating suppliers • Have program managers encourage participants to respond to evaluators • Identify and focus on “must-have” interviews (based on % of rebates)

16. Participant Self-Report • Keys to success, continued: • To address possible bias: • Sought FR estimates from both manufacturers and retailers (mostly buydown – retailers did not receive incentives) • Assessed consistency of FR estimates across and within various categories

17. End sheet Thank you for your attention.