1 / 37

RTF – GUIDELINES FOR DEVELOPMENT FOR UES MEASURE LIFE

RTF – GUIDELINES FOR DEVELOPMENT FOR UES MEASURE LIFE. Meeting 3 - Interim Discussion 10/17/2011 Skumatz Economic Research Associates, Inc. (SERA) The Cadmus Group. TODAY’S AGENDA. 10:00 Reintroductions

mikayla-maldonado
Download Presentation

RTF – GUIDELINES FOR DEVELOPMENT FOR UES MEASURE LIFE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. RTF – GUIDELINES FOR DEVELOPMENT FOR UES MEASURE LIFE Meeting 3 - Interim Discussion 10/17/2011 Skumatz Economic Research Associates, Inc. (SERA) The Cadmus Group

  2. TODAY’S AGENDA • 10:00 Reintroductions • 10:05 Walk through draft measure life guidelines for next “classes” of measures (discussion throughout) • Deemed (values updates) • Provisional, • Simplified, • Custom (Plus) • 11:15 Action items between now and next meeting • 11:20 Other comments / discussion • 11:30 Adjourn SERA and Cadmus Group

  3. MEASURE LIFE GUIDELINES: ISSUE OF MEASURE “CLASSES” RTF Savings & Cost Classes • Deemed • Can deem UES, cost, load shape from eval, engineering • Provisional • Interim cost & UES for tech, meas, practices where deployment is designed to provide data or lead to values • Simplified M&V • No site specific needed (calculator) • Custom M&V • Infrequent – “generic” protocols needed Initially Proposed Lifetime Classes* • Hardware (only) • Hardware, influenced by behavioral • Behavioral •  reclassified because strong preference from RTF & neater mapping that direction *respectively 1, 3, 5; even classes are associated PROSPECTIVE

  4. CLASSIFICATION APPROACH

  5. MEASURE LIFE GUIDELINES: DEEMED ADAPTATION • Deemed measures: Based on evaluation data / and engineering estimates. Sunset date / reinstatement. •  Application to Measure Lifetime Guidelines: Many utilities use tables of “deemed” values for EUL estimates (historical, statistical, consensus, combo) • Discussion: Some studies, but often just transferred from another state. Rarely include important adjustment factors (weather, business). May bias. • Before measures can be “deemed” for RTF, we need 1) a review of the sufficiency of statistical studies underlying the values and/or 2) a preponderance of similar values elsewhere – adapted for RTF climate / conditions. • Treatment: Table of EUL values. • Examples: Shell; others affected by behavior, etc.?

  6. MEASURE LIFE GUIDELINES: PROVISIONAL ADAPTATION • Provisional measures: Interim values – insufficient evaluation data or engineering estimates to deem / deployment will collect data. Used for low / medium “risk” measures, supported by preliminary evaluations or reasonable engineering estimates. Allow “booking” w/later potential true-up. • Application to Measure Lifetime Guidelines: Can be applied to EULs realm for low to medium risk measures. We recommend a specific hierarchy for assigning these provisional values. • Discussion: We assume provisional measures will be where most of the “prospective” measures fall. In addition, measures with “gaps” that are low risk will be fall here. • Treatment: Hierarchy • Examples: Cooktops (if low risk, poor data) potentially; also “next generation” measures, and potentially any measure (including behavioral) in the short run.

  7. MEASURE LIFE GUIDELINES: SIMPLIFIED CALC ADAPTATION • Simplified calculation: Calculate using site specific input assumptions and engineering calculations or calculators. •  Application to Measure Lifetime Guidelines: Apply to some measures currently (e.g. CFLs via operating hours). IF sufficient enhanced EUL studies (using the custom M&V protocols) are conducted that include exogenous variables / assumptions in the model, we may be able to expand the list of lifetimes that can be addressed in this way. • Discussion: Can be very appropriate and serve as adjustable deemed values for certain classes of measures – after the relationship of causal effects that are uncovered in estimation work can be identified (e.g. potentially climate, operating hours, business type, etc.). • Treatment: Develop updating procedures. • Examples: CFLs currently; later pumps, motors, maybe refrigerators or HVAC equipment via O&M, climate, business type – that sort of thing.

  8. MEASURE LIFE GUIDELINES: CUSTOM MEASUREMENT • Custom measures: Can’t deem, infrequent, not meaured, not suited to “simplified” (lack data), vary in operating conditions, interacting measures, etc.  Generic protocols. • If a large burden is imposed, collaboration or program design modification is recommended; and an appeals process is provided. • Application to Measure Lifetime Guidelines: SR - use these guidelines to develop values to get to “simplified” (lack causal relation). LR, the description assigned above suits. • Discussion: Use to 1) assess the quality of estimates underpinning “Deemed” & “simplified”; In longer run, have guidelines for “Custom Measure/Behavior” and extra steps for “Custom-Behavior” (data / tracking). • Treatment: Guidelines from last time as core. • Examples: Behavioral, process, many until moved to “simplified” or (some) deemed…depending on “risk” and other factors.

  9. WHERE WE ARE

  10. MEASURE LIFE GUIDELINES:DEEMED – WHERE WE ARE… • Additional studies – gathering more! • Regions • Gaps to date

  11. MEASURE LIFE GUIDELINES: “DEEMED” – WHERE WE ARE…

  12. MEASURE LIFE GUIDELINES:DEEMED – WHERE WE ARE… • Raw studies • Plus data on accepted values from multiple states and utilities (CSG plus data from interviewees) • Gaps, differences, • Hierarchy to adapt…

  13. MEASURE LIFE GUIDELINES: DEEMED –WHERE WE ARE General guidelines process for updating DEEMED values: • Review EUL studies for quality, using the Custom guidelines. Review in context for the measure / activity / program. • Examine influence of contributing factors (exogenous values). If no contributing factors are significant, move to next step. If contributing factors are significant, consider for inclusion as a “simplified M&V” approach. • Examine suitability for RTF region, using hierarchy from Appendix A: sector, program, NW/climate, etc. • Wrong sector / wrong measures  custom guidelines only option (to consultants – decide if different procedures are ok for low risk, like provisional….) • Add to “deemed” table if appropriate (or “simplified” table)

  14. MEASURE LIFE GUIDELINES: PROVISIONAL – WHERE WE ARE Hierarchy for assigning “Provisional” Measure Lifetimes • Measures without history, sufficient obs to support custom - the following hierarchy is recommended: • Examine # of years needed to make the TRC B/C=1. • Examine manufacturer’s claimed lifetime (or panel review / Delphi / or deem a share of the value; review “class”). • If > one mfg, compute average. Make adj. for sector turnover • If mfg EUL claims well beyond range for TRC>1, use; if not, not cost-effective and don’t implement. • Use value as provisional estimate until “true up”, (but monitor installed measures). • True-Up requirements & frequency; and risk assessment - to be determined • Risk based on: share of 6 year plan savings, how close to B/C ratio of 1; other factors

  15. MEASURE LIFE GUIDELINES: SIMPLIFIED CALC - WHERE WE ARE Guidelines process for updating SIMPLIFIED M&V values: • Review EUL studies for quality, using Custom guidelines in context of other results available for the measure / activity. • If exogenous factors significant, consider for inclusion as a “simplified M&V” approach. (no effect  deemed table) • Examine suitability for RTF region, using hierarchy from Appendix A: sector, measure, NW/climate, etc. • Wrong sector / wrong measures  guidelines only option • Develop appropriate calculation tool based on the key contributing factor (e.g. operating hours, HDD, business type(s), etc.). • Add to “Simplified M&V” table if appropriate (or deemed table if appropriate) .

  16. MEASURE LIFE GUIDELINES: CUSTOM – WHERE WE ARE Reviewed last time • Data, sampling, Questionnaire, screening, precision, data coll’n, modeling (multiple, exogenous variables, context); documentation. Working on measurement frequency (if expected measure life is 1-2 years; 3-5, etc). • Key change – specific request of exogenous / behavioral • Handles measure AND measure + behavioral • Include “classes” of O&M / behavioral / influential factors in model • Investigate for drivers • Use “overall” now; consider refinements in case of new targets, etc. • Use refined levels as input into “Simplified” (depending on N, quality)…

  17. MEASURE LIFE GUIDELINES: CUSTOM – WHERE WE ARE

  18. MEASURE LIFE GUIDELINES: CUSTOM – WHERE WE ARE Revisions to address comments & behavioral only • Extra steps - Data collection – No ID of “uptake” / delivery • Complex - % of pop receiving; adopted; date stopped; EUL estimate • Simpler – All pop received; only some adopted…? Working on equivalence… • Hierarchy of data collection preference – behavioral • Random assignment • Pre/post/control • Pre/post, etc. • Data needs • Behavior ever adopted; frequency, quality, # HH/employees adopting • Whether behavior in place / operating; what share. If discontinued… • Date of discontinuation (failure / replacement / removal) of behavior • Battery bracketing discontinuation date.. • Why • Confirmation of influence, demographics / operating conditions and exogenous factors…

  19. NEXT STEPS

  20. MEASURE LIFE GUIDELINES: NEXT TIME • Next Time: • Additional elements of Risk (mostly savings shares so far) • Refined guidelines incorporating comments • EUL tables / numbers – thanks for contributions & leads!! • RUL • TDF • Schedule: Next call 11/8/11

  21. CONTACT • skumatz@serainc.com • 303/494-1178

  22. LAST TIME’S SLIDES (IN CASE THERE ARE QUESTIONS!)

  23. RTF – GUIDELINES FOR DEVELOPMENT FOR UES MEASURE LIFE Meeting 2 - Interim Discussion 9/23/2011 Skumatz Economic Research Associates, Inc. (SERA) The Cadmus Group

  24. TODAY’S AGENDA • 2:00   Introductions, Progress / status • 2:05 Schedule / Plan review • 2:10 Walk through “Classes” • 2:20 Walk through Class 1 draft guidelines & discussion issues •  Edits plus direction for crafting the other classes • 3:10    Criteria for evaluating EULs / comments • 3:15 Next Steps, leads / contacts on reports, EULs, interviewees • 3:20 Other comments / discussion • 3:30 Adjourn SERA and Cadmus Group

  25. SCHEDULE / PLAN • Another document • Two key outcomes • Guidelines • EUL values • Lists activities / deliverables • Meetings / calls • Comments? • Call 10/17(M), 10-11:30; materials 10/11 • call 11/8(T), 2:30-4; materials 11/1 • In-person 12/6(T), 11-1; draft review; materials 11/28 • In-person 12/13 (T), RTF meeting ; materials 12/9 SERA and Cadmus Group

  26. CLASSIFYING THE EUL WORK • “Buckets” for categories of measures / issues • Two uses / states of guidelines • Retrospective - Review of existing EUL study OR conducting one • Prospective – new measure, planning purposes – or won’t have an EUL… • Three measure groups • Hardware / measure (CFL, HVAC, motor, etc.) • Behavior influencing measure (replacing filters, temp set) • Behavioral (retrocx, light cycling, water temp) • Where does software fall…? • Other issues • RUL, TDF SERA and Cadmus Group

  27. CLASSES FOR EUL GUIDELINES All classes NOT equally important / full focus SERA and Cadmus Group

  28. GUIDELINES – CLASS 1: HARDWARE, RETROSPECTIVE • Overview / reporting • Clarity on measure / end use / sector, years, authors, utility, territory • Summary of method, sample size • Discussion issues: • Specific Discussion Issues: • How prescriptive – less than instruction manual…? Level of expertise to assume? SERA and Cadmus Group

  29. GUIDELINES – CLASS 1: DATABASE MANAGEMENT • Database management • Specify data sources, how ID’d / validation • Data acquisition & attrition through steps to analysis – counts / sources / rationale • Data quality checks / procedures • Data not used • Specific Discussion Issues: • None SERA and Cadmus Group

  30. GUIDELINES – CLASS 1: SAMPLING & DATA COLLECTION • Sampling & Data Collection • Sampling procedures & protocols: • Frame, sampling / basis / stratification. • Sampling entity – premise, measure, etc. & justification. Compare to others; ID & justify sample size. • Survey instrument: • Key questions: in place & operating, failure/removal/replacement date (or brackets for dates); why removed; confirm install; basic demographics; exogenous variables relevant to measure. Include instrument. SERA and Cadmus Group

  31. GUIDELINES – CLASS 1: SAMPLING & DATA COLLECTION • Sampling & Data Collection • Survey data collection: • determine / justify data collection method (phone, etc.). Pilot. • List response rates, refusal tabulations / possible refusal surveys. Tests for bias. • Statistical descriptions: • Descriptive statistics for key variables used; comparisons. • Specific Discussion Issues: • Measure, premise, etc. How prescriptive? SERA and Cadmus Group

  32. GUIDELINES – CLASS 1: DATA ANALYSIS, CONTEXT, RESULTS • Analysis, context, results • Outliers, influential points, missing data and treatment. Screening data for inclusion in final dataset / why, how many. • Exogenous variables affecting lifetimes: hours, business types, CZ / HDD / CDD, other • Modeling: • Modeling software / procedures used; present model statistics • Alternative models / distributions tested, relative performance • Omitted / included variables & justification; addressing variation, exogenous variables • Results / standard errors / confidence band. SERA and Cadmus Group

  33. GUIDELINES – CLASS 1: SAMPLING & DATA COLLECTION • Analysis, context and results • Context and comparisons for results • Precision, measurement frequency (below) • Specific Discussion Issues: • Acceptable reasons to omit? • Confidence requirements – 80/10? 95/5? Needs for reliability. History. Cost. • Exogenous variables – rarely used. List the ones to be included – or types by end use? • Is 50% in place and operable right definition? What are uses? Is that appropriate value? • Frequency – vary by expected life range? Vary by business turnover? Vary by contribution, uncertainty, TRC sensitivity? • Adjustments in Measure life by business / table? SERA and Cadmus Group

  34. USING EXISTING EUL STUDIES / VALUES • ID gaps; recommendations to fill on priority basis: • Contribution to council savings • Sensitivity of TRC to EUL • Number of existing studies, other • Values in appendix? SERA and Cadmus Group

  35. COUNT SO FAR – REVIEW OF EXISTING EUL STUDIES Sectors • Ag – 16 • Com’l – 17 • Industrial – 21 • Residential – 33 • Low income – 0 • Other – 2 • Trails not strong for accepted EULs End Uses • Lighting – 36 • HVAC – 25 • Motors / drives – 10 • Pump – 13 • Water – 7 • Cooking – 6 • Refrig – 6 • Process – 16 • Misc - 3 SERA and Cadmus Group

  36. ASSESSING QUALITY OF EXISTING EUL STUDIES • Same guidelines as Class 1 (since only hard measures thus far…) • Overview • Database management • Sampling & data collection • Data screening / modeling & analysis / context  implications and results • Looking for high quality in sample size, data coll’n, model quality / flexibility, consideration of OTHER variables SERA and Cadmus Group

  37. NEXT STEPS • Revise Guidelines for Class 1 per today’s comments • Content, style, length, detail – use for guiding other classes • Continue expert and background interviews / reports (data request) • Working on Hardware + Behavioral and Behavioral • Review of prioritization • Portfolio, sensitivity • Continue on EUL values assessment • Look for patterns / • Help inform prospective guidelines • Expect calls for info! • Next meeting Call 10/17(M), 10-11:30; materials 10/11 • More classes of Guidelines (1-4) SERA and Cadmus Group

More Related