Part 4: M&V Plan Review

1. Part 4: M&V Plan Review Key Documents • Risk / Responsibility Matrix • M&V Plan Template • Detailed Guidelines for FEMP M&V Option Awww.eere.energy.gov/femp/financing/superespcs_mvresources.cfm

2. Phase 1 Phase 2 Phase 3 Phase 4 Phase 5 Phase 2: Project Development Contractor Responsibilities • Develop Risk & Responsibility Matrix • Develop M&V Overview • Explain and justify approach. Owner Responsibilities • Review Risk & Responsibility Matrix and M&V Overview and provide feedback

3. Phase 5 Phase 1 Phase 2 Phase 3 Phase 4 Phase 3: Negotiation and Award Contractor Responsibilities • Perform Detailed Energy Survey (DES) and document baseline information • Modify M&V plan to satisfy owner needs and desires owner Responsibilities • Review and approve M&V Plan • Witness and observe DES

4. Risk & Responsibility Allocation • How to allocate risks and responsibilities? Typically: • Performance: Contractor • Usage: owner • Financial: Shared • M&V approach should focus on: • Verifying performance • Characterizing usage • Minimizing uncertainty cost-effectively

5. Need to balance M&V rigor with project risk Measure things that need measuring Consider required precision Estimate (stipulate) where risks are acceptable Law of Diminishing Returns applies Typically, initial M&V costs will be 3 – 15% of the capital cost (see DO-2); annual M&V costs will be 3 – 15% of the savings (see DO-3) Cost Effectiveness

6. M&V Costs M&V Cost $ Incremental Value of information M&V Rigor

7. M&V Costs and Project Savings Source: Initial Applications of the FEMP Measurement & Verification Guidelines in Super ESPC Delivery Orders: Observations & Recommendations. Schiller Associates, 1999

8. Reviewing Measurement & Verification Plans for Federal ESPC Projects CAUTION: TECHNICAL EXPERTISE IS REQUIRED! • M&V Plan review requires qualitative assessment of engineering strategies Intimate understanding of project helpful: • Owner goals • Site specific issues • Agreed-upon risk allocation • Objectives and constraints for each ECM

9. Overview of 6 review steps STEP 1: Gather project documentation STEP 2: Rank ECMs based on savings STEP 3: Initiate review STEP 4: Evaluate M&V approaches STEP 5: Develop written report STEP 6: Ensure key issues are resolved

10. Overview of 8 review steps (cont) STEP 5: Evaluate M&V approach STEP 6: Develop report using review template STEP 7: Provide written review to Owner & DOE STEP 8: Ensure that action is taken on key issues

11. Step 1: Gather project documentation • Final proposal, including technical appendices • Note available documentation

12. Step 2: Rank ECMs based on savings • Identify all sources of savings for each ECM Rank ECMs on: • % total cost savings (energy & water, O&M) • % total commodity cost savings (energy & water) • Note % savings for ECMs • Focus on largest $ saving measures first • O&M generally verified separately from energy & water savings

13. Step 3: Initiate Review • Review Final Proposal • Sections of proposal other than the M&V plan may be required • Identify obvious deficiencies • Use graded approach in review efforts

14. Step 4: Evaluate M&V approaches Are M&V approaches appropriate? • Note source of savings for each ECM • Is M&V adequate for all sources? • Review Risk & Responsibility Matrix • Does M&V plan match proposed risk allocation?

15. Step 4: Evaluate M&V approaches (cont.) Evaluate quality of baseline developed • Were key variables measured? • Are assumptions reasonable and well documented? • Were performance characteristics recorded? • Are the energy and savings calculations correct? Reasonable? • Were utility or weather based models validated?

16. Step 4: Evaluate M&V approaches(cont.) Evaluate performance period activities: • What is the likelihood of success for this measure? • Higher risk = more rigorous M&V • Will key variables affecting ECM be measured? How often? • Will meaningful performance period data be used to calculate savings? • Sufficient to support guarantee? • If only post-install measurements are used, is this data likely to change? • Is the savings risk level appropriate? Aligned w/ R&R matrix? • Have all objectives and constraints for the project been considered? For each ECM? • Have O&M reporting responsibilities been adequately defined?

17. Step 4: Evaluate M&V approaches (cont.) Evaluate overall project assumptions • Are contracted energy rates based on actual rates? • Are marginal (not blended) rates used? • Are escalation factors based on latest NIST data or other credible source? Evaluate proposed M&V Costs • Do M&V costs align with planned activities?

18. Step 5: Develop written report • Transfer key notes & findings to written report • Summarize recommendations for each ECM in Summary of Findings

19. Step 6: Ensure issues are resolved • Send review to Owner staff • Note dates and names of recipients • Confirm actions are implemented to address recommendations • If not implemented, provide written explanation Documentation is critical in contracts that will most likely outlive staff

20. M&V Plan Reviews - Lessons Learned • Use a Savings/M&V Options Table • Prioritize the review • Improve proposed M&V approaches • Document recommendations

21. 1) Use a Savings/M&V Options Table • Gives a one-page picture that IDs all savings streams and associated M&V strategies • Helps spot missing M&V options • Helps you prioritize your M&V Plan review • Is easy to create, • Documents an important step in starting a review • Is a valuable reference any time M&V of the project is revisited • Helps focus future performance period reviews when details have been forgotten and the project begins providing measured results

22. 2) Prioritize Your Review • Use Savings/M&V Options Table to focus your review • ID where most of the project savings lie • Assess error potential with each savings stream to ID highest priority streams • Prioritize ECMs/savings streams in accordance with savings potential and error risk

23. Savings/M&V Options Table (in order of decreasing ECM savings)

24. 3) Improve proposed M&V approaches when needed • Change to more rigorous M&V options • Add measurements to minimize uncertainties (particularly for large savings streams where errors could be most significant) • Reduce M&V costs (switch M&V option, optimize sampling, etc.) • Optimize sampling: • minimizing number of measurements by focusing on the largest contributors • rotating sampling during the performance period • assuring random sampling where appropriate

25. 4) Document recommendations from M&V Plan reviews • Use to verify that ESCO/customer have addressed recommendations

26. Summary • GET IT RIGHT! M&V Plan reviews are critical: • Great protocols/guidance do not equal great M&V Plans • M&V Plans define project success • Weak M&V Plans = weak projects • Combine guidance with good engineering judgment to produce great M&V Plans

27. Summary (continued) • Think of reviews as a process such that your experience will lead to quality improvements • Use/develop review tools, they will: • Focus your efforts on a path through the myriad details involved • Help you minimize oversights • Provide useful reference material for reviews and a basis for QA improvements

28. More M&V resources: • M&V Guidelines, M&V Planning Tool, etc: • http://www.eere.energy.gov/femp/financing/superespcs_mvresources.cfm • Draft: Reviewing Measurement & Verification Plans for Federal ESPC Projects (and other M&V documents): • http://ateam.lbl.gov/mv/

29. Review & Discussion • Performance must be verified to guarantee savings • owner often assumes usage risk • Uncertainty is inherent in M&V • M&V costs need to be balanced against project risks

30. Review Questions • How do we measure savings? • When might an owner accept performance risk? Click hereto go to M&V Part 5