Comprehensive Usage Reduction Program Evaluation

1. Comprehensive Usage Reduction Program Evaluation Affordable Comfort 2007 Jacqueline Berger

2. Evaluation Objectives • Determine the cost-effectiveness of WRAP • Develop standard questions so PPL can measure the same criteria in future evaluations • Comply with the PUC Order 2

3. Evaluation Questions • What are the goals and how are they met? • What are the admin costs? How can they be lowered? • How effective is program recruitment? • Is the audit mechanism effective? • Is the list of program measures comprehensive? 3

4. Evaluation Questions • Is the education process cost-efficient and effective? • What is the level of post-inspection and is it appropriate? • Does PPL coordinate WRAP with other weatherization programs? • What are the energy savings? 4

5. Evaluation Questions • What is the cost-effectiveness of the various agencies? • Does PPL provide adequate support and training for contractors? 5

6. Evaluation Design • Background Research • Review of Specifications and Procedures • Contractor Survey • Baseload Observations • Full Cost Observations and Inspections 6

7. Evaluation Design • Customer Survey • Usage Impacts • Payment Impacts 7

8. Background Research • Goal – develop a complete understanding of the WRAP procedures and implementation. • Activities • Interview WRAP managers and staff • Review program documentation • Review program statistics • Outputs • Understanding for research foundation • Documentation 8

9. Background Research • Key Findings • PPL has five managers who oversee WRAP and other low-income programs in their geographical area. • Advantage: ability to focus on needs in a particular geographic area • Disadvantage: many responsibilities – difficult to oversee work of contractors • PPL spends much effort to come within 4% of expenditure goal (based on PUC requirement). 9

10. Background Research • Key Findings • Affordability customers are prioritized. • Otherwise, jobs sent to contractors on a first come first served basis. • Program coordination barriers • Long waiting lists for WAP • Long waiting lists & requirements for gas programs • Customers who use gas and electric may not have high enough usage to qualify for either program 10

11. Background Research • Recommendations • Prioritize customers based on usage. • Track program coordination and provide incentives for contractors to coordinate services with other programs. • Continue to introduce technological improvements, such as the web-based measure reporting form. 11

12. Review of Specifications and Procedures • Goal – Determine potential effectiveness of measure selection, measure installation, and energy education • Activities – Assess procedures and forms: • Education specifications • Education forms • Written technical procedures and manuals • Measure installation rates 12

13. Review of Specifications and Procedures • Outputs • Recommendations for modifications to: • Education procedures • Education forms • Measure selection guidelines • Procedures manual 13

14. Review of Specifications and Procedures • Education Findings • All participants receive at least one on-site education visit • Follow-up education is provided at the time of the inspection or by phone • Remedial education provided to customers whose usage increases by at least 10% six months after service delivery 14

15. Review of Specifications and Procedures • Education Recommendations • Education should be provided at the time of the audit. The homeowner should be present. • Customer profile should collect information on potentially large opportunities for saving – use of dehumidifiers, use of second refrigerators, lights/appliances left on at all times. • Action form should prioritize actions by the potential for energy saving in the individual home. Should list top 3-5 actions with estimated $ savings. 15

16. Review of Specifications and Procedures • Technical Findings and Recommendations* • Review cost-effectiveness calculations in audit decision trees to reflect current estimates of costs and savings. • Refrigerators • CFLs • Water heater wraps and pipe insulation may be more cost effective than water heater replacement. *Blasnik & Associates. 16

17. Review of Specifications and Procedures • Technical Findings and Recommendations • Duct sealing in basements should be focused on safety and comfort. • Blower door guided air sealing: investigate why only done in 60% of full cost jobs. • WRAP standards and field guide: more concise program field guide with separate specifications for specific areas may be useful. 17

18. Contractor Survey • Goal - Assess contractor compliance with program procedures, and assess problems in program administration. • Activities • Develop contractor survey instrument • Determine survey sample • Send survey to contractors • Review completed surveys and contact respondents for additional information/clarifications 18

19. Contractor Survey • Outputs - Understanding of: • Contractor background and experience • Support and training provided to contractors • Usefulness of program forms • Program implementation procedures • Joint service delivery with WAP • Health and safety problems found in homes • Contractors’ quality control • Inspection issues 19

20. Contractor Survey • Outputs - Recommendations related to: • Program procedures • Contractor training and support • Inspection procedures 20

21. Contractor Survey 21

22. Contractor Survey 22

23. Contractor Survey 23

24. Contractor Survey 24

25. Contractor Survey 25

26. Contractor Survey 26

27. Contractor Survey 27

28. Contractor Survey 28

29. Contractor Survey 29

30. Contractor Survey 30

31. Contractor Survey • Recommendations • Revisit audit forms and determine whether they can be consolidated. • Require home walkthrough on all jobs. • Formalize a process to respond to action sheets. 31

32. Baseload Observations • Goals - Understand how well contractors address opportunities for baseload usage reduction and whether education is effectively provided. • Activities • Sample design and selection • Observation protocols • Conduct observations • Review findings and synthesis 32

33. Baseload Observations • Outputs - Recommendations for: • Additional contractor training • Additional quality control 33

34. Baseload Observations 34

35. Baseload Observations 35

36. Baseload Observations 36

37. Baseload Observations 37

38. Baseload Observations 38

39. Baseload Observations 39

40. Baseload Observations 40

41. Baseload Observations 41

42. Baseload Observations • Recommendations • Review WRAP requirements and expectations with contractors. • Provide education to contractors on the importance of 2-for-1 swaps in refrigerator replacement. • Conduct observation of baseload service delivery. 42

43. Full Cost Observationsand Inspections • Goals - To understand: • How well contractors address opportunities for electric usage reduction • Whether correct measures were selected • Extent to which energy education is provided. • Activities • Sample design and selection • Observation protocols • Review WRAP technical protocols • Conduction observations • Review findings and synthesis 43

44. Full Cost Observationsand Inspections • Outputs – Recommendations for: • Procedures • Training 44

45. Full Cost Observationsand Inspections 45

46. Full Cost Observationsand Inspections 46

47. Full Cost Observationsand Inspections 47

48. Full Cost Observationsand Inspections 48

49. Full Cost Observationsand Inspections 49

50. Full Cost Observationsand Inspections • Comments • They were on the right track. They were not afraid of working hard. • What he did was correct, but he should have done more air sealing. • He looked at all the right things and asked the homeowner to clarify things that were not obvious. 50