Ex Ante Review Overview

1. Ex Ante Review Overview Energy Division Staff and Contractors Energy Efficiency Industrial/Agricultural Programs and Portfolio Forecasting California Public Utilities Commission September 11, 2014

2. Ex Ante Review Overview • Speaker: • Jaclyn Marks • Supervisor, Energy Efficiency Industrial/Agricultural Programs and Portfolio Forecasting • California Public Utilities Commission, Energy Division

3. Overview of Presentation and Topics Purpose: Information Exchange, Q & A encouraged 1. Overview of CPUC oversight of its authorized EE activities 2. Regulatory background on ex ante review process (emphasis on Custom projects) 3. Commission staff’s expectations when performing EAR

4. CPUC Oversight of IOU/CCA/REN Energy Efficiency Programs • Speaker: • Jaclyn Marks

5. CPUC Organization

6. CPUC Organization • President and four Commissioners • Governor appoints and Legislature confirms, are independent “Decision Makers” • Administrative Law Judges • Hired as state employees but operate independently as “Decision Makers” • Division Management • Hired as state employees – Supervisors report to Program Managers who report to Division Director who reports to Executive Director • Commission staff • Hired as state employees and report to a supervisor for the area they oversee

7. Energy Efficiency Organizational Chart

8. CPUC Regulatory Process • Rulemaking Proceedings • The main regulatory process within which the CPUC authorizes and oversees the EE activities – this is a “legal” process whereby Decisions and Rulings are issued based on a legal and policy analysis of documents placed into the record • Commission Decisions and Resolutions plus Decision Maker Rulings are the main vehicles for providing direction and making or clarifying/adjusting policy • Record is established by Decision Maker rulings and official proceeding meetings and hearings

9. CPUC Regulatory Process • Proceedings (continued) • Commission creates a proceeding via a Decision • President assigns proceeding to a Commissioner • Chief ALJ assigns one or more ALJs to the Proceeding • Party status requested via official request filing which can be accepted or denied by Ruling or Decision • Parties to the proceeding can have input invited into the proceeding record • The IOUs, CCAs, and RENs submit their proposed activities via applications

10. CPUC Regulatory Process • Commission Decisions for EE guidance and authorization • D.09.09.047 authorized 2010-2012 programs and budgets including creating the ex ante review process • ALJ Ruling dated 11/18/2009 in A.07-08-021 created the non-DEER deemed workpaper review process • D.11-07-030 (the “ex ante” Decision) in response to IOU PfM • Clarified many ex ante policy issues such as early retirement/dual baseline • Created the custom measure and project review process (Attachment B) • D.12-05-015 gave guidance on 2013-2014 EE applications • Provided extensive clarifications and amplification of ex ante policy • D.12-11-015 authorized 2013-2014 programs and budgets extended the ex ante review process unchanged • D.11-XX-XXX expected soon will authorize 2015 portfolios

11. CPUC Oversight of EE • Questions?

12. Regulatory Background on Ex Ante Review Process (emphasis on Custom projects) • Speaker: • Jeff Hirsch • James J. Hirsch & Associates • Prime Contractor – Database for Energy Efficiency Resources (DEER) and Ex Ante Regulatory Support – CPUC 12PS5125

13. Ex Ante Contract Team • Primary players • JJ Hirsch & Assoc. • Hirsch, Gandhi, Madison, Reeves, Addison, Hill, Maddox, Gates, Sweek, and others • Navigant for stakeholder interaction organization support • Energy Metrics (Rothenberg) for custom review • Itron for custom review • ERS for custom review • Other for DEER, workpapers and regulatory support

14. Ex Ante Team Activities • DEER • Assumptions, methods and values • Updated for Codes, Standard and Regulation changes as well as to incorporation latest evaluation results and related research • Non-DEER workpaper review and approval • New measures and updates to existing measures • Assumptions, methods, baselines, costs, EUL/RUL, NTG • Custom measure and project review – main topic here • Other regulatory support

15. Ex Ante Interrelations Custom Projects Site-specific; Values developed at completion Non-DEER Workpapers Measures not in DEER Sub-selection of submission Ex Ante Values Best available information and extrapolation DEER methodology DEER Common EE measures; Savings estimates and parameters

16. Why Ex Ante Review? • Ratio of ex post to ex ante values needs improvement. • Gross has fallen from ~.9 in 2000 to ~.6-.7 in 2008 • Free rider percent has remained and 40%-50% or increased • Evaluations have indicated there are areas where policy is not appropriately implemented and that assumptions, methods and data utilized are not the most appropriate. • Commission staff has an oversight responsibility to ensure adopted policy is being followed .

17. Why review before agreement with customer is signed not later? • Cost effectiveness involves both the savings estimates as well as the costs • Improving savings estimates can result in more effective use of incentive $ • History of reaction to ex post evaluation • Review and oversight moved into the process to assist CPUC and PA’s in finding issues and providing guidance designed to improve results • Provide real time oversight to accelerate improvement

18. Objectives of the ex ante review process • Improve the accuracy and reliability of the Energy Efficiency portfolio overall energy savings and cost effectiveness estimates • Foster ongoing improvements to the quality and consistency the portfolio implementers’ own internal due diligence activities relating to ex ante values

19. Due Diligence • Due Diligence is an investigation of a business or person prior to signing a contract, or an act with a certain standard of care, the process through which a potential investor can evaluate a major planned investment for its cost, benefits and risk. • We are all involved in the due diligence effort.

20. CPUC DECISION 11-07-030 • “Ex Ante” Decision adopted by CPUC in July 2011 and re-affirmed by two subsequent Decisions in 2012 for use in 2013 and beyond • Appendix B describes the custom project EAR process • Sets minimum project documentation requirements • Applies to Commission Staff (CS) selected and non selected projects • Allows review prior to customer agreement so as to approve all values (savings, incentives, life, costs, etc.) used for EE savings “claims” and cost-effectiveness calculations • Allows reviews of non-selected projects later (correct errors and set prospective requirements)

21. CPUC DECISION 12-05-015 • Amplified and added clarifications and details to direction and policy in D.11-07-030 • Clarification of project classifications (NC/NR/ER/etc.) • Details of baseline definitions and selection including requirements for use of early retirement classification • Details use of EAR “free rider” reviews to provide guidance and set rules to improve “net” performance • Details of TRC cost calculation and how that values can limit incentives • Clarification on expectation for above code/ISP activities and no “like” or “regressive” baselines

22. Process of EAR for custom projects • CMPA-Custom Measure and Project Archive • Summary list of custom projects uploaded to CMPA, bi-weekly basis (can be more often). • Commission Staff reviews list, selects projects- selected projects may be pre-application stage through claims stage. • Utility uploads project documents to the CMPA, email notification sent to various parties. • Commission Staff dispositions are posted on the CMPA, email notification sent to various parties

23. Ex Ante Review Reality • Commission Staff review only a small percentage of custom projects. • PAs generally appear to put more effort into projects Commission Staff pick for review than non-selected projects. • Because PAs appear to put additional emphasis on “picked projects”, Commission Staff reviewed projects may not be representative of the full project population.

24. Future direction of the custom EAR process • Improving the accuracy and reliability of the Energy Efficiency portfolio overall energy savings and cost effectiveness rests with the PAs reviewers. • Commission Staff will be increasing efforts to work with PA reviewers and monitor the progress of changes in the PA review/due diligence process. • Reviewed projects may be pre-application stage through claims stage. • Commission Staff will focus more reviewers activities to assess the area where due diligence is inadequate.

25. Questions?

26. Commission staff’s expectations when performing EAR • Speaker: • Keith Rothenberg • Energy Metrics • Subcontractor to JJH&A for custom EAR

27. Overview • Role of the Commission staff representative reviewer • Documentation Expectations • What we expect and what we receive • Different project phases and purposes of Commission staff review • Questions

28. Role of the Commission staff reviewer • Not to supplement the Utility reviewer. • Reviewing the Utility reviewers’ due diligence efforts. • We rely on the Utility reviewers to accomplish the Commission’s goals: • Improving the reliability of the savings estimates • Ensure project compliance with CPUC policy

29. What complicates documentation requirements? • Q. For retrofit project types, why don’t we simply calculate the energy savings directly associated with replacing the existing system/equipment and use that value for the savings impacts? • A. In some cases we do, however CPUC Policy attempts to balance the ratepayer’s interest and the customer’s interest with the realities of the circumstances driving the project. • Result: Policy has created considerations for free ridership, rules associated with code compliance, ISP assessments, TRC considerations, fuel switching considerations, dual baseline rules, EUL, RUL, etc.

30. Project Types, Baselines, Life

31. Early Retirement (AKA “Program-Induced Early Retirement”) • Requirements described in guidance document “Early Retirement Using Preponderance of Evidence” • Preponderance of evidence supporting that the program induced the replacement. • Approved dual baseline energy savings calculation approach. • Full measure cost, incremental measure cost for the second baseline. • Proposed RUL of the existing equipment supported by evidence. • Measure EUL with justification/source.

32. Examples of the expected documentation • Project type clearly described, not inferred. • Documentation to support baseline assignment • Facility description • Is there co-generation on site? • Project description, including relevant associated systems. • Existing system controls and operating status. • Existing system output capacities – current output and maximum/design capacity. • Proposed modifications with schematic as applicable. Preliminary savings calculations and supporting data with documentation to ensure replicability.

33. Expected documentation (Cont’d)- Calculation Method • Written calculation methodology. • Precise step-by-step equations. • Associate the proposed methodology with specific equipment and systems. • Energy savings principle for each measure should be discussed. • Comprehensive and complete leaving only the final verified variables and data to be determined after project completion. • Generic methodology is not acceptable. • Documents that require a reviewer to hunt through spreadsheets cell by cell to investigate the calculation methodology are unacceptable.

34. Expected documentation (Cont’d)- M&V Plans • M&V plans, reports when required. • M&V required when impacts are difficult to estimate by calculation alone. • System diagrams with measurement points. • Concise descriptions including measurement period, measurement interval, measurement equipment. • Measurement equipment accuracy, result uncertainty. • May be necessary to collect additional baseline data before the project commences. • Raw data archives.

35. Expected documentation (Cont’d) • Manufacturer’s cut sheets when used to estimate ex ante savings or when needed to ensure replicability. • Fuel switching considerations and any required analysis. • HVAC or process interactive effects values and methods used to develop ex ante savings values. • Interactions between multiple measures that act to increase or decrease savings-parametric analysis. • Dual baseline analysis for ER projects. • EUL/RUL value, analysis or source.

36. Expected documentation (Cont’d) • Pre/post production output data when used in savings calculations and the source of such records. • Pre-installation inspection report. • Post-installation inspection report. • Other Utility review documents. • Installation report where M&V performed. • Utility or implementer program manual (a single archive of these documents should be referenced).

37. Pre-Installation Review • Perform a “parallel” review with the Utility • What is parallel review supposed to be? • Provide input to the Utility on the estimated custom measure or project ex ante savings • Set precedents for future projects

38. Post-Installation Review • Continued opportunity for CS to review and provide input on the accuracy of ex ante values. • Review Utility compliance with earlier project specific guidance. • “True-up” estimates with site specific M&V data/results. • Set precedents for future projects.

39. Utility Claim Review • Conduct a review of energy savings and cost-effectiveness parameters for custom projects included into the Utility Quarterly Claim. • Review project files to ensure the CPUC directed archiving of documentation is being followed. • Appropriate default realization rates were applied to ex ante gross savings estimates for projects that were not reviewed.

40. Utility Claim Review (Cont’d) • Recommendations made by Commission staff for reviewed projects were accurately reflected in the claim. • Guidance provided in CS reviewed projects being followed in projects that were not previously reviewed by CS.

41. Early Opinions • Utilities may submit to Commission staff a request for an early Commission staff review or opinion on a specific issue. • Appropriate for policy grey areas. • Eligibility issues. • Other areas where there can be uncertainty regarding policy or Program compliance issues. • This should be used when either a utility or an implementer has concerns or is uncertain how a particular project might be handled by CS review

42. Summary of CPUC Reviewers’ Role • Not to supplement the administrator reviewer but to ensure proper administrator due diligence is being followed. • Ensure program rules are being followed. • Ensure CPUC policy is being followed. • Verify that utility staff and contract reviewers perform reviews in an appropriate manner balancing ratepayer and customer interests. • Ensure that all required documentation is in the project file whether it is selected or not for ex ante review by CPUC staff reviewers.

43. Wrap Up • Our work plays an important role in the implementation of the CPUC authorized ratepayer funded programs. • As reviewers, we are all engaged in a due diligence effort on behalf of the rate payers as well as the Utility customers. • The CPUC has authorized ex ante review to improve the accuracy and reliability of the Energy Efficiency portfolio overall energy savings and cost effectiveness estimates.

44. Wrap Up • Commission staff are relying on IOU reviewers to achieve this goal. We are interested in working with you. • There are changes to the EAR process that will allow CS to assess the progress in this area: categorization of comments, tracking of comments with project phases, project types, reviewers, review firms, etc.

45. Questions??

46. Your comments on the EAR process? • How can CS improve the process? • What works and does not work? • What issues do Utility reviewers have?

47. Standardization of required documentation? • Would a minimum required documentation template that is required to be included for every project be helpful? • What is included in the application review packet that comes to your desk?

48. What additional information would be useful for Utility reviewers?

49. Documents Available Online 1. Industry Standard Practice Guidance Document 2. Project basis as Early Retirement/Replace-on-burnout /Normal Replacement/New Construction/Add-on Retrofit and Remaining/Effective Useful Life, and Preponderance of evidence Guidance Document http://www.cpuc.ca.gov/PUC/energy/Energy+Efficiency/Ex+Ante+Review+Custom+Process+Guidance+Documents.htm 3. Commission Decisions and Rulings establishing policy and processes.