Ex Ante Review Overview

1. Ex Ante Review Overview Energy Division Staff and Contractors Energy Efficiency Industrial/Agricultural Programs and Portfolio Forecasting California Public Utilities Commission November 13, 2014

2. Ex Ante Review Overview • Speaker: • Peter Lai • Senior Utilities Engineer, Energy Efficiency Industrial/Agricultural Programs and Portfolio Forecasting • California Public Utilities Commission, Energy Division • Documents for this presentation: • ftp://deeresources.com/ExAnteProcess/

3. Energy Efficiency Organizational Chart

4. Ex Ante Commission Staff Team • Primary players and their focus areas • Jacklyn Marks • Supervisor, Energy Efficiency Industrial/Agricultural Programs and Portfolio Forecasting • Peter Lai • Custom Measures and Projects • Katie Wu • Workpapers • Aaron Lu • DEER

5. Ex Ante Contract Team • Primary players • JJ Hirsch & Assoc. (Prime Contractor) • Hirsch, Gandhi, Madison, Reeves, Addison, Hill, Maddox, Gates, Sweek, and others • Navigant for stakeholder interaction organization support • Energy Metrics (Rothenberg) for custom review • Itron for custom review • ERS for custom review • Others for DEER, workpapers and regulatory support

6. Regulatory Background on Ex Ante Review Process • Speaker: • Jeff Hirsch • James J. Hirsch & Associates • Prime Contractor – Database for Energy Efficiency Resources (DEER) and Ex Ante Regulatory Support – CPUC 12PS5125

7. CPUC Oversight of IOU/CCA/REN Energy Efficiency Programs

8. CPUC Organization

9. CPUC Organization • President and four Commissioners • Governor appoints and Legislature confirms, are independent “Decision Makers” • Administrative Law Judges • Hired as state employees but operate independently as “Decision Makers” • Division Management • Hired as state employees – Supervisors report to Program Managers who report to Division Director who reports to Executive Director • Commission staff • Hired as state employees and report to a supervisor for the area they oversee

10. CPUC Regulatory Process • Rulemaking Proceedings • The main regulatory process within which the CPUC authorizes and oversees the EE activities – this is a “legal” process whereby Decisions and Rulings are issued based on a legal and policy analysis of documents placed into the record • Commission Decisions and Resolutions plus Decision Maker Rulings are the main vehicles for providing direction and making or clarifying/adjusting policy • Record is established by Decision Maker rulings and official proceeding meetings and hearings

11. CPUC Regulatory Process • Proceedings (continued) • Commission creates a proceeding via a Decision • President assigns proceeding to a Commissioner • Chief ALJ assigns one or more ALJs to the Proceeding • Party status requested via official request filing which can be accepted or denied by Ruling or Decision • Parties to the proceeding can have input invited into the proceeding record • The IOUs, CCAs, and RENs submit their proposed activities via applications

12. CPUC Regulatory Process • Commission Decisions for EE guidance and authorization • D.09.09.047 authorized 2010-2012 programs and budgets including creating the ex ante review process • ALJ Ruling dated 11/18/2009 in A.07-08-021 created the non-DEER deemed workpaper review process • D.11-07-030 (the “ex ante” Decision) in response to IOU PfM • Clarified many ex ante policy issues such as early retirement/dual baseline • Created the custom measure and project review process (Attachment B) • D.12-05-015 gave guidance on 2013-2014 EE applications • Provided extensive clarifications and amplification of ex ante policy • D.12-11-015 authorized 2013-2014 programs and budgets extended the ex ante review process unchanged • D.11-XX-XXX expected soon will authorize 2015 portfolios

13. CPUC Oversight of EE • Questions?

14. Ex Ante Team Activities • DEER • Assumptions, methods and values • Updated for Codes, Standard and Regulation changes as well as to incorporation latest evaluation results and related research • Non-DEER workpaper review and approval • New measures and updates to existing measures • Assumptions, methods, baselines, costs, EUL/RUL, NTG • Custom measure and project review – main topic here • Other regulatory support

15. Ex Ante Interrelations Custom Projects Site-specific; Values developed at completion Non-DEER Workpapers Measures not in DEER Sub-selection of submission Ex Ante Values Best available information and extrapolation DEER methodology DEER Common EE measures; Savings estimates and parameters

16. Why Ex Ante Review for Custom? • Ratio of ex post to ex ante values needs improvement. • Gross has fallen from ~.9 in 2000 to ~.6-.7 in 2008 • Free rider percent has remained at 40%-50%, or increased • Evaluations have indicated there are areas where policy is not appropriately implemented and that assumptions, methods and data utilized are not always the most appropriate. • Commission staff has an oversight responsibility to ensure adopted policy is being followed.

17. Why review before agreement with customer is signed not later? • Cost effectiveness involves both the savings estimates as well as the costs • Improving savings estimates can result in more effective use of incentive $ • History of reaction to ex post evaluation • Review and oversight moved into the process to assist CPUC and PA’s in finding issues and providing guidance designed to improve results • Provide real time oversight to accelerate improvement

18. Objectives of the ex ante review process • Improve the accuracy and reliability of the Energy Efficiency portfolio overall energy savings and cost effectiveness estimates • Foster ongoing improvements to the quality and consistency the portfolio implementers’ own internal due diligence activities relating to ex ante values

19. Due Diligence • Due Diligence is an investigation of a business or person prior to signing a contract, or an act with a certain standard of care, the process through which a potential investor can evaluate a major planned investment for its cost, benefits and risk. • We are all involved in the due diligence effort.

20. CPUC DECISION 11-07-030 • “Ex Ante” Decision adopted by CPUC in July 2011 and re-affirmed by two subsequent Decisions in 2012 for use in 2013 and beyond • Appendix B describes the custom project EAR process • Sets minimum project documentation requirements • Applies to Commission Staff (CS) selected and non selected projects • Allows review prior to customer agreement so as to approve all values (savings, incentives, life, costs, etc.) used for EE savings “claims” and cost-effectiveness calculations • Allows reviews of non-selected projects later (correct errors and set prospective requirements)

21. CPUC DECISION 12-05-015 • Amplified and added clarifications and details to direction and policy in D.11-07-030 • Clarification of project classifications (NC/NR/ER/etc.) • Details of baseline definitions and selection including requirements for use of early retirement classification • Details use of EAR “free rider” reviews to provide guidance and set rules to improve “net” performance • Details of TRC cost calculation and how that values can limit incentives • Clarification on expectation for above code/ISP activities and no “like” or “regressive” baselines

22. Process for Pre-Claim Custom EAR • CMPA-Custom Measure and Project Archive • Summary list of custom projects uploaded to CMPA, bi-weekly basis (can be more often). • Commission Staff reviews list, selects projects- selected projects may be pre-application stage through claims stage. • Utility uploads project documents to the CMPA, email notification sent to various parties. • Commission Staff dispositions are posted on the CMPA, email notification sent to various parties

23. Ex Ante Review Reality • Commission Staff review only a small percentage of custom projects. • PAs generally appear to put more effort into projects Commission Staff pick for review than non-selected projects. • Because PAs appear to put additional emphasis on “picked projects”, Commission Staff reviewed projects may not be representative of the full project population.

24. Future direction of the custom EAR process • Improving the accuracy and reliability of the Energy Efficiency portfolio overall energy savings and cost effectiveness rests with the PAs reviewers. • Commission Staff will be increasing efforts to work with PA reviewers and monitor the progress of changes in the PA review/due diligence process. • Reviewed projects may be pre-application stage through claims stage. • Commission Staff will focus more reviewers activities to assess the area where due diligence is inadequate.

25. Questions?

26. Commission staff’s expectations when performing EAR • Speaker: • Keith Rothenberg, Energy Metrics • Subcontractor to JJH&A for custom EAR

27. Overview • Role of the Commission staff representative reviewer • Documentation Expectations • Common Deficiencies • Questions and discussion

28. Role of the Commission staff reviewer • Not to supplement the PA reviewer. • Reviewing the PA reviewers’ due diligence efforts. • We rely on the PA reviewers to accomplish the Commission’s goals: • Improving the reliability of the savings estimates • Undertake ISP studies • Enhance calculation methods as needed • Set appropriate M&V requirements • Ensure project compliance with CPUC policy

29. What complicates documentation requirements? • Documentation must support later examination by those not involved in the project implementation • Establishing the correct project type & baseline • In many cases the eligible savings don’t match the “on-meter” estimates • Particularly challenging in cases where code does not apply, and ISP defines the baseline • Project type selection and supporting documents • ISP studies and C&S/Reg updates are needed • ER documentation support is under-estimated • Calculation method and support • Should be project specific for custom projects • M&V requirements tied back to project type and calculation method • Pre/post installation measurement may be needed • Relate back to calculation method

30. Project Types, Baselines, Life, Cost Basis

31. Expected Documentation Focus: Calculation Method • Written calculation methodology. • Precise step-by-step equations. • Associate the proposed methodology with specific equipment and systems. • Energy savings principle for each measure should be discussed. • Comprehensive and complete leaving only the final verified variables and data to be determined after project completion. • Generic methodology is not acceptable. • Documents that require a reviewer to hunt through spreadsheets cell by cell to investigate the calculation methodology are unacceptable.

32. Expected Documentation Focus: M&V Plans • M&V plans, reports when required. • M&V required when impacts are difficult to estimate by calculation alone. • M&V plan should be tied to calculation method uncertain parameters • System diagrams with measurement points. • Concise descriptions including measurement period, measurement interval, measurement equipment. • Measurement equipment accuracy, result uncertainty. • May be necessary to collect additional baseline data before the project commences. • Raw data archives.

33. PA Reviewer ID Deficiencies • Submitted Application Package is often incomplete • Poor project description • Insufficient documentation • Lack of internal quality control (QC): Inconsistencies • Submitted calculations are insufficient, inaccurate, or hard to follow. • Applications do not provide equipment vintage, EUL or RUL values • Baselines not properly defined and do not address applicable codes, Federal/State regulations, and industry standard practice • Incremental measures costs not provided • Failed site inspection because equipment not operating or already has been replaced • Selected M&V plan is not appropriate for retrofit type

34. PA Reviewer ID Deficiencies • Measure Category (ER, ROB, NR, REA, NEW) • Measure Baseline • Monitoring and Verification Plan • Cost Basis • Calculation Methodology • Supporting Documentation

35. Discussion: How to make the EAR process more effective • How can CS improve the process? • What works and does not work? • What issues do PA reviewers have? • How can CS work with PAs to increase the average quality of project application documentation? Would tutorials be useful? • Would a minimum required documentation template that is required to be included for every project be helpful?

36. What additional information would be useful for Utility reviewers?

37. Summary of CPUC Reviewers’ Role • Not to supplement the administrator reviewer but to ensure proper administrator due diligence is being followed. • Ensure program rules are being followed. • Ensure CPUC policy is being followed. • Verify that PA staff and contract reviewers perform reviews in an appropriate manner balancing ratepayer and customer interests. • Ensure that all required documentation is in the project file whether it is selected or not for ex ante review by CPUC staff reviewers.

38. Wrap Up • Our work plays an important role in the implementation of the CPUC authorized ratepayer funded programs. • As reviewers, we are all engaged in a due diligence effort on behalf of the rate payers as well as the Utility customers. • The CPUC has authorized ex ante review to improve the accuracy and reliability of the Energy Efficiency portfolio overall energy savings and cost effectiveness estimates.

39. Wrap Up • Commission staff are relying on PA reviewers to achieve this goal. • Commission staff will be increasing efforts to work with PA reviewers and monitor the progress of changes in the PA review/due diligence process. • We are interested in working with you.

40. Documents Available Online 1. Industry Standard Practice Guidance Document 2. Project basis as Early Retirement/Replace-on-burnout /Normal Replacement/New Construction/Add-on Retrofit and Remaining/Effective Useful Life, and Preponderance of evidence Guidance Document http://www.cpuc.ca.gov/PUC/energy/Energy+Efficiency/Ex+Ante+Review+Custom+Process+Guidance+Documents.htm 3. Commission Decisions and Rulings establishing policy and processes.

41. Additional Details

42. Focus: Early Retirement (AKA “Program-Induced Early Retirement”) • Requirements described in guidance document “Early Retirement Using Preponderance of Evidence” • Preponderance of evidence supporting that the program induced the replacement. (not that it can survive for 1 yr.) • Approved dual baseline energy savings calculation approach. • Full measure cost, incremental measure cost for the second baseline. • Proposed RUL of the existing equipment supported by evidence. • Measure EUL with justification/source.

43. Examples of the expected documentation • Project type clearly described, not inferred. Documentation to support baseline assignment.(is ISP needed?) • Facility description (Is there co-generation on site?) • Project description, including relevant associated systems and estimated project costs (incremental/full) • Existing system controls and operating status. • Existing system output capacities – current output and maximum/design capacity. • Proposed modifications with schematic as applicable. • Alternatives and options considered for project.

44. Expected documentation • Manufacturer’s cut sheets when used to estimate ex ante savings or when needed to ensure replicability. • Fuel switching considerations and any required analysis. • HVAC or process interactive effects values and methods used to develop ex ante savings values. • Interactions between multiple measures that act to increase or decrease savings-parametric analysis. • Dual baseline analysis for ER projects. • EUL value, analysis or source. For ER, RUL value and supporting analysis and evidence.

45. Expected documentation (Cont’d) • Pre/post production output data when used in savings calculations and the source of such records. • Pre-installation inspection report. • Post-installation inspection report. • Other Utility review documents. • Installation report where M&V performed. • Utility or implementer program manual (a single archive of these documents should be referenced).

46. Types of Ex Ante Review

47. Pre-Installation Review • Perform a “parallel” review with the Utility • What is parallel review supposed to be? • Provide input to the Utility on the estimated custom measure or project ex ante savings • Set precedents for future projects

48. Post-Installation Review • Continued opportunity for CS to review and provide input on the accuracy of ex ante values. • Review Utility compliance with earlier project specific guidance. • “True-up” estimates with site specific M&V data/results. • Set precedents for future projects.

49. Utility Claim Review • Conduct a review of energy savings and cost-effectiveness parameters for custom projects included into the Utility Quarterly Claim. • Review project files to ensure the CPUC directed archiving of documentation is being followed. • Appropriate default realization rates were applied to ex ante gross savings estimates for projects that were not reviewed. • Application of review results to gross and net parameters.

50. Utility Claim Review (Cont’d) • Recommendations made by Commission staff for reviewed projects were accurately reflected in the claim. • Guidance provided in CS reviewed projects being followed in projects that were not previously reviewed by CS.