1 / 25

Marty Burns, marty@hypertek , SGIP Administrator Team (EnerNex), NIST marty@hypertek

UCAIug’s OpenADE Green Button Test Plan OpenADE Meeting at Fall Summit, New Orleans LA October 25 th 2012. Marty Burns, marty@hypertek.us , SGIP Administrator Team (EnerNex), NIST marty@hypertek.us Technical Champion, SGIP PAP20. Agenda.

davidk
Download Presentation

Marty Burns, marty@hypertek , SGIP Administrator Team (EnerNex), NIST marty@hypertek

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. UCAIug’s OpenADE Green Button Test PlanOpenADE Meeting at Fall Summit, New Orleans LA October 25th 2012 Marty Burns, marty@hypertek.us , SGIP Administrator Team (EnerNex), NISTmarty@hypertek.us Technical Champion, SGIP PAP20

  2. Agenda •  1)    8:30 – 10:00 CDT – Review of where we are and are going • We will kick off the meeting with a brief review of our activities and roadmap going forward. We can expect this to be short and will roll into the next session task as soon as we are done. •  2)    10:30 – 12:00 CDT – Identification of Complete set of Conformance Blocks • We have identified an initial set of conformance blocks in section 5 of the GreenButtonTestPlan.docx (http://osgug.ucaiug.org/sgsystems/OpenADE/Shared%20Documents/Testing%20and%20Certification/GreenButtonTestPlan/GreenButtonTestPlan.docx). These Conformance Blocks represent “orthogonal” sets of test requirements for the identified Green Button functionality. We want to convince ourselves that a) each block can contain a non-overlapping set of features that are tested for, and, b) each of the Application Profiles in section 3 can be defined in terms of sets of these Conformance Blocks. This first session will refine the list of Conformance Blocks. •  3)    13:00 – 14:30 CDT – Allocation of Conformance blocks to Application Profiles • Once the Conformance Blocks are identified, we will apportion them to the Application Profiles. Thus, testing for each Application Profile will be realized through the collection of tests in the Conformance Blocks identified. This session will produce this mapping and annotate the Application Profiles accordingly. •  4)    15:00 – 17:00 CDT -- Review/update of Help Desk Issues • Over the past couple of months, we have collected the Help Desk issues based on lessons learned from early adoption of Green Button (see http://osgug.ucaiug.org/sgsystems/OpenADE/Shared%20Documents/Testing%20and%20Certification/GreenButtonTestPlan/HelpDeskItems.docx). Most of these help desk items have been resolved with at least a proposed solution. This session will review these proposed solutions with the objective of “harvesting” a consensus around them to kick off a NAESB Standard update process for REQ.21 Energy Services Provider Interface, the underlying standard for Green Button.

  3. The Goal • If we are diligent and lucky, we will emerge from the meeting with the means to begin the standards update cycle at NAESB, and, to be in the test case allocation phase. The test case allocation phase will take the 160+ test cases from the GreenButtonTestCases.xlsm spreadsheet (see http://osgug.ucaiug.org/sgsystems/OpenADE/Shared%20Documents/Testing%20and%20Certification/GreenButtonTestPlan/GreenButtonTestCases.xlsm) and apportion them to the Conformance Blocks.

  4. OpenADE: Green Button Test Plan • What it is • Why we need one • Conformance testing versus Certification testing • Role of UCAIug and OpenADE

  5. Green Button • Green Button is • A policy • Whitehouse initiative • A brand • Green Button Logo implies specific capabilities • Interoperability standards and testing andcertification ensure those expectations are met • A set of technologies and associated standards • Green Button Download My Data (ESPI data file) • Green Button Connect My Data (ESPI automated exchange) • Green Button is an industry-led effort to provide electricity customers with access to their energy usage data via a "Green Button" on electric utilities' websites.

  6. An overnight success … years in the making OpenADE (Open Automated Data Exchange) • Early requirements effort for securely sharing energy information with third parties • NIST SGIP Priority Action Plan • Accelerates issue resolution and consensus development • OpenADE / SGIP inputs to NAESB standard REQ21 • Energy Services Provider Interface (ESPI) • Privacy: REQ22 Third Party Access to Smart-Meter-based Information • Flexible file format basis for Green Button • initial implementation use a subset of ESPI and energy usage information.

  7. SGIP PAP20: Green Button ESPI Evolution Roadmap Going Forward … SGIP PAP Activities: Coordinates with • SGIMC • PAP 20 • CSWG • SGTCC Requirements for standards and test criteria • Requirements for Green Button and ESPI Rollout Privacy assurance recommendations Test plan consistency Education on implementation strategies, standards, use cases and test criteria ITCA assurances • Facilitate/Coordinate SGIP interactions Cybersecurity recommendations for EUI exchanges Specification Deliverables: Standards(NAESB) REQ.18/WEQ.19 Maintenance Update ESPI Errata Update REQ.21/REQ.22 ESPI New Reqs Update International Version T&C(UCAIug) Green Button Download My Data Test Plan Green Button Connect My Data Test Plan UCAIug ITCA Implementation(EnergyOS) Green Button SDK OpenESPI

  8. Composition and Atom Feed 8 Syntactic Model Atom Feed View Information Model Profile View AtomFeed UsagePoint ServiceCategory links entrys 1 Entry:UsagePoint 0..* 0..* MeterReading IntervalBlock Entry:MeterReading 0..* Entry:ReadingType 1 0..* ElectricPowerUsageSummary IntervalReading ReadingType Entry:IntervalBlock 0..* 0..* ElectricPowerQualitySummary ReadingQuality Entry:ElectricPowerUsageSummary Note: This information is multidimensional. Many different reading types, summaries, and readings possible. i.e. not “flat”

  9. Roles and Relationships in Green Button Connect My Data Green Button Connect My Data Data Custodian Third Party Automated Transfer Web Service Provider Web Service Consumer Web Portal Web Portal One-time Authorization Retail Customer User (agent) Green Button Download My Data

  10. Why We Need Certification • Testing and Certification affords implementers confidence that they will likely interoperate with other independently developed implementations • Minimum agreed performance threshold gives interoperability value and therefore increases the value of the brand • Dispersion of implementations costs implementers • Exception handling code quickly exceeds implementation code • Differences risk functionality apps depend on • Dispersion of implementations costs Data Custodians • Need to deal with Third Parties with different needs and expectations • Unwanted support calls from consumers about difficulties with their apps due to inconsistent implementation

  11. Conformance Testing vs. Certification • Conformance testing evaluates all implemented features of the standard including implementation agreements • Certification testing evaluates a subset of conformance tests based on one or more specific profiles using protocol implementation conformance statements (PICs)

  12. Role of UCAIug • Establish Green Button brand • Register trademark • Guard brand on behalf of certificate holders • Pool marketing resources to promote brand • Provide for certification of implementations • Establish an ITCA (based on NIST’s SGIP IPRM V.2) • Implement testing and certification processes • Develop the test plan specifications and software tools to enable certification by authorized test labs

  13. How Testing And Certification Is to be Accomplished • OpenADE Task Force develops requirements and implementation agreements for ESPI and produces a test plan that encompasses • Green Button Download My Data • Green Button Connect My Data • OpenADE Task moves test plan matrix (spreadsheet) forward to provide basis for test plan and tools • EPRI Green Button Download and Automated Data Exchange Conformance Suite • Builds out test plan to deliverable status • Builds implementing test tools

  14. Green Button Test Plan Requirements Hierarchy GBTP – Green Button Test Plan Document GBTC – Green Button Test Cases Spreadsheet

  15. Green Button Download and Automated Data Exchange Conformance Suite EPRI Project Team Contributions Link to EPRI project description UCAIug OpenADE Task Force Requirements and Test Plan SW Development Test Implementation OpenESPI / Green Button Apache Licensed Open Source Development Project Open Source Apache 2.0 License UCAIugCertifications Tools Commercial ProductsCertification Test Tools Commercial ProductsESPI Implementations Commercial $$ Products and Services

  16. What do we have • Green Button SDK • Data set generator • Simulated data sets • Derived Schema(s) to define conformance profiles • OpenESPI realization of Data Custodian and Third Party • Test Plan So Far • Test Plan Document (GreenButtonTestPlan.docx) • Test Plan PICs Starter (GreenButtonProformaPICS.docx) • Over 160 draft individual test case descriptions (GreenButtonTestCases.xlsm)

  17. Testing of Green Button • Green Button Download My Data • Library of unit conformance tests for • Data contents • Internal reference consistency • Self description of data file (e.g. units of measure included with measurements) • Implementation Profiles • Based on CA IOU implementation agreements plus lessons learned • PICs for implementation assertion • Green Button Connect My Data • Green Button Download My Data for baseline • Protocol tests for 12 Use Cases

  18. Composition of Test Plan • Test Plan Document • Identifies standard and background • Identifies profile definitions and implementation agreements • Includes PICs statement proforma • Describes the test library • Has machine readable annex with test descriptions • Test Plan Software (implements test descriptions) • Schema • Schematron • Test Vectors (know good files)

  19. GreenButtonTestCases.xlsm • Each Row of spreadsheet (simplified) • Test ID – unique id of test • Test Requirement – identifies requirement for test • Test Method – demonstration, test, analysis, inspection • Test Procedure – how to conduct the test (script, steps, etc…) • Test Results – how to interpret results • Spreadsheet has macro to: • Generate Test Cases part of Test Plan document • Generate Test Requirements part of Test Plan Document (macro tbd)

  20. Testing Green Button Download My Data • Green Button Data is XML • <UsagePointxmlns=http://naesb.org/espi/> • XSD Tests the structure and composition of a file (XML) against a data type description • <ComplexType name=“UsagePoint”/> • Schematron is based on XSLT and XPATH to allow for assertion based assessments of the contents of an XML file • <sch:assert test="sum(Interval/value) = 100“/>

  21. Other Tests • Inspection • View of web sites • File names • Inidicia • icons • notices • Demonstration • User Interface requirements as needed • Analysis (if needed)

  22. Testing Green Button Connect My Data • Scenario testing • Protocol evaluation • Success scenarios • “connect” establishment • “connect” removal • “connect” data exchange • Push • Pull • Batch • Failure scenarios • Dropped tokens • Lost messages state

  23. Green Button Connect My Data 23

  24. Test Environment: Example with ThirdParty under test OpenESPI with Test Aspect Subproject of OpenESPI Test Stimulus Engine Proxy: DataCustodian UUT: ThirdParty Instrumentation Test Harness

  25. Technical Activities Smart Grid Interoperability Panel (SGIP) SGIP PAP20 Green Button ESPI evolution: http://collaborate.nist.gov/twiki-sggrid/bin/view/SmartGrid/GreenButtonESPIEvolution SGIP Green Button Initiative TWiki: https://collaborate.nist.gov/twiki-sggrid/bin/view/SmartGrid/GreenButtonInitiative SGIP Green Button FAQ: https://collaborate.nist.gov/twiki-sggrid/bin/view/SmartGrid/GreenButtonFAQ Standards: NAESB PAP10 Task Force Project Page : http://www.naesb.org/smart_grid_pap10.asp NAESB ESPI Task Force Project Page : http://www.naesb.org/espi_task_force.asp NAESB ESPI and PAP10 email lists: to subscribe contact Denise Rager at NAESB - drager@naesb.org NAESB Green Button Portal - http://www.naesb.org/ESPI_Standards.asp Users Group: UCAIugOpenADE Task Force Home Page: http://osgug.ucaiug.org/sgsystems/OpenADE/default.aspx UCAIugOpenADE Mail List: http://www.smartgridlistserv.org/cgi/wa.exe?A0=OPENSG-OPENADE UCAIugOpenADE Meetings: tuesdays at 3:00 EST - https://www2.gotomeeting.com/join/844935738, 415-363-0070 #844-935-738 UCAIugOpenADE Green Button Issues List: http://osgug.ucaiug.org/HelpDesk/Lists/servicerequests/GreenButton.aspx Open Source Implementations and tools: EnergyOSOpenESPI Mail List: http://groups.google.com/group/energyos_espi/subscribe?hl=en EnergyOSOpenESPI Web Site: http://www.openespi.org/ EnergyOSOpenESPI Meetings: mondays at 12:00 EST - https://www2.gotomeeting.com/join/129392235, +1 (516) 453-0010 #129-392-235 EnergyOSOpenESPIGitHub: https://github.com/energyos/OpenESPI EnergyOSOpenESPIGitHub Issues List: https://github.com/energyos/OpenESPI/issues General OpenESPI information: https://github.com/energyos/OpenESPI/wiki/ Energy OS home page: http://energyos.org/ Other: NREL OpenEnergyInfo Green Button Apps Repository: http://en.openei.org/wiki/Main_Page Green Button Data Sample/Developmental Web Site: http://www.greenbuttondata.org

More Related