Results Based Drafting for WECC Drafting Teams

1. Results Based Draftingfor WECC Drafting Teams W. Shannon Black Manager, Standards Processes

2. Overview Why change? Changes to the process Scope Early determine of the roadmap Changes to the documents New template Where things go

3. Drafting Process / Sequencing

4. Why change? • “R”equirements need to improve • Current “R”equirements • Are unclear • Administrative not results • Impossible to achieve! Measure! • Voluntary • Devoid of drafting conventions • Failure to identify the work product up front is slowing the Process • Where are we headed?

5. Where are we Headed? • Emphasis on • Defining the Scope (product) • Before we draft • Defining the Requirements • Before we draft • Using proper drafting conventions • Using common sense • Using a sequencing concept • Defining the Review the process • Quality control review at each posting

6. Define the Scope DEFINE SCOPE DEFINE REQUIREMENTS REVIEW AND APPROVE STANDARD • NEED: Why the standard is being developed or modified • GOALS: What must be accomplished to meet the need • OBJECTIVES: Criteria for success • OPERATIONAL CONCEPTS: Day in the life • STAKEHOLDERS: Individuals or groups impacted or interested • DRIVERS: External constraints Get consensus before writing requirements

7. Define the Requirements DEFINE SCOPE DEFINE REQUIREMENTS REVIEW AND APPROVE STANDARD • Necessary: It is needed? • Meet one of the Reliability Principles included in the Request? • Redundant? • Attainable: Can you meet it? • Clear: Can be understood only one way? • Measures: Can we comply? Write results-based requirements

8. Review and Approve DEFINE SCOPE DEFINE REQUIREMENTS REVIEW AND APPROVE STANDARD • Collect comments • Revise in response to comments • Recycle per planned schedule • 45 and 30 day Comment windows • Complete a quality review check list • Submit for approval • Team votes – forwards to the Standard Committee

9. Scope What you need to know and agree upon in order to develop a standard and write requirements Gathering the tools for the tool box

10. Defining Scope • View the maze from the top! • See the problems before wasting time • You think you know… • …but you don’t know you know. • Defining scope • Defines the boundaries • Defines the target

11. Defining ScopeOperational Concepts/ Defining the Problem Identify the OPERATIONAL CONCEPTS Understand the operational fact pattern What specifically happened to create the problem your team will try and resolve? How do we address it today? Make a list of the steps that are taken TODAY to meet the operational concern How should we address it tomorrow? Make a list of “first blush” suggested changes to the existing operations.

12. Defining Scope Identify the STAKEHOLDERS Who was impacted by the fact pattern? Who will be impacted by the remedy? Identify the DRIVERS What’s pushing this whole effort? Get the “agendas” out on the table

13. Defining ScopeWhy identify the stakeholders and drivers? The impact is broader than the Functional Model Vested Interested Lineman / Back Office / After-the-Fact / Schedulers “Mom and Pop” Influential Interests Government Stock holders Participating Interests Usually the “Applicable Entities”

14. Defining ScopeWhy identify the stakeholders and drivers? The end product has to reflect reality Consider potential drivers Technology Deadlines Existing Standards / Existing Processes Personal and Corporate agendas End user expectations

15. Purpose Statement What you need to know and agree upon in order to develop a standard and write requirements Gathering the tools for the tool box

16. Defining ScopeFocusing on the Purpose Statement Identify the NEED Why the document is being developed or modified? “To ensure uniform maintenance procedures…..” Identify the GOALS/OBJECTIVES What do we have to do to meet the needs? “…by defining maintenance periodicity.” Lays out the criteria for success

17. Defining Scope “Purpose” Exercise

18. Defining Scope “Purpose” Exercise • What’s wrong with this picture? • Purpose Statement for EOP-004-1 • “Disturbances or unusual occurrences that jeopardize the operation of the Bulk Electric System, or result in system equipment damage or customer interruptions, need to be studied and understood to minimize the likelihood of similar events in the future.”

19. Defining Scope / “Purpose” Small Group Exercise What’s missing? What’s included that need not be? Would it help to know the fact pattern? What’s the intent (goal/objective) of the document? Can you tell? Can an auditor tell? How do we know when we “did it”? Can we compare the Purpose statement to the final document and know “we’re done”?

20. Defining Scope “Purpose” statement • Suggested rewrite • NEED • “To improve BES reliability…” • GOAL • “…by reporting and analyzing Impact Events…” • Objective • “…for the purposes of minimizing repeat events.” • This is just one solution

21. Drafting the PurposeSmall Group Discussion Review the Team’s Requests Identify the fact pattern and issues Sometimes called Operational Concept Consider what is being done now and what changes need to be made Identify the NEED E.g.. “To create a standardized process…” Identify the GOAL E.g.. “…used by every Balancing Authority…” Identify the OBJECTIVE E.g.. “…to measure Time Error Correction.”

22. Requirements

23. Regulatory Facts of Life • Requirements are boring • This reflects uniformity and pattern • Requirements look the same • “Each BA shall do X.” • Requirements are a “language” • People don’t write like they talk • The spoken word is communicated differently than the written word • People dislike taking time to write at all! • Your job is to take the time

24. Regulatory Facts of LifeThings to Correct • Don’t provide adequate reliability • “Documentation heavy” • Can be interpreted different ways • Subjective terms • Poor grammar • Multiple requirements in a single statement • Don’t clearly state responsibility • Can’t be measured • Can’t be enforced

25. Regulatory Facts of LifeDrafting Conventions • Avoid Ambiguous Terms • Adverbs (words that end in “ly”) • Pronouns (it…this…) • And / Or • Associated / Affected / Accommodate • Coordinate / Facilitate • Maximize/Minimize • Unique / “Mutually agreed upon” • “be able to” “capable of” • Wouldn’t you rather they just do it?

26. Regulatory Facts of LifeDrafting Conventions • Do • Use the structure • Formatted style of language • NERC Functional Model • Make sure the newest staff member understands the document • Don’t • Incorporate by reference • Create new terms • Abbreviate

27. Results Based RequirementsLock Down the Structure • Basic components • Identifies each affected party • “Each Balancing Authority…” • Is mandatory • “…shall…” • Identifies the task • “…operate [in a specific fashion].” • To achieve a particular result • “…to ensure a balanced interchange.” • Measureable and objective

28. Results Based RequirementsCreate an Affirmative Result • To create an Affirmative Requirement • Meet the Purpose of THE ASSIGNED document • Simply because it’s an excellent requirement doesn’t mean it belongs in “this” standard • Increase the reliability of the system • Meet one of the NERC reliability principles • Not just admin for the sake of admin • At an affordable cost (Common Sense Test) • Is actually feasible / possible / affordable • Internally and externally consistent

29. Results Based Requirements Understood the same way by All FERC If a team sees multiple interpretations – rewrite it! If the newest guy in your shop can’t understand it – rewrite! If your supervisor can’t understand it – rewrite it! If your friends don’t understand the sentence structure – rewrite! Electrical Providers NERC

30. It’s New!Enhance your Requirements • It’s acceptable to use • Graphs • Pictures • Charts • Examples of how the Requirement works • Sample calculations

31. When Requirements go wild!Exercise

32. What’s wrong with this Picture? • Responsible entities may schedule as appropriate. • Each TO shall eliminate all lower quality timing loops that occur when two network devices line time off of each other under normal conditions without traceability to a Stratum 1 reference. • Each Balancing Authorities shall perform all calculations based on the March 10, 2010 Table found at….” • Each Scheduling Entity shall schedule in accordance with NAESB e-Tag requirements.

33. Rationale

34. What is the Rationale block?

35. Why have a Rationale block? “In today’s world of high employee turnover and complex, long-lived products, well-documented rational is essential to better product development.” Ivy Hooks. Benefits Key to understanding Reduce interpretation problems Facilitate maintenance and upgrades Preserve corporate knowledge

36. What goes in the box? • Why a requirement is needed • What assumptions were made • What analysis effort drove the requirement • Information to help understand the requirement • Source of any numbers

37. What does NOT go in the box? • A rewrite of the requirement • Hidden requirements • Copy of another requirement’s rationale • Everything you know on the topic

38. The Requirement must Stand Alone • “Third, we understand that the proposed results-based standard format may include expanded background sections, expanded descriptions of a Reliability Standard’s purpose, and/or explanations about the intent of individual Requirements. This information may provide useful context but it should not contradict or seek to supersede or interpret the requirements within a Reliability Standard. The requirements within the Reliability Standard should govern, and the application of the Standard should be clear without reference to the background or purpose sections.” • Docket Nos. RR09-7-000 and AD10-14-000, P. 109

39. Measures

40. Measures have Moved

41. Measures • One per customer! • One Measure per Requirement • Draft upon completing the Requirement • Each Measure must: • Identify the functional entity • Tangible, practical, objective • Identify what evidence or types of evidence could be used to show that an entity is compliant with the requirement.

42. Comments

43. Dealing with Comments • Important to consider all carefully • Not important to agree to all • Don’t waste time playing the assumptions game • Do not hesitate to ask for clarification • If many comments from one person or group – ask for priorities. • Categorize if necessary

44. Questions?

45. Getting Started • What’s the issue/fact pattern? • Review the Request • What needs to be fixed? • How do we do it now? • List the “Day in the Life of” today • How do we propose to fix it? • List the proposed “Day in the Life of” for tomorrow • This is the skeleton for our document!