IT and Smart Grid Interventions

1. IT and Smart Grid Interventions Naveed Arshad LUMS

2. What is Smart Grid? • ‘System of systems’ concept • A Smart Grid is an electricity network that can intelligently integrate the actions of all users connected to it - generators, consumers and those that do both – in order to efficiently deliver sustainable, economic and secure electricity supplies. • The smart grid is a broad collection of technologies that delivers an electricity network that is flexible, accessible, reliable and economic. Smart grid facilitates the desired actions of its users. It may include distributed generation, the deployment of demand management and energy storage systems or the optimal expansion and management of grid assets

3. What Does the Smart Grid Concept Look Like?

4. Smart Grid Characteristics • Optimized for best resource and equipment utilization • Distributed by its structure (assets and information) • Interactive (customers, retailers, markets) • Adaptive and scalable (for changing situations) • Proactive rather than reactive (to prevent emergencies) • Self-healing (can predict/distinguish/bypass abnormal situations) • Reliable and secure (from threats and external disturbance) • Efficient and reliable • Open for all types and sizes of generation • Environmental friendly (using renewable energy resources) • Integrated (monitoring, control, protection, maintenance, EMS, DMS, AMI)

5. Traditional Grid vs. Smart Grid

6. Transforming Guidelines for Smart Grid

7. Smart Grid For Different Industry Players

8. Why focus on distribution systems? • Increasing penetrations of distributed energy resource (DER) in power distribution systems • On a 5-year basis (2015-2019), DER in US is growing almost 3 times faster than central generation (168 GW vs. 57 GW). • In 2016, distributed solar PV installations alone represented 12% of new capacity additions. • California DER, 7GW in 2017, 12 GW by 2020 (peak load 50 GW) Annual Installed DER Power Capacity Additions by DER Technology, United States: 2015-2024 U.S. DER Deployments Source: The U.S. EIA and FERC DER Staff Report Source: Navigant Report, Take Control of Your Future

9. The need for advanced modeling, monitoring, and control in distribution systems • The cold hard facts about modern power distribution systems • Modeling • Incomplete topology information in the secondary systems • Phase connection • Transformer-to-customer mapping • Even the three-phase load flow results are unreliable! • Monitoring • Most utilities do not have online three-phase state estimation for their entire distribution network • Control • Reactive Control • System restoration, equipment maintenance • Limited Proactive Control • Volt-VAR control, CVR, network reconfiguration

10. Big Data in Distribution Systems: Volume • In 2017, the U.S. electric utilities had about 78.9 million AMI installations covering over 50% of 150 million electricity customers. • The smart meter installation worldwide will surpass 1.1 billion by 2022. • In 2012, the AMI data collected in the U.S. alone amounted to well above 100 terabytes. • By 2022, the electric utility industry will be swamped by more than 2 petabytes of meter data alone. U.S. Smart Meter Installations Projected to Reach 90 Million by 2020 Source: U.S. Energy Information Administration Source: Institute for Electric Innovation

11. Big Data in Distribution Systems: Variety • Advanced Metering Infrastructure • Electricity usage (15-minute, hourly) • Voltage magnitude • Weather Station • Geographical Information System • Census Data (block group level) • Household variables: ownership, appliance, # of rooms • Person variables: age, sex, race, income, education • SCADA Information • Micro-PMU • Time synchronized measurements with phase angles • Equipment Monitors

12. Big Data in Distribution Systems: Velocity • Sampling Frequency • AMI’s data recording frequency increases from once a month to one reading every 15 minutes to one hour. • Micro-PMU hundreds (512) of samples per cycle at 50/60 Hz • Bottleneck in Communication Systems • Limited bandwidth for zigbee network • Most of the utilities in the US receives smart meter data with ~24 hour delay • Edge Computing Trend • Itron and Landis+Gyr extend edge computing capability of smart meters • Increasing data transmission range and computing capabilities of smart meters • Centralized → distributed / decentralized

13. Big Data in Distribution Systems: Value • The big data collected in the power distribution system had utterly swamped the traditional software tools used for processing them. • Lack of innovative use cases and applications to unleash the full value of the big data sets in power distribution systems1. • Insufficient research on machine learning and big data analytics for power distribution systems. • Electric utilities around the world will spend over $3.8 billion on data analytics solutions in 2020. Source: GTM Research • 1. Nanpeng Yu, Sunil Shah, Raymond Johnson, Robert Sherick, Mingguo Hong and Kenneth Loparo, “Big Data Analytics in Power Distribution Systems” IEEE PES ISGT, Washington DC, Feb. 2015.

14. Applications of Big Data Analytics and Machine Learning in Power Distribution Systems Spatio-temporal Forecasting Electric Load / DERs – Short-Term / Long-Term Anomaly Detection Electricity Theft, Unauthorized Solar Interconnection System Monitoring State Estimation & Visualization Distribution System Controls Deep Reinforcement Learning Network Topology and Parameter Identification Transformer-to-customer, Phase connectivity, Impedance estimation Equipment Monitoring Predictive Maintenance Online Diagnosis Customer Behavior Analysis Customer segmentation, nonintrusive load monitoring, demand response

15. High Level IT Landscape Knowledge Mgmt Portal Collaborations Mgmt Information System Analytics Enterprise Resource Planning GIS Billing CIS/CRM Asset Management Operations Technologies

16. The Technologies ET OT IT • Information Technologies • ERP • Portals / Gateways • Asset Management • Business Intelligence (BI) & Data Warehousing / Mining • all horizontal information management applications • Operation Technologies • EMS, SCADA, DMS, DSM • GIS • Asset Management • FACTS • WAMS • all analysis, automation & process control applications • Energy Technologies • Distributed generation • Demand response • Energy storage • High temperature superconductivity • Renewables: • Integrated gasification combined cycle • Hydrogen economy • Photovoltaic generation • Fuel cells • Liquefied natural gas • Wind generation

17. Smart Grid Deployment Phases PlanningAnalysis FeasibilityAssessment TechnologySelection SystemDeployment • Strategic Business Planning/Visioning • Business Needs Definition • Regulatory Compliance • Technology Trends/EmergingTechnologies • Technology Assessment • Business Case / Economic Justification • Preliminary Cost Scenarios • Project Planning Approval • Team Formation • Functions & Features Matching • Performance Assessment • Integration Requirements • Vendor Selection • Cost/Benefit Confirmation • Project Approval / Capital Commitment • Implementation Planning / Phasing • Employee Training • System Testing • Operating Procedure Updates • Value Recognition • ROI Measurement 17

18. Requirements to consider • Communication technology covering backbone and access networks • Latency and security aspects • Bandwidth • Coverage • Scalability • Ownership • Terminal density • Interface flexibility • OPEX and CAPEX costs • Market characteristics • Reliability • Security (integrity, availability, confidentiality, authentication, non-repudiation) • Data and communication protocols Classification • Network requirements • Security requirements • Latency requirements • Data requirements • Technology related requirements

19. Design and analysis process Discovery of ICT requirements Creation of ICT reference model Testing with pilot specifications

20. IT Interventions is Distributed Systems in Indian Distribution Companies • SCADA • Substation Automation System (SAS) • Distribution Management System (DMS) • Distribution Automation (DA) • Outage Management System (OMS)

21. Advanced Distribution Management System

22. SMART Grid Building Blocks Infrastructure Smart Home Infrastructure (Smart Thermostat, Load Control Devices, Gateways.) AMI Infrastructure (Smart Meter, Data Concentrators, AMI head end) SMART Grid Infrastructure (Sensors, Grid Control Devices, Grid head end) Transformation Services Other Services Integration & Data Services Comms Home Area Network (HAN) Local Area Network (LAN) Wide Area Network (WAN) Customer Service Meter Data Management Demand Response Management Customer Relationship Management Data Service Cut-over Management Application Development & Maintenance Customer Portal Management Credit and Revenue Protection Metering and Billing Service Management Security, Standards & Interoperability Distribution Management SCADA Distribution Management Outage Management Business Intelligence Business Change Management Consulting (Roadmap, Strategy) Operations Network Planning Network Management Asset Management Work Management Project Management IS Infrastructure Management Systems Integration Power Resource Management Trading and Contracts Settlement Risk Management PHEV Distributed Generation Storage devices Training

23. SCADA Definition • A complex computer based system that uses modern applications to analyse the electric power grid system to acquire data, monitor and control facilities and processes. • SCADA applications can support dispatchers, operators, engineers, managers, etc. with tools to predict, control, visualize, optimise, and automate the EPU.

24. Summary of SCADA History • Originally EPUs used electro-mechanical automation • Dial-up modems used for remote access • In 1970s computer-based SCADA commenced • Suppliers (e.g. IBM, Siemens, GE) supplied complete proprietary systems • More advanced with client-server computers • Advanced functions became common (e.g. EMS. DMS, load forecasting, dispatch, protection engineering, regulatory reporting, etc) • Communication link evolved from noisy narrow bandwidth telephone lines to SONET, Microwave, radio, power line carrier, cellular networks

25. Traditional SCADA Components • SCADA Master Terminal Unit (MTU): The server that acts as SCADA system • RTU (remote terminal unit) : remote telemetry data acquisition units located at remote stations • IED (intelligent electronic devices) smart sensors/actuators with intelligence to acquire data, process it, and communicate • HMI (human-machine interface) : software to provide for visualisation and interaction with SCADA

26. Overall SCADA System Architecture • Can be broken down into 3 categories • NIST representation of SCADA system • Control Center • Programmable Logic Controllers(PLCs), Remote Terminal Units (RTUs), IEDs • Communications Network • SCADA host software

27. Control Center • Provides for real-time grid management • SCADA Server • Also known as the MTU (master terminal unit) • HMI for visualisation and human interaction • Programming/Engineering workstations • Data historian, a database storage for operational activities • Control server, hosts software to communicate with lower level control devices • Communication routers • Could be connected to other regional control centers (desired for large networks)

28. Communication Link • Phone line/leased line, power line carrier • Radio • Cellular network • Satellite • Fibre optic

29. Communication topologies • Star • Ring • Mesh • Tree • Bus

30. Implementation Examples • Many possible topologies • Direct connection • Connection with slave • Other. See IEEE C37.1

31. Protocols and standards • Allow communications between devices • MODBUS: master-slave application-layer protocol • Attackers with IP access can run Modbus client simulator to effect many types of attacks. • DNP3 : Distributed Network Protocol is a set of open communication protocols • IEEE recommended for RTU to IED messages • Has no in-built security: Messages can be intercepted, modified and fabricated. • IEC 60870 suite: • Substation control centre communication (IEC 60870-5-101/104) • Communication with protection equipment (IEC 60870-5-103) • IEC 62351 intends to implement security (end-to-end encryption; vendors reluctant to implement due to complexity) • Other proprietary protocols

32. Field Components • Acquire telemetry, relay data from system • Covert it to digital signals if necessary • Send data to MTU or engineering stations • Receive control, settings, resets from MTU SCADA MTU Field component Control, Settings Device Ports Telemetry Meters Relays, etc

33. Field Components: RTU • Reads status and alarms through relay and control circuit auxiliary contacts. Meter reading. • Manual/remote control e.g. activate alarm. RTU control outputs connected to control relays • No data storage • Some PLCs equipped to be RTUs • May aggregate IED data • Either open standard or proprietary based • Modbus, DNP3, IEC 60870-5-101/104 • Serial communication • RS232, RS485

34. Field Components : IED • Similar to RTU, is open or proprietary based • Acquires data from electrical devices, e.g. relay or circuit breaker status, switch position. • Reads meter data such as V, A, MW, MVAR. Some modern meters have IED capabilities, they can communicate their readings with RTU or MTU. • Control functions include: • CB control, voltage regulators, recloser control. • Newer substations only use modern IEDs • IEDs can support horizontal communication

35. SCADA and internet connection

36. SCADA Security • Traditionally isolated networks • No security measures deemed necessary; security by obscurity • Only threats were insiders and physical sabotage • Modem war-dialing was also possible threat • With interconnected EPU, SCADA is connected over wide area networks and internet • That has exposed SCADA to attacks

37. SCADA Security Holes • Increased automation widens SCADA network’s attack surface

38. Typical SCADA threats (actors) • Espionage • Spies (industrial and state actors) • Terrorists • Script kiddies • Insiders, e.g. disgruntled employees • Criminal elements (blackmail) • Business competitors • Hacktivists (ideological activists)

39. SCADA Vulnerabilities • Vulnerabilities are weaknesses in the cyber system that threats (actors) exploit to carry out attacks • Examples of forms vulnerabilities: • Technical • Hardware • Software and protocol • Network • Policy

40. Vulnerability examples • CVE-2015-1179: Allows remote attackers to inject arbitrary web script; found in Mango Automation systems • CVE-2015-0981: Allows remote attackers to bypass authentication and read/write to arbitrary database fields via unspecified vectors. • CVE-2015-0096 (MS15-018) : Stuxnet, a worm targeting ICSs such as SCADA. • Other examples from 2014: CVE-2014-8652 , CVE-2014-5429 • GE Energy's XA/21: 2003 flaw responsible for alarm system failure at FirstEnergy's Akron, Ohio control center

41. Attack Examples • Stuxnet: Intercepts and makes changes to data read from and written to a PLC • Night Dragon : Suspected SCADA data exfiltration from Exxon, Shell and BP • Others: Havex (Trojan targeting ICSs and SCADA), Blacken (Targets users of SCADA software Simplicity) • Many others targeting the PCs used in SCADA.

42. Securing SCADA • Define SCADA security networking policy • Access control • Identify all SCADA assets and their connectivity • Schedule regular vulnerability assessments • User training and awareness (e.g. what to do when you pick up a USB stick in parking lot) • Technical • Isolate SCADA from internet as much as possible • Encryption of data • Implement strict firewall rules between SCADA network and all other networks. • Perform anomaly detection

43. Securing SCADA • Put in place effective policies • Limit access to SCADA network; implement tight security access controls • Use hardened hardware • Patch regularly, don’t use unpatched software or vulnerable systems • Implement vendor security features (No defaults) • Audit (include red teaming) SCADA IT systems for security holes

44. The Smart Grid Maturity Model is A management tool that provides a common language and framework for defining key elements of smart grid transformation and helping utilities develop a programmatic approach and track their progress

45. SGMM at a glance 6 Maturity Levels: Defined sets of characteristics and outcomes 5 4 3 175 Characteristics: Features you would expect to see at each stage of the smart grid journey 2 1 0 SMRStrategy, Management, & Regulatory OSOrganization & Structure GOGrid Operations WAMWork & Asset Management TECHTechnology CUSTCustomer VCIValue Chain Integration SESocietal & Environmental 8 Domains: Logical groupings of smart grid related characteristics

46. Smart Grid Maturity Model: levels PIONEERING Breaking new ground; industry-leading innovation OPTIMIZING Optimizing smart grid to benefit entire organization; may reach beyond organization; increased automation INTEGRATING Integrating smart grid deployments across the organization, realizing measurably improved performance ENABLING Investing based on clear strategy, implementing first projects to enable smart grid (may be compartmentalized) INITIATING Taking the first steps, exploring options, conducting experiments, developing smart grid vision DEFAULT Default level (status quo)

47. Smart Grid Maturity Model: domains SMR Strategy, Mgmt & Regulatory TECH Technology Vision, planning, governance, stakeholder collaboration IT architecture, standards, infrastructure, integration, tools OS Organization and Structure CUST Customer Culture, structure, training, communications, knowledge mgmt Pricing, customer participation & experience, advanced services GO Grid Operations VCI Value Chain Integration Reliability, efficiency, security, safety, observability, control Demand & supply management, leveraging market opportunities WAM Work & Asset Management SE Societal & Environmental Asset monitoring, tracking & maintenance, mobile workforce Responsibility, sustainability, critical infrastructure, efficiency