960 likes | 1.16k Views
Let’s Get Real : Disaster Recovery and Business Continuity in Public Safety. Is Yours Just a Paper Plan or a Real Way to Prepare and Respond to Incidents and Disasters?. Presentation Overview. Key DR/BC Concepts and Issues Report card and dashboard Scenarios
E N D
Let’s Get Real: Disaster Recovery and Business Continuity in Public Safety Is Yours Just a Paper Plan or a Real Way to Prepare and Respond to Incidents and Disasters?
Presentation Overview • Key DR/BC Concepts and Issues • Report card and dashboard • Scenarios • Requirements: What has to operational by when for work to be done by how many at what locations serving what customers who are where? • Facilities • People • Systems • Integration • Coordination • Daily readiness and simulated escalations • Testing and independent verification and validation • Implementation and triage • Recovery, discovery, and improvements • Player Scorecard: Who Is In the Game and Why? • DR/ BC Framework • Action Steps to a Real Plan • First steps • Critical functions • Funding and leveraging scarce resources • Think out of the box • Integration with the big picture DR/BC plan and activities of your jurisdiction • Conclusions
The Report Card and Dashboard • All aspects of the plan, test, and implementation should be scored simply (Red, Yellow, and Green) • Key indicators of planning and readiness need a dashboard to enable assessment and action • Score or status • Trend • Key issue
Public Safety Scenarios • Public safety entities have a more difficult challenge • Your IT DR/BC plan is intertwined with risk scenarios • You may be affected by the risks of a given scenario and your IT plan must address those risks appropriately to maintain operations • You also have a role in response to the scenario so the events will affect your operational requirements
Scenarios Overview • Threat driven geographic circles of impact • Kinds of threats and events • Responsibility • What will you do, what is shared, what do others have to do for themselves • Tolerance for risk and uncertainty • Lesson learned: if you have a well known and documented local risk: • Have a real plan or get ready for a career change…
Scenarios • Identify Possible and Likely Natural Disasters and Environmental Conditions By Kind and Duration of Effects • Tornado • Hurricane • Tsunami • Flood • Snowstorm • Drought • Earthquake
Scenarios • Identify Possible and Likely Natural Disasters and Environmental Conditions By Kind and Duration of Effects • Electrical storms • Fire • Subsidence and landslides • Freezing Conditions
Scenarios • Identify Possible and Likely Natural Disasters and Environmental Conditions By Kind and Duration of Effects • Contamination, Toxic releases and environmental hazards • Epidemic • Pandemic • Animal or crop disease outbreak
Scenarios • Organized and/or Deliberate Disruption • Act of terrorism • WMD • Acute and short lived (bomb) • Acute and long lived (dirty bomb) • Chronic • Long term (contaminants and biohazards) • Permanent (radioactivity, etc.) • WLD (suicide bombers, car bombs, utility sabotage) • Bioterrorism or genetically modified or inorganic organisms • Direct contact • Infectious • Contact • Airborne
Scenarios • Organized and/or Deliberate Disruption • Act of Sabotage • Product or food tampering • Act of war • Theft • Arson • Labor Disputes / Industrial Action
Scenarios • Loss of Utilities and Services • Electrical power failure • Loss of gas supply • Loss of water supply • Petroleum and oil shortage • Raw materials • Refined materials • Communications services breakdown • Loss of drainage / waste removal and trash pickup
Scenarios • Equipment or System Failure • Internal power failure • HVAC failure • Equipment failure (excluding IT hardware)
Scenarios • Serious Information Security Incidents • Cyber crime • Malware • Zombie attacks • Denial of service • Loss or alteration of records or data • Disclosure of sensitive information
Scenarios • IT system failure (local or hosted) • Hardware • Software • Commercial application • Locally developed application • Data • Communications
Scenarios • Other Emergency Situations • Workplace violence • Public transportation disruption • Neighborhood hazard • Health and safety issues
Scenarios • Multiple and compound hazards and events • Purposeful • Coincidental • Causally connected • Interrelated
IT Requirements • What systems need to function • How fast • Maximum and optimum time frame for each system or function to be restored • How well • Sometimes minimal functionality is sufficient
IT Requirements • Where will it be used and by whom and will the communications infrastructure support it? • Employees • Users or beneficiaries • By what priority will systems be restored • The priority will be modified by what contingencies • E.g. a long term total evacuation changes the operational needs for criminal justice systems and personnel
Facilities • Hot, warm, cold • Mirrored, recoverable, reload-able • Properly located • EOC • Non-EOC • Operational • IT facilities • For user interaction with IT systems
Facilities • New kinds of mutual aid and sister city/county/state arrangements • Work with friends, colleagues, associations, and vendors • To match you with a comparable entities that are located outside the various geographic threat circles • Who can mirror your IT operations (hardware, software, operating systems, and culture)
People • The right numbers, skills, location, redundancy, etc. • Skills and abilities inventory • Employees • Contractors • Vendors • Mutual aid and “the cavalry”
People • Force in depth—who is the backup to the backup to the backup? • Consider the actual health and physical abilities and disabilities of a person when assigning tasks for a disaster scenario • The disaster is not the time to find out the electrician in the hazmat suit has a heart condition • What family and personal duties may interfere with performing official duties (e.g. save your own kids or save a stranger)?
Systems • Daily operational • Interdependent systems • Emergency only • Identity security and access management for physical and logical security • Follow FIPS 201 for federal/state/local interoperability
Integration • With whom should you work closely? • Identify integration issues between: • Internal systems and public safety entities • Other governmental systems • Related actors • Non-governmental systems and processes • Example: 911 and 311or its equivalent • Normally separate but related • Emergencies blur the line • Co-location, cross training, and system integration
Coordination • Within organization • Within unit of government • Across units of government • Across levels of government • Across public and private boundaries
Daily Readiness and Simulated Escalations • A disaster a day (“What, that’s not normal?”) • Realistic scenarios • Captured lessons • Learning and actually responding to lessons learned within risk framework • A quality and security framework for daily operations has substantial overlap with DR/BC
Security Capabilities Models Security Leadership Security Sponsorship Security Strategy Security Program Security Program Structure Security Program Resources and Skillsets Security Policies Security Policies, Standard and Guidelines Security Management Security Administration Security Monitoring User Management User Management User Awareness Information Asset Security Application Security Database / Information Security Host Security Internal Network Security Network Perimeter Security Technology Protection and Continuity Physical and Environment Controls Contingency Planning Controls Causes Strategy Management Knowledge Technologies Support Effects Like similar capability models from the Carnegie Mellon SEI, SCMM models brings benefits: • Helps close security holes • Serves as a foundation for growth • Guides security leadership • Is evolutionary, not chaotic • Supports point solutions KPMG SCMM Model
Capability Maturity Continuously improving process Optimizing (5) Managed (4) Predictable process Standard, consistent process Defined (3) Repeatable (2) Disciplined process Initial (1) Informal process Like the SCI CMM models, the KPMG Security Capability Model has five levels of maturity:
Testing and Independent Verification and Validation • Does the planned response or action step actually work? • Who verifies that it does? • What do you do if it fails the test?
Implementation and Triage • Someone better be in charge • Dispute resolution processes • Who will be your Sensibility and Sanity Checker (off site, not affected by the disaster, and actually getting enough sleep to make sound decisions)? • Baton Rouge example with Mayor Holden
Recovery, Discovery, and Improvements • What will the new normal be and when will it happen • Learn from history, both recent and long past • Document while the event occurs if at all possible (make it someone’s job) or soon after before memories fade
Player Scorecard Who Is In the Game and Why
Overlapping and Inter-Related Responsibilities Disaster Preparedness and Recovery and Business Continuity Physical Security Quality Assurance Methodologies Public Safety Cyber Security
The Usual Suspects in Public Safety • Police • Fire • Other sworn officers (transit, game, building or branch based, etc.) • National Guard • Public Health • Public Works • Transportation • Environmental Protection
The Usual Suspects in Emergency Management • Federal, state and local emergency management entities • National Guard • NOAA, NWS, NSSL, other National Laboratories, • Corps of Engineers
IT Entities • CIO, CTO, and Enterprise IT Shops • Distributed IT Departments and leadership • Government IT contractors • DR/BC specific entities • Applications developers and software • Hardware • Service providers (ASP, MSP, call centers, etc. • Communications providers
Policy Makers • Executive, legislative, and judicial • Those who hold the seat and those who actually make the decisions… • Go below the top level to ensure clarity, alignment, and redundancy • EOC designees • Emergency authorizers
Non-Governmental Organizations • Media • Broadcast and satellite • Emergency Broadcast System Members • Print • New media • The Web • Government site mangers • Commercial site managers • Citizens and bloggers • Self-organizing communities (e.g. Craig’s List)
Non-Governmental Organizations • Charities • Businesses and business associations • Community organizations • Vital private services (hospitals, nursing homes, etc. )
Business Operations and Technology • Create a matrix, not a linear or organizational view • Strategy • Organization • Processes • Applications and data • Technology • Facilities
First Steps • Leadership: clarity, alignment, and commitment • Authority or consensus? • Stakeholders roles and responsibilities • Be clear about risk tolerance • Applications and IT assets inventory • If needed, dust off and update your Y2K work • Good data on plan status, readiness, test results, response, and compliance
First Steps • Make a friend in accounting—actuarially accurate threat scenarios are more likely to be funded as risk and cost can be properly balanced • Review existing plan or make a plan • Borrow or buy a template • Review peer plans and conduct site visits • Communicate until it hurts
Nail Down Your Critical Functions • Law and order essentials (people, mobility, tools, survival basics, etc.) • Communications • Personnel management (policies, scheduling, notification trees and systems, counseling, etc.) • Data and the connections to data and people • Transactional systems