1 / 80

MARC COOP Workshop

MARC COOP Workshop. September 12, 2008. Dan Esser CBCP Columbia, MO daniel@contingencynow.com. Expectations for Session. Help those who are started expand their plans Help those who are not started with how to start Use the Template to launch the COOP Planning process

tanek-vega
Download Presentation

MARC COOP Workshop

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. MARC COOP Workshop September 12, 2008 Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  2. Dan Esser CBCPColumbia, MOdaniel@contingencynow.com Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  3. Expectations for Session • Help those who are started expand their plans • Help those who are not started with how to start • Use the Template to launch the COOP Planning process • Session will not tell everything you need to know, but will put you on the right track Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  4. Format • For the most part there will be an overview of points to be covered. After that we will cover each in detail. You may hear some things more than once. • If I’m going too fast – Tell me to slow down. • If I’m going too slow – Tell me to speed up. • If I digress – Tell me to get back on point. • If you have a question or comment that is pertinent to what we are discussing – Stop me and ask or comment. Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  5. Points for Discussion • Definition of COOP • Why COOP is Important • How to Start / Expand Current Planning • Things to Add to the Template • Why “Complete” isn’t • Things you want to talk about... Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  6. Jackson Article on Making Plans Actionable • Continuity Plans are often not used during Testing or Recovery • Why Not? • The Information is There • The Actions to Recover are Not Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  7. RTO / RPODefinitions We Need Throughout • RTO – Recovery Time Objective • The Time within which a business process must be restored after a disaster to avoid unacceptable consequences associated with a break in service • RPO – Recovery Point Objective • The point in time to which data must be recovered – "acceptable loss" in a distressed situation Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  8. Essential Functions • C-Now: Functions that, if interrupted beyond the RTO, would cause serious or irreparable harm to people or institutions in the jurisdiction. • Template: Functions that enable city/county agencies to provide vital services, exercise civil authority, maintain the safety and well being of the citizens, sustain the industrial/economic base in an emergency. Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  9. Essential Functions (cont.) • Just keeping the “essential” stuff going is not enough • Each Jurisdiction has to be planning toward full recovery • Not Essential does not mean Not Important Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  10. COOP Review • What is COOP? • Why COOP? • What are the Components? • What are Sources of Risk? Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  11. What is COOP? • COOP is the ability to continue essential functions or restore within a predetermined time frame (RTO) and with data loss held within the maximum acceptable loss (RPO). • Creating a COOP Plan is about making all the decisions that can be made prior to an incident before anything really happens. Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  12. Why COOP? • At a Departmental or Building Level • Any Department that cannot perform its essential functions • Does not help the rest of the jurisdiction • May impede the rest of the jurisdiction • Applies to Small and Large Incidents • YES – IT CAN HAPPEN TO YOU! Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  13. COOP Components • Department/Building Safety Response • Department/Building Incident Response • Continuation/Recovery of Essential Functions • Recovery of All Functions • Return to Normal Operations – Permanent Space Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  14. Hazards • Natural Hazards • Earth, Wind, Fire, Water • Human Intervention • Terrorism • Human Error • Health Hazards • Infrastructure Disruptions Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  15. Dept./Building Safety Response • Knowledge and Practice of • Evacuation to Safe Area • Accounting for Occupants and Visitors • Shelter in Place • Establishment of an Organizational Structure and Occupant Emergency Plan Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  16. Dept./Building Incident Response • Structure Related to Size of Jurisdiction • At its Most Basic • Take Care of People • Gather Information • Assess the Situation • Determine and Act on Next Steps • Information to Management • Information to Public/Press by Jurisdiction Leaders Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  17. Building on the Current Template • Components outlined in FPC 65 are present • Legal Underpinnings are present • Annexes are Repository for Recovery Information • Open Ended - Jurisdictions can add things they need or re-order what is there Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  18. Getting Started • Six Major Pieces for creating an Actionable COOP • Preparation / Management Buy In • Impact Analysis & Risk Assessment • Mitigation Strategies • Operations Restoration • Information Technology Restoration • Regular Exercise / Learning / Updating Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  19. Preparation / Management Buy-in • Identify COOP Plan Coordinator • Identify other resources available to help with planning and their roles • Management Support and Funding Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  20. Impact Analysis & Risk Assessment • Identify functions • Determine scope • Set initial RTO’s and RPO’s • Identify essential functions • Identify key enabling Technologies and Processes • Identify Risks that could disable service • Identify Mitigation strategies • Identify Recovery Strategies Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  21. Mitigation? What’s That? • How to keep an incident from being a disaster • Workarounds unique to each organization • Examples for: • Payroll Direct Deposit • Payroll Checks • Power Flickers during storms • Power Outage • Everything is stored in the office • Homes and Cars Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  22. COOP for Operations • Manage the incident and start the recovery process • Plan for total building loss • Consider how to provide alternate work space • Set final function RTO’s and RPO’s and identify critical services • Identify technologies required for critical services • Look out for SPOFs Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  23. COOP for Operations (cont.) • Create workarounds for absence of technology • Identify services that can go forward with limited or no technology • Pre-Arrange for ongoing communications • A capability to put messages on incoming phone lines located outside the PBX equipment of the jurisdiction • Pre arranged capacity to move lines to an alternate place • Identify and pre-position key start-up items or supplies (mitigation) Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  24. COOP for Operations (cont.) • Set up contact information and processes for employees and officials • Set up contact information and processes for key vendors/suppliers • Set up processes for quarterly or semi-annual plan updates, annual testing Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  25. COOP for Information Technology • Coordinate IT recovery with the overall incident manager or team • Identify critical servers and associated infrastructure • Base this on user functional needs and which parts of the technology support those • Arrange for equipment and a place to restore critical servers • May include redundant systems located in a different place for those things that must be up 24/7/365 Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  26. COOP for Information Technology (cont.) • Pre-position supplies, documentation of restoration steps and copies of software • Set up contact information and processes for IT employees and officials • Set up processes for quarterly or semi-annual updates, annual testing Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  27. Exercising and Testing • It’s a TEST! EEK! -- Failure Anxiety! • Sometimes this causes people to plan the test rather than test the plan • Establish a “Safe Environment” for the exercise Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  28. OK, but where do “I” start? Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  29. Step 1 - Management Buy In • Things you may hear • There’s no budget for that. • Why spend money on something that may not happen? • We have insurance. Why do we need this? • Information Technology has all of that handled. Everything is backed up. • Get out of my office I don’t want to talk about this. • Your Favorites…? Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  30. Management Buy In (cont.) • Leadership needs to know the impact of not having essential functionality • If there is resistance to the concept, ask for a Pilot study to see if more is needed • Low Cost and can be restricted to a few buildings or departments • Results will determine more Analysis is needed • Pick High Impact Departments (IT, Payroll, Accounts Payable, Tax Collection, Courts) Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  31. Management Buy In (cont.) • Note the need to identify: • Essential Functions that cannot be interrupted • Essential Functions that can be interrupted, but must be back soon and how soon that is • Important Functions whose recovery can be delayed and how long • Systems, processes and assets already in place • Effect of the absence of functions on • Departments and buildings of the jurisdiction • Residents and businesses in the jurisdiction Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  32. Management Buy In (cont.) • The information gathering step provides data for Impact Analysis • If doing the entire jurisdiction it may be more than a one person job • Appoint a Study Coordinator • Identify others of like mind • Put a team together • If a team is not practical, consider surveys Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  33. Impact Analysis – Information Gathering Step 2 – Scope • For the Jurisdiction • How many buildings? • How many departments? • Which departments are housed in multiple buildings? • How many employees? • What are the dependencies? • How many Servers and Where are they? • Who operates in normal business hours? • Who operates outside normal hours? • Are there Organization Charts? Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  34. Impact Analysis – Information Gathering Step 3 – Functions & Ranking • Contact knowledgeable people from areas being studied • ID functions and priorities according to depts. • Tools available include Worksheets 1-5 and Annex A • Try to get both RTO and RPO to assist with ranking Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  35. Impact Analysis – Information Gathering Functions & Ranking (cont.) • Consider Revising Annex A to use something similar to this • Use it to give functions an initial ranking Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  36. Impact Analysis – Information Gathering Step 4 – IT Set Server Priorities • Use the functional priorities to set server priorities • Calculate preliminary RTO’s / RPO’s for each server in light of: • Current Backup Schedules • Current Replacement Hardware Availability • Current Alternate Facility Availability • Current Required building infrastructure Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  37. Impact Analysis – Information Gathering Step 5 – Reality Check • Virtually everything the jurisdiction does is going to depend on IT in some form. • How do the RTO’s and RPO’s desired by the operating departments compare with the Information Technology recovery capabilities? • Boil out any “happy talk” about what people think they can do without having ever tested it. • If the needs of the operating departments cannot be met, what next steps are appropriate? Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  38. Step 6 – Analysis of the Impacts • For each essential operating function, what is the impact on the jurisdiction if it is not functioning? • When is “x” hours/days/weeks intolerable? • Revenue lost • Public dissatisfaction with services • Bills not paid • Danger to the public • Danger to employees • Lost Records? Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  39. Analysis of the Impacts (cont.) • Based on analysis revisit and revise RTO’s / RPO’s • Operations (work with managers) • Information Technology • It is an iterative process Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  40. Risk Assessment • The Template has good reviews of Natural Hazards in the Kansas City Area. • The review of Technological Hazards is also worthwhile and jurisdictions need to review these for each building. • Where an asset is at high risk, it is important to take steps to mitigate the risk or relocate the asset. • An available tool (handout) has Risk Analysis Process questions each jurisdiction can use. Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  41. Situational Awareness • Each Department Head / Building Manager needs to be aware of what is nearby and what threats it might pose. • Railroads • Rivers (Barge Traffic, Dams & Levees) • Highways • Airports • Federal Office Buildings Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  42. Step 7 – Results to Leaders • Present differences between current recovery ability and operational RTO’s / RPO’s for essential functions • Present the costs of being unprepared, money, human cost, citizen dissatisfaction, bad press • Identify existing capacities and strengths • Resist assumptions that things will happen without preplanning Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  43. HOORAY (sort of) • Leaders say go ahead with COOP • Don’t spend much money • Don’t give up hope. Not everything happens at once. Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  44. Step 8 – Plan as Repository • Earlier we noted the COOP is a Repository for key information • Complete the Annex Development Process • Add new annexes as necessary. Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  45. Adding to Annexes • You already have some of this • Each jurisdiction decides how much they need • List of Essential Functions with revised RTO’s and RPO’s in Recovery Priority order by building or department (update annually) • Current Organization Chart (update annually) • Contact Information and a jurisdiction chosen method for using it (call trees to vendor supplied systems) • Employees (update semi-annually) • Local Emergency Numbers (update annually) • Regulatory entities (update annually) Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  46. Adding to Annexes (cont.) What? Go Kit? • Prepositioned Materials (Off Site Boxes) • Department start up (forms, schedules, reference material, custom stamps) • Jurisdiction Command Center if needed (tool kit) • Internal Dependencies (departments that depend on each other) (update annually) • External Dependencies (Vendors, Other Jurisdictions) (update semi-annually) • Work-in-progress recovery procedures (update annually) • Minimum Standards for Replacements (update annually) • Workstations • Desk Equipment Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  47. Adding to Annexes (cont.) • Desktop PC Hardware • Laptop PC Hardware • Desktop and Laptop Software Configuration • Personal Workstation Printers • Networked Printers • Copiers • Fax Machines • Scanners Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  48. Adding to Annexes (cont.) • Vital Records and Databases • Records that, if absent, would cripple services or the legal position of the jurisdiction • List by Department (identify if multiple users) • Classify – Paper or Electronic or Both • Develop Strategies to Recover or Rebuild • Work with IT on backup/archiving for Electronic Records • In the plan, note each vital record, it’s RTO and RPO • There are going to be some paper records than cannot be backed up or stored off site and the jurisdiction may have to assume the risk of loss Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  49. Step 9 – Management Structure • Establish who can “declare” and activate COOP • Establish EOC or Command Center, as needed • Coordinates recovery efforts between IT and Operations • Handles employee notifications / updates • Uses prepositioned supplies if needed (tool kit) • Reports to the top authority in the jurisdiction • Initiates damage assessment and repair of primary facilities • One objective of the EOC is to put itself out of business as soon as possible Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

  50. Where’s Information Technology? • This is the point in the planning process where IT planning becomes different than Operations • We will return to IT Step 10 after the remainder of the Operations steps Proprietary and Confidential. Do not disclose without the written authorization of Contingency Now Inc.

More Related