400 likes | 583 Views
IT Business Continuity. Does Your Firm Have The Right Contingency Plan?. Presented 6/17/2006 by: George G. McBride, CISSP, CISM. Business Continuity. What is it? Why do you need it? What happens if you don’t do anything? Who is supposed to do it? How much is it? What can you do?.
E N D
IT Business Continuity Does Your Firm Have The Right Contingency Plan? • Presented 6/17/2006 by: • George G. McBride, CISSP, CISM
Business Continuity • What is it? • Why do you need it? • What happens if you don’t do anything? • Who is supposed to do it? • How much is it? • What can you do? (C) 2006 Aon Consulting Corporation
The Reality: • “What we do is not glamorous, but it is important” • John Ellinger, Ohio State University • “Our plan will not hold up in the event of a wide-spread communications failure” • 52% of 669 Business Continuity Certified Professionals • “We do not have an Emergency Notification System” • 48% of the same 669 BC Certified Professionals (C) 2006 Aon Consulting Corporation
What is this stuff? • Some definitions to remember: • Business Continuity (BC): • Helps prevent operational interruptions, crises, and disasters…with planned levels of impact • The plan to making sure that you keep making money • Think “proactive” • Disaster Recovery (DR): • How do you recover and resume business operations after a disruptive event? • Typically an “Information Technology” group issue • Think “reactive” (C) 2006 Aon Consulting Corporation
Why the difference? • We started with tape back-ups where disaster recovery was the sole responsibility of the IT guys • Then we realized that the “dependent” departments and business units were also required • End-User recovery was developed into the DR plan to get the users back up and running • As business requirements changed and the need to recover all systems grew, recovery of all critical systems was required…but it was more than just the systems. • The business continuity plan was born (C) 2006 Aon Consulting Corporation
The root cause of many problems • DR is typically managed by IT and BC is typically managed by security, finance, or the business units individually • Companies, driven by corporate governance and regulatory compliance, built out the Business Continuity Planning position to “fill the gaps” • Example: systems outside the Data Center may be part of a business continuity plan, but a DR plan • Business Continuity Management serves to further reduce the gap by combining these efforts in a single “program office” (C) 2006 Aon Consulting Corporation
All of your tax money still doesn’t always help out our Government: • Department of Homeland Security report delivered on June 15, 2006 stated the following: • It criticized the states and cities in several key areas, including: • Failing to address emergency needs for sick, elderly or poor people unable to help themselves. • Being too slow to issue disaster warnings and other alerts to the public. • Failing to designate a clear chain of command during major disasters. • Also said: “...that nationwide response plans for major disasters are antiquated and often uncoordinated.” (C) 2006 Aon Consulting Corporation
Business Continuity Management (BCM) • The “holistic” management process of BCP and DR • Identifies and analyzes potential impacts that threaten an organization (think “threats”) • Definition of impact scenarios • Define recovery requirements • Builds solution and documentation • Defines implementation process • Testing and acceptance • Maintenance and updates • It is the “Framework” of the BCP and DR Efforts (C) 2006 Aon Consulting Corporation
BCM Program Structure • The BCM Program is best managed with senior leadership, information technology, and business unit personnel • BCP and DR functions may be separately managed, but must be coordinated and synchronized • Requires senior leadership to manage, gain required support, and champion (C) 2006 Aon Consulting Corporation
Just a few more definitions! • RPO: Recovery Point Objective • The time (relative to the incident) to which the data can be recovered • If you have overnight back-ups, the RPO will be the previous day • If you have a RPO of 10 minutes, you’ll need some type of data synchronization • RTO: Recovery Time Objective • The time that will pass before an infrastructure will become available • RTOs are generally per process (C) 2006 Aon Consulting Corporation
Do you really need it? • 43% of companies that suffer a catastrophic data loss due to a disaster never reopened (U of Texas Study) • 51% close within two years • 2 out of 5 companies that experience an IT systems disaster and loss of data go bankrupt in 5 years or less. (Gartner Group Survey) • 50% of companies experience “Significant Revenue Loss” with 4 hours of less of downtime (ESG Research Study) (C) 2006 Aon Consulting Corporation
Typical external incidents that would cause the “declaration” of an incident: • Terrorist attack or elevated threat level • Flood • Earthquake • Hurricane/Tornado/Tsunami • Riot / Civil Disruption • Levee Breaking • Accident involving the transportation of toxic chemicals • Disruption of network or telephone • Power Failure • Pandemic (C) 2006 Aon Consulting Corporation
Other internal incidents • Power Outage Factoid: • 35% result of local utility • 22% result of internal power failure • 10% result of natural disaster • Denial of Service (DOS) attack • Distributed DOS attack • Server/Hardware failure • Connectivity between sites affected • Loss of critical personnel or multiple personnel • Sabotage or perhaps theft • Hacking Activity • Fire or just smoke damage • Software upgrade gone bad • Data center cooling or UPS failure • Bomb threat (C) 2006 Aon Consulting Corporation
What does the BCP/DR Plan Answer: • How will employees communicate? • Not just with each other, but with customers, vendors, etc. • Where will they go to do their jobs? • Are there “alternate” sites? Sometimes working from home isn’t an option either. • What needs to be done? • What is core? Secondary? • Who will do it? • The staff? The support staff? The vendors and partners? • Do certain things take priority over others? (C) 2006 Aon Consulting Corporation
13 Must Have Steps for a BCM Program • These are the minimum that should be done • If you get this far, you’re probably ahead of 90%-95% of all other companies • This is a continuous process, not a one-off event • The goal is clear. The path may not be. (C) 2006 Aon Consulting Corporation
1: Business Impact Analysis (BIA) • A BIA identifies the businesses critical processes • Also identifies the components of the process such as data inputs, outputs, required systems, IT infrastructure, etc • Determines and measures the effect to the business of not having the system(s) and process(es) available • Defines financial, regulatory, business image impacts, etc. • May prioritize processes • The RTO and RPO values are also identified here (C) 2006 Aon Consulting Corporation
2: Contingency and Succession Plan • Just like the US Government…Who is in charge when the leadership cannot be found • Should identify key decision makers • Should pre-approve expenditures to some dollar amount to allow the data center manager the ability to purchase needed items without fear not being reimbursed • Named individuals on vendor and partner contracts to initiate services or request changes (C) 2006 Aon Consulting Corporation
Forward Challenge '06 • By William M. Arkin of the Washington PostSunday, June 4, 2006; B01 • On Monday, June 19, about 4,000 government workers representing more than 50 federal agencies from the State Department to the Commodity Futures Trading Commission will say goodbye to their families and set off for dozens of classified emergency facilities stretching from the Maryland and Virginia suburbs to the foothills of the Alleghenies. They will take to the bunkers in an "evacuation" that my sources describe as the largest "continuity of government" exercise ever conducted, a drill intended to prepare the U.S. government for an event even more catastrophic than the Sept. 11, 2001, attacks. (C) 2006 Aon Consulting Corporation
3: Rotation and Back-Up Personnel • Designate: • Primary • Secondary • Tertiary • Rotate designees through various portions of any tests and seek their input into BIAs and documents • We’re not just talking about rebuilding your mail server, but accounts payable, billing, etc (C) 2006 Aon Consulting Corporation
4: Crisis “Bunker” • Establish a primary location and back-up locations to where the management team will conduct operations • May be different then the back-up data center • In a crisis, you’ll be there quite often. Choose the location carefully: • Close enough to get to • Far enough away to not be impacted by the same incident (may want to choose a location with a separate electrical grid or water supply) • Moving from Baton Rouge to Miami to avoid the flood may not help you during hurricane season (C) 2006 Aon Consulting Corporation
5: Awareness And Availability • Conduct awareness and training sessions • For all affected employees • Senior leadership participation is critical to show support and dedication to efforts • Web home pages, printed, voice-mail announcements or whatever works in your organization • Make sure all employees are aware of the BCM Program and have access to the documentation when they need it. • Make sure that the BCP/DR plans aren’t stored exclusively on the server! (C) 2006 Aon Consulting Corporation
Where should the BCM Program documentation be stored? (C) 2006 Aon Consulting Corporation
6: Realistic Testing Exercises • Think “Big” and worst-case scenarios • Any event incurred will likely be less • Consider weather and environment • Peak billing cycles, during a class action suit, just prior to final arguments, etc • Through in some curves! Add some stress! • Have some fun though too • Tests range from a paper based walk-through to a complete test where the server room is unplugged and declared “unavailable”. (C) 2006 Aon Consulting Corporation
A picture of a “Realistic” Test (C) 2006 Aon Consulting Corporation
Why do complete tests? • Yes, it’s more expensive, time consuming, and “risky” • Here is what I have seen: • Lack of an effective plan for laptops • Don’t forget USB Memory sticks and data on iPODs! • Facilities took too long to become operational • Employees cooking out in the sun. Consider if it was raining or there was thunder and lightning • Ran out of food! • All of the people crammed into a mobile data center crammed with equipment resulted in system failure and soon enough, people failure! (C) 2006 Aon Consulting Corporation
Finally • Consider the World Trade Center towers after September 11, 2001: • The towers provided 20 million square feet of office space • During this time, there was only 10 million square feet of available office space in the city • How do your requirements fit in to the other businesses requirements? (C) 2006 Aon Consulting Corporation
7: Crisis Communications • Assume communications will be released to the public. • Clearly indicate plans, efforts to date, status of fellow workers, etc. • Should instill confidence and faith in employees • Should involve public relations and investor relations (if applicable) to manage public and press inquiries • Pre-developed scripts and communications • Just like some airlines use when there is an aviation incident (C) 2006 Aon Consulting Corporation
8: Alternate Communications and Notifications • How will employees who need to get their job done know what to do? • Incident declaration notification • Primary, secondary, and back-up contact numbers • Home, mobile, black-berry, e-mail, home address • Regular channels of communication for all employees of status: • How do you notify a building of 600 employees not to come to work because there is a flood in the data center and power is cut. (C) 2006 Aon Consulting Corporation
9: Develop Partnerships • You can’t do it alone • Partner with: • Emergency medical services • Red Cross • Police and Fire departments • Equipment Vendors • Software Vendors • Hot-site and equipment rental organizations • Other organizations through trade shows, professional affiliations, etc (C) 2006 Aon Consulting Corporation
10: Continually Improve • Test regularly: • Start with the paper based walk-through. You’ll find enough to fix with that. • Work your way up to the complete test where you pull the plug on the data center and bring all systems back up to production level and restore back-ups • Rotate people. Assume that the Managing Director isn’t available and test the contingency plan • Through a curve-ball here and there. Assume your primary back-up center is not available • Conduct a post-mortem (C) 2006 Aon Consulting Corporation
11: Adapt To Change • Everything changes frequently. • I’ve probably got more data on my 2 Gig USB Memory Stick then my laptop • Who is responsible for backing that up? • Blackberrys? Are they important? • New systems? New applications? • Client data is extremely dynamic? • Summation, Concordance, Attenex Information • Work that I did at home? Did I upload the data to the storage server? (C) 2006 Aon Consulting Corporation
12: Educate • Hire the best. Look for the following certifications: • Master Business Continuity Professional • Certified Business Continuity Professional • Industry leading certifications: • Business Continuity Institute • Disaster Recovery Institute International • Local ISACA, ISSA, IEEE, and ACM Organizations will have professionals interested in BCP/DR • SANS has some great papers as do many trade journals where you can see what others are doing in your industry • Contribute your own materials! (C) 2006 Aon Consulting Corporation
13: Check the view from the outside • A consultant can bring a global perspective of what other companies in a similar line of business are doing • Serves to provide advice and corrective actions to identify shortcoming and gaps that you may not have been able to identify • Is typically certified and experienced in a single area rather than most companies where the BCM team may wear many hats (C) 2006 Aon Consulting Corporation
Typical Mistakes Made • Lack of business inputs: • Business Continuity Management REQUIRES business unit input • Lack of support for management: • Management not committed = personnel not committed • Not demonstrating required level of effort • Business Impact Analysis study not complete or thorough (C) 2006 Aon Consulting Corporation
Mistakes • Too narrow of a scope. A “holistic” plan has not been developed • Lack of awareness. Too many times the plan was not available • Finally, NO, your Y2K Contingency plan will no longer work! (C) 2006 Aon Consulting Corporation
What can you expect to pay for BCM • University of Georgia says “It is not unusual for an enterprise to spend 25% of its information technology budget on disaster recovery.” • State of Illinois says “Industry trends, however, suggest that development, testing, and maintenance of a continuity plan usually amount to between 2% - 5% of the organization's IT budget” • My experience says that it is closer to the 5% amount (C) 2006 Aon Consulting Corporation
Cost components • Development and funding of the BCM Program (and perhaps Program Office) • Developing a BIA Methodology • Conducting the BIAs • Developing the BCP and DR plans for the assets and processes • Testing the plan • Implementation of corrective actions • Awareness and training • Vendor relationships, hot-sites, retainers, etc • Consultant costs (C) 2006 Aon Consulting Corporation
Checking Your Own Plan • Questions you can ask to see how effective your plan is: • As an “employee” of the company, you should have access to the business continuity plan • If not, can you find it? • What is the last update date? • Who maintains it? • Where is the master copy stored? • What processes were identified through the Business Impact Analysis? (C) 2006 Aon Consulting Corporation
Checking The Plan • More questions: • When was it tested? • To what extent was it tested? • Where is the back-up site? • Where do the personnel work from? • Were the issues identified in the testing post-mortem corrected in subsequent plan updates? • Are partner and vendor relationships up to date? • Have people updated their contact information? (C) 2006 Aon Consulting Corporation
Questions? • Feel free to ask now or contact me at: George G. McBride Aon Consulting 1 Industrial Way West Eatontown, NJ 07724 +1.732.544.8080 (Office) +1.732.544.0101 (Fax) George_McBride@aon.com (C) 2006 Aon Consulting Corporation