210 likes | 303 Views
9/11 - Respond and Recover: A Case Study. Joy Heath-Porter Sidley Austin Brown & Wood LLP LawNet 2002 August 21, 2002. Who are we?. Large, multi-office, multi-national firm 14 offices; six countries 1,500 lawyers; 3,000 total personnel
E N D
9/11 - Respond andRecover: A Case Study Joy Heath-PorterSidley Austin Brown & Wood LLPLawNet 2002August 21, 2002
Who are we? • Large, multi-office, multi-national firm • 14 offices; six countries • 1,500 lawyers; 3,000 total personnel • Two offices in New York before September 11; 600 person office in the North Tower of the World Trade Center
Status on September 10 • Involved in one of the largest law firm mergers ever • Two networks with different hubs • Two different e-mail and document management systems • Two different telephone systems
Status on September 10(cont’d.) • Same financial system, but on different versions with different database engine • Still in the process of consolidating technology • Had just moved equipment to WTC previous weekend
Brief Chronology • 7:50 a.m. Tuesday - how we found out; first steps • Mid-morning Tuesday - disaster recovery steering group assembled; began to map out recovery strategy • Noon Tuesday - offsite storage provider contacted; arrangements made to drive tapes to Chicago
Brief Chronology (cont’d.) • Throughout Tuesday - made sure phones were manned, so that staff calling in would have a “live” voice; determined status of our people; identified and arranged for recovery resources; Chicago and New York offices “closed,” but many staff members were working • C.O.B. Tuesday - system recovery priorities established; servers identified for recovery
Brief Chronology (cont’d.) • Tuesday night - 800 dial-in information numbers established and published; information made available via the Firm’s Web site; temporary space identified • Wednesday morning - all offices contacted for spare equipment; arrangements made for rental equipment; PC vendor contacted for purchase of equipment; cabling began
Brief Chronology (cont’d.) • Wednesday noon - Tapes arrived; restoration process began • Wednesday p.m. - voice mail accounts on alternate voice system began to be created for lawyers • Wednesday night- new e-mail server built; account creation in process; Internet mail redirected (brownwoodlaw.com)
Brief Chronology (cont’d.) • Thursday a.m. - all lawyers had e-mail accounts and mail was flowing; equipment was shipped out from other Firm offices • Thursday p.m. - Internet café established in alternate NY sight; image build began for rental equipment • Thursday night - secure extranet site established for technology information
Brief Chronology (cont’d.) • Friday a.m. - document management was available to NY users • Friday noon - equipment began arriving from other offices and from vendor • Friday p.m. - financial systems restoration was complete and consolidation began; image for rentals and new PCs finalized
Brief Chronology (cont’d.) • Saturday a.m. - Rental equipment began arriving; imaging began; teams from Chicago began driving to New York transporting equipment not available in New York (cell phones, card keys, etc.) • Throughout weekend - Phones, network equipment, printers, desks, PCs rolled out in temporary space
Six Days Later • Monday 10:00 a.m. - all New York personnel assembled in a mid-town hotel for a briefing • Monday noon - all New York personnel reported for work in temporary space
Today • All servers returned to New York • All technology migrated to firm standards • New York office moved into new, permanent home over July 4 weekend
Key Aspects of Disaster Recovery Approach • “Maniacal” approach to tape backups - daily offsite storage of tapes; offsite location outside metropolitan area • Focus on running our daily operation to perfection • Standardized hardware (network, PCs, phones) with standard images
Key Aspects of Disaster Recovery Approach (cont’d.) • Emphasis on documentation and standard operating procedures • Commitment to serving our lawyers the way our lawyers serve the Firm's clients
Lessons Learned/Affirmed • You cannot plan for a plane hitting your building - accept the fact that you will have to improvise to some degree in virtually any disaster • You cannot have too many friends - clients, vendors, co-counsel, even “competitors” • Practice, practice, practice - staff need to be able to perform basic disaster recovery tasks "in their sleep"
Lessons Learned/Affirmed(cont’d.) • If you have multiple offices, leverage them • The fact of back-ups is not enough - they must be quality-checked, verified, and viable • Efficiencies of centralization must be weighed against risks from a disaster recovery perspective
Lessons Learned/Affirmed(cont’d.) • Recruit for character and creativity, as well as technical expertise • Understanding the relative priority of your critical applications cannot be overstated - in a disaster situation, restoring contact information may be more important than restoring documents
Lessons Learned/Affirmed(cont’d.) • Web access to your systems is not a luxury; it is a necessity • Stress with principals the importance of using firm-standard repositories - if documents are in three different places, restoring access becomes much more difficult
Lessons Learned/Affirmed(cont’d.) • Think about disaster recovery when making a buy/build decision • Smart, dedicated people ultimately make it happen
How does theStory End? • We were never “down” • Clients were impressed • Our most important assets - our smart and dedicated people - were not lost • The loss of people is the only disaster truly to fear - if you are prepared, everything else can be recovered or replaced