220 likes | 333 Views
MARFC Operational Backup A Case Study January 26, 2006. Outline. Problem Proposed Solution Results Issues. Problem. MARFC moving to new facility AWIPS unavailable for up to 6 days during move How to conduct operations during AWIPS outage Maintain full operational output
E N D
Outline • Problem • Proposed Solution • Results • Issues MARFC Operational Backup
Problem • MARFC moving to new facility • AWIPS unavailable for up to 6 days during move • How to conduct operations during AWIPS outage • Maintain full operational output • Official Products • All web information • Worst case planning? MARFC Operational Backup
Proposed Solution • Utilize in-house Linux server • X-client Laptops • ER WAN Connection • Data feed via LDM • Transmission via LDAD • Additional seats for flooding and support • AWIPS OB4 Basis MARFC Operational Backup
Proposed Solution - Server • Linux Server • Dell 4600 PowerEdge (late 2002) • Dual 2 GHz Xeon CPUs • 1 GB Memory • Raid 1 SCSI 73GB HDs • Dual Power Supplies • RH 7.3 MARFC Operational Backup
Proposed Solution - Access • 4 Operational seats • X access via 3 client laptops • Part of ER RFC Backup Project • 1600x1050 resolution • 1 ½ screens • External mouse and keyboard • Additional monitor for “non-AWIPS” display • Server console for HAS • 2 add’l clients for support and flooding MARFC Operational Backup
Proposed Solution - Comms • ER WAN Connection • Data feed via LDM • Redundancy via ERH and SRH • Transmission via LDAD • Redundancy via PBZ with ERH backup MARFC Operational Backup
Proposed Solution - Apps • Based on AWIPS OB4 • Synchronize local apps changes between AWIPS and server • Crons as well • Allow all auto processes to continue • Shut off delivery via tokens • Identify files to sync to go live • OFS, DB, control MARFC Operational Backup
Results • Difference in capabilities • No D2D • No ArcView use • FOP • Inundation mapping • No 12Planet (within AWIPS only) • 1 ½ vs. 3 screens MARFC Operational Backup
Results • Operations • Started with HAS AM shift 12/5/2005 • Product delivery easily shut off on AWIPS and initiated on backup via tokens • Continued in backup mode until Thursday, 12/8, morning hydro shift • AWIPS available for testing Wednesday night, 12/7 • Token reset shut off delivery from backup and initiated on AWIPS MARFC Operational Backup
Results • Performance • No forecaster perceived slowness in any operational application • No missed delivery of any text or graphical product • SUCCESS! MARFC Operational Backup
Issues • Keeping in-step with AWIPS • Upgraded to OB5 on AWIPS prior to move • Stayed on OB4 due to DB table changes • Not sure of best approach to incorporate changes of this nature • OB6? • Help with problems • Maintenance overhead – 2 systems • Overall RFC solution? MARFC Operational Backup
Issues • Redundancy is critical! • 1 hour into backup operations one HD on server failed • Continued with one HD with no problems MARFC Operational Backup
Issues • Off-site computing capabilities • If incapable of full operations support, what do you cut out • Customers expecting information • HD failure forced laptop server implementation MARFC Operational Backup
Issues • Off-site computing capabilities • Part of ER RFC Backup project • Dell 5150 “desktop replacement” laptop • 1 year old • Pentium 4 3.0 GHz • 1 GB Memory • 1 100GB HD; 7200 RPM ATA MARFC Operational Backup
Issues • Off-site computing capabilities • Operations test • 3 client laptops • Laptop display • Full cron load • Morning forecast operations MARFC Operational Backup
Issues • Off-site computing capabilities • Operations test – results • Slowdowns • NMAP initiation • Heavy disk access – e.g. Informix extractions • Slower than in-house server • Acceptable by forecast staff knowing limitations MARFC Operational Backup
Performance Comparison • Identical cron instructions • Handoff and execution of OFS jobs from AWIPS • Laptop slowdown with heavy disk access activity • More analysis needed MARFC Operational Backup
updatedb, db_purge, sys_clean ??? db_purge MARFC Operational Backup
Off-site Capabilities • “Shoebox” server, aka ShuttlePC • “Server-like”; luggable • Dual CPUs • Multi-GB memory capabilities • 7200 RPM SATA II • SCSI-like performance? • Larger, cheaper than SCSI • RAID 1 MARFC Operational Backup
Off-site Capabilities • Less than $3000 • Dual 64-bit AMD Opteron, 2.4 GHz • 2 GB memory • Dual 120 GB 7200 RPM SATA II • RAID 1 • GB ethernet • 19” LCD • External keyboard and mouse MARFC Operational Backup