1 / 34

SGPPL eB2.1 Conversion

SGPPL eB2.1 Conversion. Operational Review. Our Task:. To convert 24 US databases and 11 European databases, 109GB of data, from Progress 8.3E04/MFG/Pro 9.0 SP2 to Progress 9.1D08/MFG/Pro eb2.1 Domain in a timely, safe, and efficient manner

miriam
Download Presentation

SGPPL eB2.1 Conversion

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SGPPL eB2.1 Conversion Operational Review

  2. Our Task: • To convert 24 US databases and 11 European databases, 109GB of data, from Progress 8.3E04/MFG/Pro 9.0 SP2 to Progress 9.1D08/MFG/Pro eb2.1 Domain in a timely, safe, and efficient manner • Target date for US conversion: Thanksgiving weekend 11/25 – 11/28 • Target date for European conversion: TBD

  3. Our Plan: • Divide and conquer 1.) Data Maintenance 2.) Hardware platform 3.) New QAD environment needed: conversion/production • Conversion Planning/Testing • Conversion • Project begins - March 2004

  4. 1.) Data Maintenance • Of the combined 35 databases, some contained 8 years of data • Data needs to be archived • Some have never been Dumped and Loaded • US range - 210MB-15GB/Avg. 3.3GB • Eur. range - 550MB-8.5GB/Avg. 2.5GB • Once archiving is completed, Dump and Loads are needed.

  5. 2.) Hardware platform • Hardware environment - UStwo HP9000 RP7410 :Central - 4 procs/8GB mem Aurora – 4 procs/8GB mem • EMC CX400 SAN2 trays of 15 - 36GB disks, raid 1_0 hardware mirrored • Internal fiber connection between the two servers for COP transactions

  6. 2.) Hardware platform • Hardware env – Europe- one HP9000 N4000: Eurqad 2 procs/6GB mem • - locally connected JBOD drives, 2 trays of 18GB disk drives, mirrored with MirrorDisk/UX

  7. 2.) Hardware platform Do we have…. - Horsepower to do the conversions - Disk space to convert in - Seamless, timely rollback option. …………..??????

  8. 3.)New QAD environment • “Early Adopters” of eB2.1 • New Progress/QAD software needs to be installed on development/test server • Single source architecture • All building block databases need to be created for all different languages including new double-byte Chinese • 2000 customized programs that needed modifications • Bolt-ons: Euro Acctg, Eagle, Optio, Cameleon

  9. 3.) New QAD environment • Mfgutil “work” area • New scripts, utilities, tools needed for multiple conversion tests • Existing monitoring tools, scripts need to modified/checked • “In-place” conversion our ONLY option “Buffer copy” not available yet • Conversion to Progress “storage area” unlikely

  10. Data Maintenance • Archive/delete plan developed to purge all databases down to 4 years and approved • Existing scripts reviewed, modified • Twelve largest tables targeted • Archive/delete and subsequent reload of deleted data tested in an isolated test environment .

  11. Data Maintenance • AI and BI constraints forced weekend archiving • Process continued throughout the summer • Initial planning would have left us with nearly 5 years of data by conversion time • Process re-evaluated, databases revisited • Dump and Loads loom

  12. Hardware Solution • HP9000 RP7410 rented from HP • New EMC CX500 purchased with two trays of 73GB spindles - this purchase was approved to not only help us with this conversion and provide a seamless rollback option BUT also to give us enough capacity to migrate our European dBs onto a SAN solution and to separate our HP-UX and NT envs. .

  13. Hardware Solution • Plan: 1.) Convert the US dBs using Loaner server and new SAN 2.) vgexport/vgimport new file systems to their respective Production server. This plan meets all of our conversion objectives, and upon successful completion, allows us to mimic the same strategy for our European conversion, using the Loaner server and now available CX400 drives.

  14. Hardware Solution • Loaner cloned with HP-UX 11i and connected in parallel with US servers and CX400. All servers “see” the CX400 • New CX500 assembled and then added to above cluster. All servers “see” both SANS • Initial vgexport/vgimport tested from Loaner to Production server and proved successful.

  15. Hardware Solution

  16. Hardware Solution • New VGs with cloned mountpoints for all database file systems created • Utilities created to mount/unmount production and cloned file systems, perform all vgexports/vgimports, and make any redundancy/load balance corrections. • Hardware infrastructure in place

  17. Data Maintenance • eB2.1 release date pushed back • A Thanksgiving conversion grows more and more unlikely • Downtime requested over the four day weekend for Dump and Loads • New hardware in place would provide the same power, capacity, and rollback options

  18. Data Maintenance • Thanksgiving morning – dBs shutdown, backed up, restored to cloned file systems, and D/Ls begin • 24 US dBs took nearly 24 hours, 10 more hours to backup, restore, batches, etc • Started with 81.5GBs of Active 8K blocks, finished with 60GBs – reclaimed nearly 27% • US range - 170MB-12GB/Avg. 2.5GB • Data in better state – ready to convert

  19. New QAD environment • Complete Progress/QAD environment created on new platform for both conversion process and post-conversion production • Scripts, utilities, tools developed and continually modified • Upgrades to source and numerous patches installed • Programming continues…

  20. Conversion Planning/Testing • Early on, initial out-of-the-book conversion with mfgutil done on our smallest dB (210MB) in test environment just to test process • Estimates of total conversion time extrapolated out from that initial test neared 60 hours • Focus on US - Can’t forget Europe

  21. Conversion Planning/Testing Major Issues: • No longer have a 4 day weekend • Bugs found in conversion programs - BOTH envs. need to be converted with same programs • Only single instance conversion tests have been run so far • No automatic way of launching multiple conversions at once • Is it physically possible to convert the entire environment over a 2 day weekend?

  22. Conversion Planning/Testing On the plus side: • Data is in a much better state • Faster, more powerful conversion infrastructure in place • QAD manipulated mfgutil .ini files to allow a pause, after an up-front, one time interactive data entry for ALL dBs, allowing us to stagger conversions based on pre-determined ‘sleep’ times

  23. Conversion Planning/Testing • First test of complete environment on weekend of 12/04/05 • Entire conversion completed in just under 12 hours • Data maintenance and new hardware platform accounted for an 83% drop in conversion time based on initial test

  24. Conversion Planning/Testing • Analysts log into Loaner server, verify conversion • Data and all process log files scrutinized • Bugs found and reported • More conversion tests planned - sleep times/launch order reviewed/adjusted - conversion times continue to improve • Several “mirror” tests ran • Programming continues…

  25. Conversion Planning/Testing • Vgexport/vgimport utilities finalized - can’t be tested • Some European tasks performed simultaneously all along • With all factors accounted for: Feb. 18th for US / March 18th for Europe chosen as conversion dates

  26. Conversion US - 2/18/05 • Environment shut down for “pre” reports, backup and conversion at 6 PM • Conversions launched around 11 PM • Despite 2 more months of data, conversion time whittled down to 9 hours and 40 minutes • Verification happening throughout Saturday 2/19 - pre & post reports, business flow testing, etc

  27. Conversion US - 2/18/05 • “Go/No Go” decision was made - “GO” • CX400 (9.0) file systems unmounted, vgexports/vgimports ran, new CX500 (eB2.1) files systems mounted

  28. Conversion US - 2/18/05 • Backups, batches ran • System turned over to analysts for final conversion verification, Saturday 2/19, 3:00PM Conversion Results: - Largest COP dB, 12GBs, converted in 7:20 - Largest MFG dB, 8GBs, converted in 4:20 - Smallest MFG dB, 170MBs, converted 0:10

  29. Conversion US - 2/18/05 “Better to be lucky than good” • Couldn’t effectively load test 700 users • Monday 2/21/05 President’s Day - Central COP server experiencing memory “pressure” - half of our sites off, users not as savvy - issue identified as increase in “working set” of each process - server/OS performed admirably - memory purchased and added Tuesday evening • US complete

  30. Conversion Europe • 36GB spindles removed from CX400 and installed into CX500 - 2/26/05 • Fiber card adapters installed into Eurqad, connected to expanded CX500.

  31. Conversion Europe • European file systems created on CX500 • Databases archived and Dumped and Loaded - 12/10-12/11. • 11 dBs took 7 hours, backup/restore/batches another 4 hours • Started with 27.5GBs of Active 8K blocks, finished with 22.4GBs – reclaimed nearly 19% • Eur. range - 510MB-6.9GB/Avg. 2.1GB • Test conversions/verifications/mods happening all along • Programming continues…

  32. Conversion Europe • New VGs with cloned mountpoints for all Euro. database file systems created • Utilities modified to mount/unmount production and cloned file systems, perform all vgexports/vgimports, and make any redundancy/load balance corrections. • Exact same conversion strategy would be followed

  33. Conversion Europe - 3/18/05 • Plan: 1.) Convert the Euro. dBs using Loaner server and newly added 36GB spindles 2.) vgexport/vgimport new file systems to the Prod. server. - Once again, seamless rollback to the JBOD - Upon successful completion, Euro dBs on a much faster disk solution • By conversion weekend, total Euro conversion time reduced to 5 hrs. 30 min.

  34. Today’s Environment

More Related