1 / 33

Automation to the Rescue

Automation to the Rescue. Jim Malnati and Robert Jamieson. Agenda. Client Experiences What else can Operations Sentinel do? Futures Summary. TravelSky Technologies Ltd. Major provider of airline applications in China Major applications for reservation, inventory and departure

nitza
Download Presentation

Automation to the Rescue

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Automation to the Rescue Jim Malnati and Robert Jamieson UTF 2008

  2. Agenda • Client Experiences • What else can Operations Sentinel do? • Futures • Summary UTF 2008

  3. TravelSky Technologies Ltd • Major provider of airline applications in China • Major applications for reservation, inventory and departure • Significant year-on-year growth plus the Olympics • 8 production OS2200 partitions on 3 Dorado servers • Plus many dev/test/DR partitions/servers • 2 DMX subsystems for each appl – SRDF to DR • Over 100 ES7000 partitions for OMSE • CRS txn rates forecast at 4,000/sec and 50,000 IO/sec • Scalability, availability and response critical UTF 2008

  4. Challenges at TravelSky • Increased complexity without additional staff • 3 to 8 partitions • Increased txn workload • Increased IOs • Increased batch jobs • Shared resources e.g. tape drives, locking • Olympics 2008 has meant significant political pressure • SPO and Opcon/xps are relied upon UTF 2008

  5. SPO at TravelSky • OS2200 console • Automated operator key-ins • Monitor key system variables • Alert and automated actions if exception detected • Pass KPIs to OMSE (ES7000) for network traffic balancing • SOMA used to automate tape drive sharing • DMX automation and monitoring • SRDF Async/Sync/Semi-Sync, RDF ports etc • Consolidated console and logging tool UTF 2008

  6. DMX Monitoring and Alerting • Probe function for monitoring • SRDF state, RDF directors, RDF link • Configurable with default of every 5 mins • Alerts raised for errors and command success • Severity used to differentiate • Informational to Critical • Use Alert Details to get help text (saved IP from experts) • STATUS application for graphical single view • Icon colours indicate state or problem UTF 2008

  7. SRDF Synched Port offline SRDF Split Probe ON Status Diagram – Colours indicate States UTF 2008

  8. SRDF Function Triggers • SPO Console Application Menu UTF 2008

  9. SPO SRDF Alerts and Help Text UTF 2008

  10. Logging • Numerous log files maintained including symm_host UTF 2008

  11. Opcon/xps at Travelsky • Task based license – 600 tasks per day for OS2200 • Job scheduling for most OS2200 batch work • Database dumps • System file backups • Application jobs • Simplified upgrade from 3 to 8 production hosts • Planning upgrade to new Opcon/xps release • Planning for improved SPO-Opcon/xps integration UTF 2008

  12. SPO at Asian bank (Libra client) • Monitor MCP console traffic • Alert if no payment gateway transactions received in 10 secs • Alert if more than 10 txn timeouts in 10 secs • Automated operator key-in if deadlock detected • SRDF monitoring and automation • Metering usage monitoring and logging • Proved they didn’t need to worry about runaway tasks • Showed what their processing profile was like UTF 2008

  13. Deadlock Automation • Problem • Deadlock with IBM port caused all txns to queue • Credit cards could not be used • Analysis • Used to rely on end-user advising operator advising support • Take 5-30 minutes to resolve while many txn blocked • Development of SPO script in 30 mins • Resolution occurs in under 30ms with informational alert raised • Benefit • No loss of revenue, no user/operator/support cost, satisfied card user UTF 2008

  14. ClusterDisplay CurrentODTs SPO Workstation Desktop AlertIcon STATUSfor DMX UTF 2008

  15. SPO Status Diagram for DMX • Initialised state indicated by blue colour • Red indicates alert – probe not active, R1-R2 not in sync • Yellow indicates RBCV is split UTF 2008

  16. SPO Menu Driven Functions • Submit check of the RDF status UTF 2008

  17. Other Automation users • 3 Korean banks use DMX automation • Aust DoD, Cathay, EDS, LTNZ and others use Opcon/xps • Angkasa (MCP), LTNZ, EDS use SPO • SPO Basic Edition comes with IOE – MCP and OS2200 • Monitor the ClearPath system • Needs Window server • Some training and consulting UTF 2008

  18. What Else Can SPO Do? • Monitor ClearPath, Windows, Unix, AS/400, environmental devices etc • Including tape library controllers (ACSLS, VTLs) • Integration with other tools • AppManager, problem ticket systems, SNMP feeds • Opcon/xps • Provides a unified console and logging facility • Can provide basic BPM UTF 2008

  19. Fidelity Information Services (FIS) • Large MCP user: provides banking application services • FIS understands the value of automation • Cost reduction program (SPO to replace other mgmt apps) • Focus on improving quality to customers • Needed to upgrade SPO 6 (unsupported and on UnixWare) • SPO 9 now in production; Operations Sentinel 11 trial • Late 2007: • SPO 9 Getting Started Service • SPO 9 Formal Training • Operations Sentinel 11 trial implementation UTF 2008

  20. Operations Sentinel 11 Dev. Goals • Complete rewrite of SPO graphical interface • Leverage Microsoft investments in GUI technology • Take advantage of .NET, C#, XML, etc • Remove dependence on 3rd-party products (SNAP/Geneva) • Provide a Microsoft-compliant interface • Implement global configuration • Standardise tools and utilities (common look and feel) • Implement customer-requested features UTF 2008

  21. SPO 9 Graphical Interface UTF 2008

  22. Operations Sentinel 11 Demonstration UTF 2008

  23. SPO Audible Alerting (SPAA) • Visual alerts are essential: some require audible emphasis • Need to synchronise ACK/clear of alerts • SPAA C# program installed on each SPO Workstation • SPO Alert Policy specifies: • The sound file to be played • List of workstations on which to play it • Option to kill the SPAA GUI (stop sound) after time period • Test program to drive SPAA with event reports UTF 2008

  24. SPO Audible Alerting (SPAA) UTF 2008

  25. UTF 2008

  26. Shared Object Manager Application (SOMA) • Tape drives shared between Unisys, IBM, Tandem, and Sun hosts • SOMA means fewer tape drives • Error-free movement of driveson an as-needed basis • Multiple “tape pools”configured • Individual drives and/or hosts excluded • SPO 11 SOMA soon SOMA SPO andOpCon/xps Windows 2000or 2003 Secure Wide Area Network UTF 2008

  27. Less Than Optimal Drive Sharing UTF 2008

  28. A More Cost-effective Approach UTF 2008

  29. Operations Is Affected • Greater equipment efficiency requires manual intervention by Operations staff • Operators are diverted from system monitoring • Many systems must be examined for surplus tape drives • Repetitive, manual keyins required on multiple systems • Entire process is time consuming and error prone, especially with heterogeneous hosts • Batch processing is delayed while operatorslook for available drives UTF 2008

  30. Overview of SOMA • Automatic, error-free movement of shared tape drives between heterogeneous systems (Unisys, IBM, Tandem, etc) • Configured package of SOMA and SPO automation • All actions logged, for audit trail and metrics • SPO Status provides a “head-up display” so Operations can easily track tape drive movement • Built from standard Unisys SPO product and the SPO C# API UTF 2008

  31. HOSTA HOSTA HOSTB Tape Drive Sharing Without Automation Operator searches other hosts for free tape drives while HOSTA waits. HOSTA: “I need 4 tape drives. Now!” 1 2 Host with a free drive is found and the operator “downs” the drive through system console keyins. 3 HOSTC HOSTD 4 If the tape drive is downed correctly the operator can bring it up on HOSTA. HOSTA: “Well, that’s 1 of 4….” 5 The cycle repeats, as other important console events are ignored and the system waits. 6 UTF 2008

  32. HOSTB HOSTA HOSTA HOSTA Automated Process Flow 1 “I need (4) drives of type (high-density-tape-pool)” SOMA 2 Automated Shared Object Manager: “Get (drive) from HOSTB” 3 4 “Down (drive) for me” “OK, (drive) is down” 5 “UP (drive)” 6 The cycle repeats three more times, without operator intervention. UTF 2008

  33. UTF 2008

More Related