100 likes | 112 Views
Vodafone ES NOC runbook Automation with HP OO. Ricardo Lopez Cencerrado 30/08/2012. NOC runbook automation: timeline. VF ES internal Automation initiative launched Jan 2009 Target was a Pilot with HP OO for NOC alarm T&D Pilot definition and VF ES internal ROI evaluation
E N D
Vodafone ESNOC runbook Automation with HP OO Ricardo Lopez Cencerrado 30/08/2012
NOC runbook automation: timeline • VF ES internalAutomationinitiativelaunchedJan 2009 • Target was a Pilotwith HP OO for NOC alarm T&D • Pilotdefinition and VF ES internal ROI evaluation • PilotscopediscussionstartonMay 2009 • Pilotscopeagreed/presentedby Nov 2010 • Pilotimplementation time (2 months) • Pilotconclussion + ROI pptbyMarch 2011 • Project start: June 2011 • Final Acceptance: March 2012 (in productionnow) • Sales cycle can be long: • Automationaffects NOC organization/people
Pilot Objective: Operations Automation • Operation Automation definition by VF ES: • “Run-book automation tools orchestrate the automation of IT operations management processes, unifying the communication of the underlying, supporting, IT management tools in support of the process” • Focus of the pilot for OA tool: Incident Management • Validate HP Operations Orchestration (HP OO) as an efficient tool for Operation Automation at Vodafone Spain NOC (dual NOC with TeMIP & OVO). • ROI exercise result is critical for any decision on future project for HP OO deployment • Validation is both Technical and Economical • HP CMS Objective: Start VF ES into thinking in the automation of current human operator tasks (OpEx reduction as focus) • And get the automation flow definition and deployment project for CMS services
HP OO Pilot: scenario and scope • HW/SW platform: standard VF ES vmware configuration for Microsoft based applications • 1 single server for all HP OO components: • Studio, Engine, BBDD, etc. • Pilot Test cases to implement 6, selected from 12 candidates: • All 6 cases follow a typical flow: • Event (TeMIP) -> diagnostic -> Repair -> Escalate (Remedy ARS) • 3 cases based on Telco Network Equipment interation • CLI interaction with equipment (Ericcson AXE) • 3 cases based on platform or application management: • Backup, server memory usage, apps comms (Tuxedo EAI check)
HP OO Pilot: Challenges • Interfaces withTelcoEquipment are new to HP OO • Reuse of predefined HP OO availablerunbooksislimited • Interface withRemedy ARS v5.1 at VF ES is GUI based “cut & paste”, notsuitedtoautomation • Not a nice WDSL basedintegrationpossible • Short project plan (4 weeks) and no previousexperience (CMS implementsthePilot/Project) • ROI calculationto be performedby VF ES (!) withsupportby HP • Detailedidentification of feasible OO automationsto be performedby VF ES personnel • HP OO Pilotresultswerepossitive (and ROI validatedby VF ES)
HP OO Project: scope • HW/SW environmentvalidatedby HP and VF ES • Installation kit prepared and readytoinstallby HP CMS (onsite) • Remoteaccessto VF ES pilotpremisesforprojectimplemetnation • Final scope: 14 flowsimplemented in HP OO • All use cases are manuallytrigerredfromTeMIPconsole • Allexecuteaccesstoexternalsystems/core VF ES network • VF ES corenetworkaccessisexecutedby HP ServiceActivator • All use cases includeautomatic TT creation (Remedy ARS)
VF ES OO NOC Automation: Project details: Project Economics Final SW price was reduced to 20K€ (HP OO pricing update –> Customer happy) Final project order around 150 K€
HP Internal: VF ES ROI case • Assumptions (providedby VF ES): • Eachflow (automation use case) has a cost of 8 FTE/month • At Pilot time weestimated 20 use cases : 160 FTE per monthsavings • VF ES NOC Operatorcost per year: 30K € • At pilot time: Automation use cases toimplement: 20 de uso • Investement: • CAPEX: 194K € (Initialproposal --> Final orderwas 150 K€) • OPEX: 28K € (Year) • Coststotals: • 20 use cases x 8 FTE/month 160 FTE/month 8 NOC Operators (can be amortized) • 1 year: 240K € cost • 3 year: 720K € cost • ROI: • 1st year: 240 – 194 = + 46K € (Possitive ROI in 1st year) • 3th year: 240*3 – 194 – 28*2 = + 470K € • Note: withthereduced SW pricethe ROI isevenbetter • Butfor HP: projectimplementationwascomplex/difficult (services) and very time consuming (HP OO flowsfor NOC are verycomplex !!)