Experiences Using SNOW in IT

1. Experiences Using SNOW in IT Emmanuel Ormancey (IT/OIS) MaiteBarroso Lopez (IT/PES) Massimo Lamanna (IT/DSS)

2. Introduction • ITIL, the process • Service catalogue • SNOW, the tool • Conclusions Feedback on Snow - 2

3. Introduction • After more than one year of using SNOW in production, present the experience of 3 groups in IT • DSS (Data & Storage Services) • OIS (Operating System & Infrastructure Services) • PES (Platform & Engineering Services) • All three heavily using SNOW in different ways • This is not a complete survey but more a tool to trigger a department-wide discussion to gather feedback from other groups • And to discuss the next steps to be taken Feedback on Snow - 3

4. ITIL: the process • Processes being in use: • Incident management • Request fulfillment • Basic principles already known to IT • Main changes: • Introduction of CERN-wide service desk • Easy transition • Difference between incident and request • Users not ITIL-compliant, still not clear distinction • Different tool (will be covered later) • Still some aspects to be implemented: • SLAs, KPIs, usage of priority/impact/urgency: • E.g.: We don’t have a set of agreed KPIs per process, with automated monitoring, so FE and SE owners can easily see progress Feedback on Snow - 4

5. ITIL: the process • Other processes discussed: • Problem management • Change management • Introduction of new processes: • Discussion with all IT groups (usually 1-2 meetings) where the process is described, with little implementation details (a demo) • Feedback from groups (through individual mails?) • One day the process will be implemented in SNOW • We need a plan, with milestones, dates, details of what will be implemented when • A pilot phase with 1-2 groups to get early feedback? Feedback on Snow - 5

6. Service catalogue • It should be the only place where a comprehensive set of services being delivered are declared. • Not the only: we still have SDB • Complex matrix of CS, SE, FE • Built over a long period, before real usage • Originally designed to hide IT internal structure (FEs) to users, only high level services • Finally FEs are exposed to users at the service portal, through the search facility • They have become the real services we work with, both for users and service managers • The high level view, or user view, that was supposed to come out from the SEs and CSs, is not widely used. Feedback on Snow - 6

7. Service catalogue • User perspective not clear • Structure too complex for the end user • What is the difference between FE and SE for the user ? Feedback on Snow - 7

8. S-NOW Portal • One-stop entry point for our services • Locating information • Please note: incomplete list of tools: • Services web sites (old and new Drupal infrastructure, twikis), Savannah, Remedy PRMS, Remedy ITCM, GGUS, GOCDB, ITSSB, ServiceDB, Lemon, SLS, … • Working environment to do support • Ticket handling • Reporting Feedback on Snow - 8

9. S-NOW, the tool • Important investment • By the support team • In-house team responsive to bugs • By the services adopting S-NOW • Migration + workarounds during transition • One of major reasons for this choice was “out of the box tool used elsewhere” • Usual dilemma between: “use it as it is (and adapt yourself)” and “endless adapt it to your needs” • Unclear where we are • Additional modules providing more functionality exists • Lightweight integration desirable • API? • Smooth transition • Common effort to “protect” the users! Feedback on Snow - 9

10. S-NOW, Locating information Search for ‘Mail Android’ FE/SE mail • The portal offers nice functionality for users • “Search for” functionality • Some glitches are the reflection of the complex CS/SE/FE network • e.g. “A+” service Android/I-Phone Create Egroups Report Spam Email web access Problem with Outlook

11. S-NOW, general interface • Usability: frame layout and general URL schema • Difficult to navigate between pages, open links in separate windows, copy/paste/save links etc. using standard web browser features/navigation • Inefficient. Needs investment • SSO: • Was the SSO standard compliance a requirement ? • Partially implementation of the SSO standards SAML2 & WS-Federation • Login works, logout fails • User is redirected to SNOW instead (close the browser) • Hacks made by IT (no S-NOW support for this) • IT hacks are being evaluated, but not yet successful Feedback on Snow - 11

12. S-NOW as a reporting tool • Too much emphasis to the shiny reporting capabilities: • What is really useful? • Indicators like Time_to_Solve, Time_to_Route, “#3rd level”/”#2nd level” are interesting • OK for auditing the Service Desk • Much less interesting for day-by-day service managements • Custom reporting (e.g. CastorTape) missing • Simple indicators not sensitive to the quality delivered and to the contribution of the various teams • Example from CASTOR 2nd level support: • Tickets are from expert users • 2nd level helps us to solve by collecting information • Which is great! • But strictly speaking no ticket is closed by them • Hence their contribution is not visible in the indicators • But greatly appreciated anyway  Feedback on Snow - 12

13. S-NOW as a content management system (1/2) • Knowledge Base library • Full life cycle: editing, workflow, feedback, … • Some glitches: IMHO the main one is the shaky editor • More important: • Difficult to organise/present (the only structure is the FE association) • On the other hand very convenient for 2nd level Feedback on Snow - 13

14. S-NOW as a content management system (2/2) • Entities like CS/SE/FE have their respective web page • Not useful to give a basic overview our services • Not reach enough • Duplication of Drupal web pages • SLA/SLD • SLA/SLD as description of the service • Now in the IT web infrastructure • SLA is not ticket handling tim Presentation title - 14

15. S-NOW: ticket handling • Superior features compared to Remedy • Web application so no client to install • Better interface, easier to use • Transparency, re-assignment of tickets, comments field and visibility • Interfaced to GGUS • Record producers • More granularity • More SE / FE • Better case assignment to the FE • Good idea of urgency/priority even if not used • To be improved: • Although no show stopper a long list of improvements exists (see later) Feedback on Snow - 16

16. S-NOW, the tool • Implementation of incident management: • Lacking rota feature, last meeting in July, as of today still no date, known plan • We have survived with tricks, in all this time we could have learnt to work otherwise? • No usage of urgency/impact/priority • Only impact can be changed by service manager • Priorities come through other sources • No SLAs implemented • Some incidents can stay there for a long time without proactive checking, no reminders sent • When the feature was enabled, S-NOW was spamming us • No time limit for user to respond (Waiting for user) • Only the FE is compulsory to be filled, so we work with FEs and no SEs Feedback on Snow - 17

17. S-NOW: ticket handling • Integration with other systems • It turned out to be complex and not yet completed • The integration with GGUS is OK • But REMEDY ITCM is still with us • Note that #GGUS tickets for a typical service is several a week across different services, but it can easily reach several per day on a single FE • We still have ITCM alive, tickets get duplicated/lost in between Feedback on Snow - 18

18. S-NOW and ITCM integration Sysadmin FE manager SNOW • Not done at the beginning • Main tool for groups to handle large hardware • Collaboration tool between sysadmins and Service managers • Critical on delivery timely and efficient hardware repairs REMEDY ITCM3641 INC118240 Can I reboot node abc? Can I reboot node abc? Yes! Tell me when done Context loss OK! INC118960 Thanx, node back in prod Simple (but real) case. Complex cases force us to use email...

19. S-NOW, Request Fulfilment • Implementation of Request fulfillment: • Often just a complex implementation for a simple flow: many steps to achieve the same, at the end is the same person signing/agreeing the different steps • No way to close a request after a first dialogue with the user, before escalating to fulfilment • It is possible to reject the request, but this is not nice for the user • Not possible to convert incidents into requests without sending it back to first level (service desk) • A large majority of issues would be better dealt with INC… Feedback on Snow - 20

20. S-NOW: complex interface • UI not always efficient • High number of clicks/tear-down menus • Why choose “Resolved” and the “Restored”/”Not Restored”/”Spam”/”...”? • Maybe a “click through” exercise could be nice to enhance the usability • To act on a ticket one should belong to an egroup • And then you get all the mails • “4th level” experts feel it as a “spam” • Power users should have power commands • Bulk operations are needed • Why the system does not offer an API? • E.g. programmatically query #open tickets etc.. • It will greatly encourage light-weight integration Feedback on Snow - 21

21. S-NOW, the case logs • Case Logs are not understandable • This induce back and forward between second and third level to understand what is happening • Important loss of efficiency, waste of time Example of internal part of log of an incident Feedback on Snow - 22

22. S-NOW, the mails • Awaited improvements: • Service owner • Need for an escalation system • Need to be able to generate reports • Reduce the number of mails • For each case assigned to someone: • One mail from NO-REPLY-service-desk • One mail from service-desk

23. Conclusions • The process • IT Service Managers made a big effort to adapt to the ITIL process, and to adapt to the new tools • The evolution of the process should be discussed step by step with the Service Managers • Define ‘where things should be’: KBs, SLA, SLD, etc. • Avoid duplication!S-NOW is not the best place for everything • Implements a Service Catalogue maintenance lightweight process • Goals: updated service list and clear for users • The tool • IT services invested a lot in it • The tool is good, better than the previous one. • Many improvements are needed to improve efficiency: • Simplify the views and usual actions • Proper rotas implementation • SSO, KBs, text editor. •  The bottom line • A closer collaboration between IT Service Managers and ITIL team would improve the whole process. • Service Managers experience should be taken more into consideration, in a constructive manner instead of opening RFQs or running individual negotiations. Feedback on Snow - 24

24. Questions?