Eli Lilly: Leveraging SAP XI in Your Landscape: Costs, Benefits, Lessons Learned

1. Eli Lilly: Leveraging SAP XI in Your Landscape: Costs, Benefits, Lessons Learned Timothy S Yates tyates@dataxstream.com Scott A Sanneman sanneman.s@lilly.com

2. Learning Points • This presentation will give you a solid starting point for determining / cost justifying an SAP XI Implementation. • This presentation will give you specific considerations that need to addressed up front in order to control scope creep and cost. • This presentation will give you specific technical lessons learned that can potentially impact implementation timeline, delivery quality and overall implementation cost.

3. Background On Eli Lilly XI Implementation • General Company Information • SAP Project History • SAP Integration Infrastructure History • SAP XI Project

4. Eli Lilly General Company Information • Pharmaceutical company founded on May 10, 1876 • > 41,000 employees worldwide • ~20,000 OUS, 8000 R/D • Research and development facilities located in 9 countries • Manufacturing plants located in 13 countries • Products marketed in 143 countries • Headquarters: Indianapolis, Indiana, U.S.A • > 15 Billion in net sales in 2006

5. Eli Lilly SAP Project History The Eli Lilly philosophy was to “implement common business processes worldwide enabled by an integrated information environment” • Introduced the Global Business Integration Project (GBIP) to the organization in 1998 • Intent on standardizing common data and systems worldwide • Scope included HR, Finance, SCM & PMP. Implemented 1st 2 countries (France & Spain) in 1999 • Strict release timeline of 6 month cycles for new countries • 16 countries implemented SAP, all currently on SAP R/3 4.6c • SAP payroll for the USA introduced in 2006 • All other (Non-SAP) countries following many SAP processes through integration (i.e. HR data for all Non-SAP countries integrated into SAP via global interfaces) • XI successfully implemented, intended to eventually replace existing data movement infrastructure • Lilly is currently undertaking an upgrade to ECC6.0.

6. Eli Lilly SAP Infrastructure History • In The Late 90’s Identified The Need For An Integration Infrastructure To Support The Integration Needs Of The Project. • Tools Like Mercator and MQSeries Where Selected Because They Represented The Best Of Class Tools For Enterprise Architecture Integration At The Time. • The Last 6 – 7 Years Have Been Spent Developing A Set Of Lilly Specific Tools Based On These Technologies.

7. Eli Lilly Existing Infrastructure Challenges • Infrastructure Built Using Different Tools • Requires diverse skill set, Upgrade complexities, Change control • Tool Stability, Agility, Reproducibility • Using several tools requires many resource to diagnose issues • Future Of Current Tool Set Not Clear • Mercator Was Purchased Last Year By Accential Software. Accential Software Was Purchased By IBM This Year. Implications Unknown In Short Term. In The Long Term Should Be Good But Changes To Tools May/Should Eventually Drive Significant Upgrades. (Past Changes Have Been Implemented Without Backward Compatibility)

8. Eli Lilly SAP Infrastructure Eli Lilly has developed a custom designed routing system as part of existing infrastructure, which includes: • Transformation and Routing • Mercator (IBM DataStage TX), Gentran, Informatica, C Programs • Messaging • IBM MQ Series • Delivery • C Programs (Wrapper / Unwrapper) • Database • Oracle • Source Control • Rational Clear Case • Configuration Web Interface • Java • Maintenance, Trouble Shooting, Migration and Consistency Checking • UNIX - Scripting and Security, Manual FTP

9. Eli Lilly Existing Infrastructure Diagram

10. XI Concept at Eli Lilly Scope • Implement functionality to determine capabilities and positioning in GBIP architecture • Pilot: Design and implement 40 interfaces • Technical Criteria – Re-implement minimal interfaces to cover most scenarios (File, IDOC and MQ to JMS adapters) • Build entire XI infrastructure, including configuration, development, testing (QA) & production environments • Pilot implementation would be seamless to existing business users when compared to existing infrastructure!

11. XI Concept at Eli Lilly • Reduce Number Of Tools To Support Interfaces • Built On Robust SAP Technology With SAP Support • Will Reduce Long Term Support • Eliminates Need To Fund Additional Data Sharing Infrastructure Improvement Projects. • Technology Is Mature And SAP Community Is Moving In This Direction • Will Be Required To Leverage New SAP Integration Development

12. Starting Point For Determining / Cost Justifying SAP XI Implementation • Building The Foundation • Project Types • Project Approaches • Understanding Project Cost / Savings • Project Cost Considerations • Project Cost Savings

13. Project Types • New EAI / B2B Implementation • Adding SAP XI To Existing EAI / B2B Implementation • Legacy EAI / B2B Replacement Implementation • Legacy EAI / B2B Consolidation Implementation

14. New EAI / B2B Implementation • Current Infrastructure • No Existing EAI / B2B Tools In Use Today • Business Drivers • EAI / B2B Functionality Required • Leverage Existing SAP Skill Sets (ABAP / JAVA) • Alternative Options • Build Point To Point Interfaces With Available Programming Tools • Implement Solution From Different Vendor • Challenges • Building Out SAP XI Support Infrastructure • Internal EAI / B2B Experience • Finding Good External EAI / B2B Experience

15. Adding SAP XI To Existing EAI / B2B Implementation • Current Infrastructure • Currently Have Other EAI / B2B Tools In Use Today You Intend To Keep • Business Drivers • New SAP XI Functionality Required • Leverage Existing SAP Skill Sets (ABAP / JAVA) • Alternative Options • Implement Solution From Different Vendor • Continue / Expand Use Of Current Tools • Challenges • Building Out SAP XI Support Infrastructure • Finding Good External EAI / B2B Experience • Supporting Additional Tool SAP XI

16. Legacy EAI / B2B Replacement Implementation • Current Infrastructure • Currently Have Another EAI / B2B Tool In Use Today You Intend To Replace • Business Drivers • New SAP XI Functionality Required • Leverage Existing SAP Skill Sets (ABAP / JAVA) • Consolidation Of Existing Designs • Current Infrastructure • Complexity • Deficiencies • Substantial Upgrades Required • Current Vendor Support / Direction Concerns • Alternative Options • Implement Solution From Different Vendor • Continue / Expand Use Of Current Tools • Challenges • Building Out SAP XI Support Infrastructure • Finding Good External EAI / B2B Experience • Supporting Additional Tool SAP XI (In Phased Replacement Approach) • Adapting Source and Target Applications To Leverage New Tool Set

17. Legacy EAI / B2B Consolidation Implementation • Current Infrastructure • Currently Have Multiple EAI / B2B Tools In Use Today You Intend To Replace/Consolidate Into XI • Business Drivers • New SAP XI Functionality Required • Current Infrastructure • Complexity • Deficiencies • Substantial Upgrades Required • Current Vendor Support / Direction Concerns • Consolidation Of: • Existing Designs, Tools, Skill Sets, Teams and Resources • Alternative Options • Continue / Expand / Consolidate Use Of Current Tools • Challenges • Building Out SAP XI Support Infrastructure • Finding Good External EAI / B2B Experience • Supporting Additional Tool SAP XI (In Phased Replacement Approach) • Adapting Source and Target Applications To Leverage New Tool Set • Managing Functional / Technical Trade Offs To Maximize Overall Project Benefit • Managing Complexity and Timing Of Merging Multiple Tools

18. Project Approach Types • Proof Of Concept • Implement A Representative Set Of SAP XI Functionality Required To Meet Overall Architectural Goals. • Big Bang • Implement All Required SAP XI Functionality In One Software Release. • Only Really Practical For New and Small Replacement Implementations. • Phased • Implement Required SAP XI Functionality Over A Series Of Software Releases • Only Practical Approach For Large Replacement and Consolidation Projects

19. Project Cost Considerations • SAP XI Platform Software • Third Party Adapter Software • Hardware • Consulting Services • Project Approach

20. SAP XI Platform Software Costs • There Are Three Primary Components That Make Up The Cost Of The SAP XI Platform • SAP XI Software • Database Software • Third Party Infrastructure Tools • Alert Software • Job Scheduling • Archiving • Etc • Interesting Point About Our Project. The Software Had Already Been Purchased In A Bundle Of Functionality Years Earlier • SAP Had Since Moved From An Instance Based License To A Volume Based License. • The Volume Based License Presented Us With Several Challenges • How To Measure Volume? • What Volume Level Would We Need Licensed If We Fully Implemented XI?

21. Third Party Adapter Requirement Costs • If you are implementing EDI you will need a third party adapter. • If you require complex content conversion you will need a third party adapter. • Understand and identify the initial cost and maintenance cost associated with them when developing your project costs.

22. Hardware Costs • Hardware Components • Servers • Storage • Network • Drivers Of Hardware Costs • Landscape Design • Number Of Environments Support SAP Promote and Support Structure • Production Volume Requirements • Testing Requirements • Infrastructure Techniques • Volume / Stress Testing • Tuning

23. Consulting Services Cost • Cost Of Third Party Assistance • Consulting Services Cost Vary Drastically By Provider And So Can The Quality Of The Services Provided • Drivers Of Consulting Services Costs • Project Scope • What Constitutes The Boundaries Of Your Scope Of Project? • Project Timeline • How Long Will The Project Take? • Balanced Pace: Business Capacity, Technical Capacity, Testing Capacity

24. Project Approach Impact On Cost • Each Of The Approaches Has A Cost Associated With Them. Typically Each Phase Is Aimed At Reducing Overall Project Risk • Initial Project Assessment • Many Consulting Firms Will Do This For Free. Get More Than One And Compare Findings • A Workable Assessment Will Still Typically Require A Paid In Depth Assessment • Prototype Environment • Get Your Feet Wet Learn The Tool Identify Early Areas Of Concern And Monitor As Project Progresses • Identify Project Specific Cost Drivers That Will Impact Your Specific Project • Proof Of Concepts Phase • Test The Functionality Required To Completely Implement Your Vision • Find The Hidden Challenges / Costs Early • Overall Implementation • Internal VS External Resources • Timeline / Pace • Other External Drivers That Impact Overall Implementation Cost • R/3 Upgrades, Existing Infrastructure Upgrades Required, Unplanned Business Events

25. Project Cost Savings • Cost Savings • Maintenance Cost Of Existing Software • Reduction Of Redundant Hardware Support Costs • Cost Of Skill Sets To Support Existing Functionality • Maintenance and Support Cost Of Custom Designed Infrastructure Functionality • Cost Avoidance • Cost Of Required Infrastructure Software Upgrades • Cost Of Required or Need Infrastructure Enhancements

26. Maintenance Cost Of Existing Software • This Cost Savings Will Vary Depending On Your Type and Scope Of Project • Each Infrastructure Tool Has A Software Support Cost Associated With It. Therefore Each Tool You Retire Represents A Potential Maintenance Cost Savings. • Consideration: Some Maintenance Contracts On Infrastructure Tools Have Different Levels Of Support, The Higher Levels Can Have Significant Yearly Costs Associated With Them. • In Large Projects It Will Take Time To Phase Out The Use Of Certain Tools. So Make Sure Your ROI Calculations Consider This.

27. Reduction Of Redundant Hardware Support Costs • This Cost Savings Is Primarily Driven By A Reduction In Hardware Required To Support Multiple Infrastructure Tools • There Is A Monthly Support Cost Associated With Any Server Hosted Within A Data Center • On Tool Consolidations Projects, A Significant Number Of Physical Servers Can Be Eliminated And Their Associated Support Costs • Hardware Upgrades To Aging Tools Servers Can Also Be Avoided

28. Cost Of Skill Sets To Support Existing Functionality • Each Infrastructure Existing Infrastructure Tool Requires A Team Of Different Software Specific Skills To Develop, Administer, and Monitor. • Consolidation Of Tools Can Significantly Reduce The Number Of Resource Skills Required To Support An Infrastructure • Reduce The Number Of Resource Required Overall To Support The Infrastructure • This Particular Area Obviously Will Be Subject To Political Debate Because Of Its Potential Impact On Head Count. • We Found In Our Project That Our DataStage TX Resources Easily Adapted Their Mapping Skills To The XI Skills Required, And Became Productive Quickly.

29. Maintenance and Support Cost Of Custom Designed Infrastructure Functionality • Many Existing Integration Infrastructures Are Developed From Tools Introduced To The Market Over The Last Ten Years. • As a Result Many Existing Integration Infrastructures Rely on Custom Developed Components To Meet Required Infrastructure Functionality. • This Custom Functionality Can Be Quite Complex And Require Expert Level Skills In Multiple Infrastructure Software Products To Effectively Maintain. • These Infrastructure Components Are Unique Thus When An Issue Occurs Either You Will Require Knowledgeable In-house Resources Or You Will Need Expert Level External Help. • The Cost Of Supporting This Existing Custom Functionality Is Usually Significant.

30. Cost Avoidance Of Required Infrastructure Software Upgrades • Many Existing Infrastructure Tools Require Upgrades To Keep The Product In Support • If Your Infrastructure Is Made Up Of Many Different Tools. Chances Are One Of Them Is Going To Be Driving An Infrastructure Upgrade • Some Tool Upgrades Provide Very Little Challenge From An Effort And Staffing Perspective, Some Are Significant. • Understand The Impact Of And Drivers Of Future Tool Upgrades, They Can Be A Driver Of Cost Savings From An Avoidance Perspective. • When Upgrading A Custom Developed Infrastructure Made From Different Software Tools, Projecting The Impact / Required Scope / Cost Of An Upgrade Can Be A Challenging Undertaking.

31. Cost Avoidance Of Required or Need Infrastructure Enhancements • If You Have Developed Your Integration Infrastructure From Off The Shelf Tools and Custom Designed Components. You Will Find Yourself Continually Adapting That Infrastructures Capabilities To Keep Up With Changing Business Needs and Changing Technology Needs. • As You Continue To Plan Ahead For Future Business Requirements, You Will Find The Need To Scope Design Changes And Improvement Projects. • Most Of Our Future Scoped Functionality Existed In SAP XI Out Of The Box • In Addition Because It Is An SAP Industry Supplied Tool As Future Functionality Is Required, It Will Most Likely Be Added As The Tool Matures.

32. Project Controls and Project Scope Definition • Scope In Infrastructure Tools Early • Understand Your Interface Volumes • Understand Your Adapter Requirements • Don’t Underestimated Content Conversion • Assessing The Impact Of Environment Copy Backs • Try Not To Get Sucked Into The Number Of SLD’s Required Vortex

33. Scope In Infrastructure Tools Early • Leave some room in your plan for development of simple infrastructure tools. • These tools will help you implement better more maintainable designs as you project progresses. • Some will be required from a technical perspective. Some will significantly simplify other activities later. • Don’t cut scope on this in early phases of your project.

34. Understand Your Interface Volumes • Specifically what impact does XML have on your volume estimates. • Don’t get to integration testing and figure out that your design brings XI to its knees! • Study your interface volumes. Specifically at production and initial conversion loading conditions. • Make sure your design accounts for these volumes and can handle the load effectively. • Specifically do the math on content conversion for each interface. • Flat file to XML conversion. • Calculate a conversion factor. • Multiply production volume estimates by the conversion fact. • The troublesome interfaces will become apparent very quickly. • Reworking an interface design can be a costly mistake and can significantly impact project timelines.

35. Understand Your Adapter Requirements • What adapters do you need to accomplish your goals? • Understand that your are not going to implement EDI in XI without the help of a third party adapter. • Understand which adapters you will require and their delivered functionality and functional gaps. • Some adapters support multiple capabilities such as FTP and SFTP by changing configuration settings. Don’t buy custom adapters you already have. • Some standard adapters will require custom adjustments via custom Java modules. Understand and scope these custom requirements early in your project.

36. Don’t Under Estimated Content Conversion • Content conversion in SAP XI can prove to be one of the most difficult tasks to overcome in an implementation. • Content conversion is somewhat cryptic to configure and effort balloons as content structure complexity increases. • We underestimated this effort the most in our project implementation. • The more complex content conversion requirements we had forced us to eventually rework a number of our interface and implement SAP Conversion Agent by Itemfield.

37. Assessing The Impact Of Environment Copy Backs • In the initial phases of building out a landscape this is not an important consideration. • Once a landscape is built however you will most likely not want to copy back XI. This is counter to most other SAP applications best practices. • Each SAP XI instance has unique ID configuration for the environment. Copy backs will erase this unique configuration. • You will not need to copy back your SAP XI landscape so don’t spend a lot of time planning it or debating it. It will only cause you issues.

38. Try Not To Get Sucked Into The Number Of SLD’s Required Vortex • There is no right answer here. You will find documentation that states there should only be one. You will find documentation that says there could be more than one. • When a consultant tells you it depends on your project he is right. • Our experience is that less is easier to manage. • Set a direction and move on.

39. Technical Lessons Learned • Packet Processing Not Supported By IDOC Adapter • User Defined Java Functions • Custom Cross Reference Functionality • Required Setup To Use ABAP Maps • Message Size After Content Conversion • Large Message Processing • Message Pointers • ABAP Mapping • JMS Adapter • Conversion Agent By Itemfield • Adapter Tracing

40. Packet Processing Not Supported By IDOC Adapter • On the surface this does not appear to be an issue or a concern. • Each outbound packet from SAP is split into individual messages in XI. • However this has a negative effects on downstream applications like Gentran. When outbound EDI data is processed by Gentran it is typically grouped by trading partner. This caused Gentran to generate significantly more EDI envelopes than normal. • It also impacted other downstream applications by making them process lots of individual messages instead of groups of messages. This created performance issues for certain applications. • Only way to process IDOCs as a group was to process using a collection BPM.

41. User Defined Java Functions • Writing Custom Code Inside A GUI Map Is Sometimes A Good Thing • We used user defined JAVA functions in the following way. • We identified and created user defined functions that replicated all existing DataStage TX mapping functions not supported by XI. • When we identified that a particular mapping requirement was creating specific difficult challenges for GUI mapping techniques we would build the required functionality in a user defined JAVA funciton. • One technical challenge to user defined JAVA function was that they are map specific. If you want to reuse a function you must copy it from another map. • In general our functions required 20 lines of code or less but significantly simplified our map conversions and mapping rules.

42. Custom Cross Reference Functionality • SAP provides GUI mapping rules that allow you convert mapping data via a cross reference. The issue is the cross reference data is map specific and can not be reused. • We had many instances where the same cross reference data was required across multiple maps. • We create with minimal development effort a custom cross reference solution that utilized a custom ABAP table, a custom function module and a custom user defined JAVA function. • The user defined JAVA function had to be copied from map to map but the logic and cross reference data was stored in the function module and ABAP table. Thus allowing us to reference the same cross reference data across multiple maps.

43. Required Setup To Use ABAP Maps • The exchange profile needs to be modified before mapping can be done on the ABAP stack. • Modify the exchange profile setting “com.sap.aii.repository.mapping.additionaltypes”. • We added the entry “R3_ABAP|ABAP-Class Map;R3_XSLT|ABAP-XSL”. • This allows both ABAP class maps and XSLT maps via the ABAP kernel.

44. Message Size After Content Conversion • Make sure you consider the size of you message after it is converted to XML when designing your interfaces. • A inbound 5mb flat file can grow to a 50mb xml document. • An interface that appears to be well within processing limits quickly becomes too big.

45. Large Message Processing • Why Does XI Not Handle Large Messages Well? • Netweaver’s dual stack design (ABAP & Java) results in a bottleneck whenever large data sets move from processing on one stack to another. The standard way for the two stacks to communicate is via RFC.

46. The following diagram shows the communication between the ABAP and Java stacks during normal message processing. Large Message Processing

47. Large Message Processing • Message Pointers • Lilly’s existing interfaces were enormous by XI standards (>500MB). • These large files usually contained full refreshes of the data due to technical constraints on the receiving, non-SAP system. • XI’s architecture is not structured to handle large amounts of data. • The key to processing large messages with XI is to minimize stack boundary traversal. • We were able to do this by saving the message as a file and only passing a pointer to the data (ie filename). It was the job of the processing agent to delete the file after it was confirmed that it was sent to the receiver.

48. Large Message Processing • ABAP Mapping • Some messages were brought in via the J2EE Adapter Engine, but not converted to XML. This unconventional design decision was made because the conversion to XML required added processing and memory requirements that caused message processing to take too long. We decided to pass the data directly to the map unconverted. • In an effort to further streamline message processing and in addition to the removal of XML conversion overhead, we also decided to do away with some of the stack traversal overhead. We accomplished this by implementing our mapping requirements with ABAP.

49. Large Message Processing • ABAP Mapping continued • Doing the mapping in ABAP was doubly advantageous because message mapping and XSL mapping requires that the data be in XML format. Since our data was still in flat file format, our only options were Java mapping and ABAP mapping. With the added requirement of reduced stack traversal, our only remaining option was ABAP. • Our data was in a structured text format. ABAP turned out to be a very good language for this type of data mapping. Our large team of competent ABAP developers made short work of the ABAP maps.

50. JMS Adapter The JMS configuration is not out-of-the-box! JMS drivers must first be installed……….including: • The correct JAR files in order for the JMS adapter to function correctly, which is one of the challenges we faced with the use of JMS. • Performance tuning for XI to operate effectively that impacted JMS. SAP provides a document that outlines performance tuning requirements in XI version 3.0 (see appendix for information).