Enterprise Integration

1. Integrating software in the Enterprise As “applications architecture” becomes more complex with a trend away from monolithic ERP systems to implementing best-of-breed point solutions, the integration of applications plays an important role in EA implementation. How can you use integration to assist in application deployment and ensure the suite of applications used by the organisation is scalable, reliable and consistent? Enterprise Integration Presented by Cameron Hart

2. Flow Software Founder & MD Software Developer by trade so I like things practical Building integration solutions since 1998 Lived through System Integration, ETL, EAI, SOA, and now ESB. Small, Medium, and Enterprise 500+ customers

3. Contents Enterprise Architecture needs integration Integration Overview Fundamentals of an Integration solution Practical Examples Cloud and EDI Integration

4. Enterprise Architecture & Integration

5. Enterprise Architecture Key Goal - Aligning IT systems with the organisation’s business strategy Viewing IT strategy holistically encompassing the entire organisation (and trading partners)

6. Enterprise Architecture cont… Integration Solution EA has seen a trend away from monolithic application architecture Modular approach using best-of-breed solutions specific to individual departments need

7. Enterprise Architecture cont… Integration Solution Better Fit Quicker to deploy Quicker to adapt

8. Enterprise Architecture cont… Integration Solution EA doesn’t stop at the organisations boundary Integration with Trading Partners necessary EDI

9. EA & Integration Modular approach means you have a higher reliance on third party application vendors that have predetermined the models, business processes, and data operations of their software What’s left is the task of making them work together for the organisation – selection, configuration & integration

10. Integration Challenge • Multiple vendors and technologies • Point to Point solutions get too complex • Automation is actually what is needed • Business processes not data

11. Integration Spectrum Integration Automation Different solutions are needed to address different integration requirements Data Migration Irregular Data Loads Regular Data Feeds Data Synchronisation Application Integration EDI / Inter-Company Integration Business Workflows Data Driven Process Driven

12. Integration Overview

13. Rise of Integration • The modular approach created new challenges • Interoperability • Consistency • Point-to-Point • ETL • EAI • SOA • ESB • These are not products – they are ideas, approaches, methodologies, processes, or frameworks. • Each of these methodologies only cover a part of an overall integration solution

14. Fundamentals of Integration Lots of different names; Integration Software Broker Middleware ETL SOA Hub & Spoke ESB EAI Message Queues Dataset Definitions Format Transformation Data Mapping Event Handling Business Logic Transport Protocols Data Enrichment & Validation Data Persistence Routing Central Configuration Monitoring & Auditing Security

15. Dataset Definitions Examples of Metadata SQL DDL XSD Documentation • Define structure of data • Different app, Different structure • ie Product List, Customer List, Price List • Master Data • iePurchase Order, Creditor Invoice,Shipment Notice • Source documents / Transactions

16. Format Transformation Transformation • Data Format is the physical representation of a dataset • ie Purchase Order can be represented as CSV or XML • Different applications support different formats • Integration solution translates between formats

17. Data Mapping Common Mappings Account identifiers Product identifiers Units of Measure Tax representations • Manipulation of single data set to meet application requirements • ie Purchase Order in SAP different to purchase order in MS CRM • Often occurs as part of format transformation • Two applications using exactly the same format can need data mapping

18. Event Handling • Exchanging data like a Purchase Order is a series of events • Creation • Modification • Line Addition • Line Modification • Line Deletion • Often multiple events occur at once

19. Event Handling in Detail Example – products shipped WMS - picking completed INVENTORY - product decreased WEBSITE - lines updated as shipped FINANCIAL - Invoice created JOB - Service Reminder created • Purchase Order • Modification • Change of state • Change of delivery details • Line Modification • Shipment of goods • Invoice of line • Change of product • Change of price

20. Business Logic Business rules determine how data is shared and integrated Less rules in the end application enable quick adaption Different rules needed for each event

21. Transport Protocols Transports Support multiple protocols Protocol conversion

22. Data Enrichment and Validation • Incoming data may need enrichment before sending to the final destination • Data needs to be validated • Receiving apps expect valid data

23. Data Persistence Temporary persistence to support integration processes Necessary for state control and Long Running Processes Benefits from Long Term persistence

24. Routing End applications should not know the receiver Routing Branching Consolidation

25. Central Configuration Easy Management Independent configuration of publishers and consumers Prevents lost integration processes

26. Monitoring & Auditing • Exception Reporting • Application Errors VS Data Errors • Data Errors handled by business users • Business Activity Monitoring (BAM)

27. Security • Support major security standards • SSL • Signatures • Authentication

28. Review Key Features of Integration • Routing • Central Configuration • Monitoring • Security • Data Modelling • Integration is not just a tool or a service • Integration should be considered an application like any other application in your Enterprise Architecture • Often used to fill the gaps other applications are not providing – this leads to failure if your integration solution is just a tool Data Transformation Event Handling Transport Protocols Data Enrichment Data Persistence

29. Integration Benefits you should look for Reduce processing time Increase productivity Reduce keying errors Reduce mundane tasks – staff retention and morale Ensure processes are followed consistently

30. Integration Benefits cont… Reduce points of failure Quick adaption to changing business requirements Little or no modification of end applications Simplified management Business Activity Monitoring / Exception Reporting Future Proof

31. Practical Examples

32. Practical Examples • Automated Data Migration • User Acceptance Testing (UAT) • Acquisition strategy • Single Sign On (SSO) • Procurement

33. Automated Data Migration Often performed periodically involving many steps and manual manipulation of data Prone to errors and is a big task on Go Live Issues in historic data often appear long after Go Live Automated Data Migration is regular and self repairable. Continues after Go Live. Con: Reliant on application functionality

34. User Acceptance Testing (UAT) Automated Data Migration allows for better UAT UAT can fail because of bad user data Application errors during UAT can damage user data Recent, relevant data improves UAT Users can spot errors easier

35. Acquisition Strategy If the organisations strategy involves acquisition Integration solutions provide another option to migrating IT systems of acquisition's Important where acquisitions have unique IT requirements supported by current systems Similar for Franchises and Buying Groups

36. Single Sign On (SSO) Active Directory (ADFS) oAuth SAML OpenId

37. Procurement Integration Solution SYSTEMS INVOLVED IN PROCUREMENT PROCESS PROCUREMENT MIDDLEWARE FINANCIAL VAN Auto Re-order Formats EDIFACT Purchase Order XML Requisition Mapping Goods Receipt PDF FAX Goods Costing Punch out Transports Data Validation Invoice EMAIL Transports FTP Credits 3 Way Matching AS2 HTTP Reconciliation Business Rules Structural Validation Remittance STAFF SUPPLIERS

38. Hot Integration Topics

39. Integration in the Cloud You can integrate to the Cloud whether you have in house integration solution or a hosted integration solution Integrating Cloud applications requires those applications to have Internet-ready APIs (Web Services, REST) Integrating anything over the Internet requires Internet-ready APIs Integration over the Internet is SLOW – carefully plan around volumes (of events) Reduce load by performing data comparison early – only send deltas

40. EDI • Integration doesn’t stop at the corporate boundary – it just changes it name to EDI • EDI uses the same technologies as Integration • More complex as you only have control of one side of the Integration • You can only achieve the lowest common denominator • Events, Data, Protocols, Monitoring, Business Processes

41. Summary Integration is important for a modular EA strategy Integration is an application – treat it as such Integration products are not an integration solution by themselves

42. Contacting Me Cameron Hart, Flow Software Ltd 0800 69 3569 cameron.hart@flowsoftware.co.nz