IMPLEMENTING A SERVICE BUS ARCHITECTURE WITH BIZTALK 2009 AND THE ESB TOOLKIT 2.0

1. IMPLEMENTING A SERVICE BUS ARCHITECTURE WITH BIZTALK 2009 AND THE ESB TOOLKIT 2.0 A Case Study

2. Implementing a service bus architecture with biztalk 2009 and the esb toolkit 2.0 Presenter: Ed Jones, RBA Consulting ed.jones@rbaconsulting.com CHS Team: Management: Darin Hawley Anoop Abraham Tom Green Technical: Jeffrey Dreke Steve Krueger

3. Agenda • Business Scenario • Solution Overview • Creating the Itineraries • Implementing Exception Management • Demo • Questions?

4. Business scenario CHS required a system that would accept incoming shipment data in the form of flat-files in multiple formats, process that data through a series of resolutions, and then output the data in both its raw and processed forms into an ERP system.  While most data will be processed, some will be ignored.  Over time it is expected that the various processes may change in size, scope, and sequential order. 

5. Solution overview Map to Canonical Batch Schema Custom Pipelines & Pipeline Components • MAPPING Flat Files Validate/Debatch • ------------------------------------------------------ • ------------------------------------------------------ Debatched Shipment Messages • Orchestration • MAPPING • ------------------------------------------------------------------------------------------------------ • ------------------------------------------------------------------------------------------------------ • Custom Pipeline • ------------------------------------------------------------------------------------------------------ • ------------------------------------------------------ • ------------------------------------------------------ • MAPPING • Custom Pipeline • ------------------------------------------------------ • ------------------------------------------------------ Service Providers On Ramp Off Ramp Itinerary BRI Resolver Custom Pipeline

6. Translate Product Translate Customer Translate Distributor • Orchestration • Orchestration • Orchestration Process Shipments • ------------------------------------------------------------------------------ Un-translated Data Type A ERP System • ------------------------------------------------------------------------------------------------------ Ignore Shipments Type B File System

7. Phase One: Incoming data • Data is received in the form of FTP/File Drops by business partners in multiple flat-file formats. • Challenges: some of the incoming files were in formats that were not easily translated using the standard BizTalk flat-file tools. • Used Custom Pipeline components to build an Xml Disassembler to manage the problematic formats. • Data was then mapped to a canonical “Batch” schema

8. Phase two: Debatching • Once received in the canonical format, the batch message is passed through an orchestration which performs two functions: • Data is validated to ensure completeness. • Message is “de-batched” into individual shipment messages • Shipment messages are dropped to the message box, after which they are assigned an itinerary and processed individually.

9. Phase three: assigning an itinerary • Messages are assigned their appropriate itinerary based on shipment type and message type. • Type A will be processed and sent to the ERP system • Type B will not be processed and is written directly to a file location. • Itinerary is determined by using the BRI resolver

10. Phase four: Executing the itinerary Each shipment message is processed through a number of “Resolutions” which in the end will allow the data to be placed into an ERP system The resolutions use a combination of database lookups and Business Rules Engine calls to determine validity and translate the data into a usable ERP format. Exception Management is used throughout all phases. Users are given, via the ESB portal, the ability to repair and resubmit failed messages. Repaired messages are re-run through their itineraries.

11. Creating the itinerary

12. Esb.config

13. Assign the itinerary The easiest way is to use the ItinerarySelectReceive Pipelines that come with the ESB Toolkit.

14. “Wire up” the itinerary

15. Capture and Advance the Itinerary

16. Promote the necessary properties by creating correlation sets and assigning them to the final send shape.

17. Exception management We implemented exception management throughout all processes. We captured “Rules” exceptions as well as “System” exceptions. All exceptions were routed to the ESB Exception Management database via the “out-of-the-box” messaging solutions.

18. Creating fault messages Create a multi-part message type

19. Create an instance of the fault message, set its properties, and send through a Direct-Bound Port

20. ESB.Portal • Exception Management Portal allowed us to view exceptions and repair and resubmit messages. • Challenges: • There is a bug in which all messages that do not have an xml declaration are classified as plain text. • ESB.Portal has limited resubmit capabilities out of the box—you will probably want to make code changes to the ESB.Portal website.

21. demo

22. Thank you! Contact info Email: ed.jones@rbaconsulting.com Web: http://www.rbaconsulting.com Blog: http://extremelytalentedmonkeys.blogspot.com

23. Go wild!