Challenges in process driven Service Oriented Architecture

1. Challenges in process driven Service Oriented Architecture Presentation at the IT University 11th of February 2008 Jack Ekman, Netcompany

2. Agenda • Introduction to SOA • SOA: Service infrastructure at a major pension company • Project overview • Paradigm shift in SOA - From application to process driven development • SOA Fundamentals • Quick introduction to Business Process Management (BPM) • Challenges and solutions in a SOA implementation • When is SOA a success • Which business advantages can we achieve by using SOA? • Discussion

3. Paradigm shift in SOA: From Application Orientation to Business Process Orientation • SOA is ultimately about business processes over applications • SOA initiatives should be driven by business needs more than technical needs • SOA must be anchored both in the technical and the business organization.

4. What is a Business Process? ”A business process or business method is a collection of interrelated tasks, which solve a particular issue. “ [Wikipedia] Examples: • Create new customer • Create Order

5. What is the goal of SOA business wise? • Increase business Agility • Enable Business Activity Monitoring (BAM) • Automation and streamlining of business processes • Leveraging of existing application assets • Reuse Business assets as reusable services

6. What is the goal of SOA architecture wise? • Loosely coupling of systems • Standards based interface • Simplified communication model for applications

7. Elements in a Service Oriented Architecture Business Benefits Business Agility SI as a Business Integration Platform Business Activity Monitoring Automation and streamlining of business processes SI today Leveraging of existing application assets Business assets as reusable services Point to Point integration Integration benefits Loosely coupling of systems Standards based interface Simplified communication model for applications Basic SOA foundation SOA foundation Adapters BPM engine File Transfer Service Registry Security Messaging External Services Transaction Management Monitoring Routing Quality of Service Transfor-mations Audit Log Configuration Error Handling Tracking

8. Agenda • Introduction to SOA • SOA: Service infrastructure at a major pension company • Project overview • Paradigm shift in SOA - From application to process driven development • SOA Fundamentals • Quick introduction to Business Process Management (BPM) • Challenges and solutions in a SOA implementation • When is SOA a success • Which business advantages can we achieve by using SOA? • Discussion

9. The Story goes… The Story behind the project • Started in 2003 • Motivated by • The need to change and replace existing systems • The need to be able to support new business processes • The need to optimize existing processes • The need to clearify existing and upcoming architecture

10. Project overview • The customer plus 5 consultant companies (Netcompany, IBM, CSC, PA Consulting, 7N) • Ca. 250 people, 150 of them consultants. • 14 parallel development projects • 21 applications (3 of them are completely new) • The business driver is the development of a new pension system

11. System architecture 1/2 • SOA solution based on Custom built Enterprise Service Bus • Technical platform is Java / J2EE / BEA Weblogic • Heavy use of Business Process Management (BPM) • Common domain model / data model described in XSD architecture • One of the biggest implemented solutions in Europe

12. System architecture 2/2 • Technologies: • J2EE • Dependency injection: Spring-framework • Persistent: Hibernate • Unit test: JUnit • Build/Configuration management: ParaBuild & Maven 2 • Version control of code and configuration: CVS & ClearCase

13. The development process 1/2 The Business Process team Service Infrastructure . . . . Applications teams Test

14. The development process 2/2 • Process driven development in iterations • Business processes are defined and analyzed by a process team • Processes are used as a starting point for • Services • Business Process Management (BPM) processes • Test • Coordination of logic between applications

15. What is a business process in PFA Terms? The roles or subject Verbs and objects The result of the process The activities Problem: where to describe the activities or needed information to carry out the activity? The start event Problem: where to describe the starting conditions? Timeline Project model definition: Time always moves from left to right

16. Who are using the business processes? Testing Test scenarios Use cases Start and end conditions Roles Manual Workflow Test Processes Organization change Data Business impact Roles -> subjects Service Infrastructure (SI)

17. Consequences of using a process driven approach • The processes are used to identify: • Services • Requirements for business data in the services • Dataflow between applications (BPM logic) • Furthermore, processes are used to: • Plan tests • Plan organizational changes • For formal documentation of business processes

18. Consequences of using a process driven approach The first important message SOA is not better than the quality of the business processes

19. Agenda • Introduction to SOA • SOA: Service infrastructure at a major pension company • Project overview • Paradigm shift in SOA - From application to process driven development • SOA Fundamentals • Quick introduction to Business Process Management (BPM) • Challenges and solutions in a SOA implementation • When is SOA a success • Which business advantages can we achieve by using SOA? • Discussion

20. Agenda • Introduction to SOA • SOA: Service infrastructure at a major pension company • Project overview • Paradigm shift in SOA - From application to process driven development • SOA Fundamentals • Quick introduction to Business Process Management (BPM) • Challenges and solutions in a SOA implementation • When is SOA a success • Which business advantages can we achieve by using SOA? • Discussion

21. Elements in a Service Oriented Architecture Business Benefits Business Agility SI as a Business Integration Platform Business Activity Monitoring Automation and streamlining of business processes SI today Leveraging of existing application assets Business assets as reusable services Point to Point integration Integration benefits Loosely coupling of systems Standards based interface Simplified communication model for applications Basic SOA foundation SOA foundation Adapters BPM engine File Transfer Service Registry Security Messaging External Services Transaction Management Monitoring Routing Quality of Service Transfor-mations Audit Log Configuration Error Handling Tracking

22. So why do we need SOA anyway? • Integration without a Enterprise Service Bus • Integration with Enterprise Service Bus • Moving towards a SOA

23. Integration without ESB (1/5) Direct Integration B A • The applications must agree on: • Protocol • Message format • How to find each other

24. A C B Integration withoutESB (2/5) Direct Integration

25. A D C B Integration withoutESB (3/5) Direct Integration

26. A D C B Integration withoutESB (4/5) Direct Integration UPIC JMS HTTP FTP SOAP FTP

27. A D C B Integration withoutESB (4/5) Direct Integration Spaghetti! UPIC JMS HTTP FTP SOAP FTP

28. D C B E Integration with ESB SOA using an Enterprise Service Bus (ESB) • Applications use one way (the ESB) to communicate • Easy to add new applications • Complexity is in the ESB • Flexibility via adapters • Loose coupling between systems • Integration based on standards A SOAP JMS ESB FTP RMI F

29. Portal OS OS A real life ESB Mægler BPM Papyrus IS CRM FICO RMI SOAP SOAP Fil JMS HTTP Service Infrastruktur HTTP UPIC RMI FTP SOAP Legacy UnitLink UnitLink Mainframe Mainframe F & P F & P SKAT SKAT SHAFT SHAFT

30. So what is a service [Conceptual level]? Copied from the real world ie. A ”Fix my teeth” Dentist service A ”Change oil on my car” Mechanics service A ”Create a Bank Account” Bank ServiceOne formal and quite technical definition"a mechanism to enable access to one or more capabilities, where the access is provided using a prescribed interface and is exercised consistent with constraints and policies as specified by the service description." [OASIS]

31. So what is a service [Technical level]? Service Infrastruktur Format a Format b App A Transformation Format A => Format B App B Adapter Adapter

32. Agenda • Introduction to SOA • SOA: Service infrastructure at a major pension company • Project overview • Paradigm shift in SOA - From application to process driven development • SOA Fundamentals • Quick introduction to Business Process Management (BPM) • Challenges and solutions in a SOA implementation • When is SOA a success • Which business advantages can we achieve by using SOA? • Discussion

33. Example: Get Contract List implemented without BPM Portal New system X System 1 System 2 System 3

34. Portal BPM AIA Mainframe UL Example: Get Contract List implemented in BPM Advantages: • Other systems can easily reuse the functionality • Less maintenance for the Portal, for instance when introducing a new insurance system • Easy reuse of process logic

35. BPM can be used on several levels in a SOA • Technical processes • Transforming a file to a sql call in a database • Orchestration of sub processes across systems • Managing very long process flows • End-to-end business processes • Create new customer in several systems • Manage contract changes

36. The practical solution: Make the code graphical! • Her, et screendump fra WLI

37. Portal OS BPM is a natural part of a SOA Mægler BPM Papyrus IS Service Infrastruktur BPM UnitLink KR F & P SKAT

38. Agenda • Introduction to SOA • SOA: Service infrastructure at a major pension company • Project overview • Paradigm shift in SOA - From application to process driven development • SOA Fundamentals • Quick introduction to Business Process Management (BPM) • Challenges and solutions in a SOA implementation • When is SOA a success • Which business advantages can we achieve by using SOA? • Discussion

39. Challenges in process driven SOA development 1/2 • IT and business are closer to each other. This increases the need for a detailed level of understanding of the business processes. • First we focus on the business processes, then we focus on the services

40. Challenges in process driven SOA development 2/2 • Which level of details should the processes have? • Which granularity should the services have? • Are there Pro’s and Con’s of a common data model? • How do we handle service lifecycle management?

41. How detailed should the processes be? 1/4 • Who is responsible for adding information to the processes?

42. Service granularity 2/3 How much should a service cover?

43. How detailed should the processes be? 3/4

44. How detailed should the processes be? 4/4 Assertion: Business people thinks in 2D and technical people thinks in 3D. Both ways of thinking are needed. • Experiences from our process work • Business people thinks in “Sunshine” versions (everything always go well) and perceive processes as processes. • Technical people thinks in “rain”-versions (takes error situations into account) and business process networks (i.e. the processes are a part of a larger business flow). They are not capable of seeing, which steps, that has to be taken to carry out a business process. • Solution: Levels with different degree of details

45. Service granularity 1/2 How much should a service cover? Services are customized to fit within certain areas of business needs A specific service pr. business requirement A general service Number of services 1 1000+

46. Service granularity 2/2 How much should a service cover?

47. Challenges with a common data model 1/2 What happens when a common data model is changed?

48. Challenges with a common data model 2/2 • Potentially many impacts and derived impacts • Great need for coordination and deployments of changes • Potentially a great need for regression test • Solution: Versioning of services • However, this increases the complexity of the solution

49. Lifecycle management of services 1/2 • When is it needed? • Design time governance • What services do we have? • Which ones can be reused? • Run time governance • Which applications uses which services? • Which services are the most used • Change governance • What will be impacted, if a service is changed • What happens if we discontinue a service

50. Lifecycle management of services 2/2 • Solution: Create a service repository • Contains data about all services and service calls • Can be used to lookup services • Can be used to make impact analyses • Can be used for service version control • But is it sufficient? • Where can security policies be managed? • Where can access control to services be managed? • Where should other types of service related metadata be managed? • How can consistency among business processes, services, security policies etc. be ensured?