Software Engineering Challenges Web Services and SOAP

1. Software Engineering ChallengesWeb Services and SOAP Richard Taylor & Eric Dashofy UC Irvine http://www.isr.uci.edu/classes/ics123s02/

2. Traditional Software Systems • are developed by a single organization • undergo a phased development process • with multiple phases of review, inspection, testing • have a synchronized release schedule • have a proprietary design and proprietary component interfaces • have a monolithic code base • go through a painful evolution

3. Our Vision The future of software development requires integratingnetwork servicesthat are very far awayand owned by strangers Rohit Khare 16 May 2002 O’Reilly Emerging Technology ConfUC Irvine & KnowNow, Inc.

4. Three Elements of Our Vision • The abstract unit of software is a network service • Prior levels of abstractions were subroutines, libraries, processes, objects, … • In this view, the only way to understand software is to watch its message traffic • Latency is an absolute limit of system architecture • Just as “real” architects grapple with gravity, so with the speed of light • London will always be 30ms from New York, regardless of Moore’s Law • Decentralization means crossing agencyboundaries • The essential complication above and beyond distributed (parallel) computing is extending a system to achieve consensus between several separate agencies • Every computing and communication device is owned by someone.

5. How Our Vision is Emerging Today • The technological shift from a component model to a service model of software is already underway • SOAP and Web Services technologies make it easy to write new protocols • The Internet is inexorably shifting to higher-latency, decentralized connectivity • Ad-hoc, peer-to-peer networking, mainly wireless, is being deployed at every scale from one meter (handhelds) to billions of meters (space probes) • Enterprise Application Integration (EAI) is only now reaching ‘outside the firewall’ to business partners • The next revolution in productivity requires reengineering processes end-to-end

6. So What’s This Talk About? • There is a ‘missing link’ between the near-future predicted by today’s Web Services hype and realizing the true potential of our vision • It is a new architectural style for software integration we call Application-Layer Internetworking (ALIN) • ALIN applies networking concepts to software engineering • The primary way that developers will use ALIN is an unheralded implication of SOAP 1.2: Routing • … so this talk is about a new style and a new device

7. What’s EAI? • Enterprise Application Integration is the unrestricted sharing of data and business processes throughout the networked applications or data sources in an organization • The Linking of Data, Business Processes, Applications that result in a common virtual system

8. What’s EAI? • Enterprise Application Integration is real-time integration of the information systems within an organization that support the different steps of a business process • Consider the following typical business process • An order might be captured by: • A CSR using an order entry system • A consumer using a web application • A business partner system using a web service interface • Regardless of the channel, the order must be financially and operational processed: • The financial system processes the billing • The operational system processes the manufacturing, scheduling, provisioning, clearance, etc. depending upon industry.

9. Without EAI AR Internal Credit AP External Logistics • Numerous Point-to-Point Interfaces • Manual Hand-Offs • Web Applications Increase the Number of Enterprise Integration Points • Inconsistent Business Processes • Inconsistent Qualities of Service ECommerce External Inventory

10. Without EAI • Numerous Point-to-Point Interfaces • Manual Hand-Offs • Web Applications Increase the Number of Enterprise Integration Points • Inconsistent Business Processes • Inconsistent Qualities of Service

11. With EAI EAI Infrastructure AR Business Process Automation Internal Credit Portal AP Data Transfor-mation Logistics External Web Service ECommerce Message Routing Inventory External Message Storage Adapter

12. With EAI • Reduced Number of Common, Consistent, Known Interfaces • Consistently Automated Business Processes • Increased Quality of Service (e.g. Order can Still be Captured even when Financial and Operational systems are Unavailable) • Framework for Integration of Web Services

13. Adapter Adapter Components in an EAI Solution • Adapters • Also known as “Connectors”, “Translators”, “Wrappers” • Provide seamless connectivity to the undrelying application or data • Convert information and events into data that can be utilized by the infrastructure • Two way translation between the proprietary lingo and the common lingo • Many adapters available off the shelf from EAI Software vendors such as Tibco, or Packaged Software vendors such as SAP. EAI Infrastructure Business Process Automation Data Transformation Message Routing Message Storage

14. Adapter Adapter Components in an EAI Solution • Message Broker • An intelligent broker that directs the flow of messages between sources and consumers of information • Message-Oriented Middleware (MOM) • A set of products that connect applications running on different systems by sending and receiving application data as messages • Message Routing • Set of tools that route messages between sources and consumers based on pre-defined business rules • Message Storage • Also identified as “Message Warehousing” • Central repository for temporary storage of transactions • Message Queuing • asynchronous communications style and provides a loosely coupled exchange across multiple operating systems • Publish Subscribe EAI Infrastructure Business Process Automation Data Transformation Message Routing Message Storage

15. Adapter Adapter Components in an EAI Solution • Data Transformers • Mapping of data elements to achieve common denominator • Transformation to and from Formats Used by Different Systems • GUI Configuration • Handles simple or complex data structures EAI Infrastructure Business Process Automation Data Transformation Message Routing Message Storage

16. Adapter Adapter Components in an EAI Solution • Process Modeling • Often GUI based • Describing the flow of information in the context of business processes • Using the input/output of processes as the Integration points between diverse business processes • Process Brokering • Execution of discrete steps within a business process • Ability to recover from failed steps • Workflow engine • Process Management • Monitor business processes • Correlate metrics to specific business process steps EAI Infrastructure Business Process Automation Data Transformation Message Routing Message Storage

17. EAI Power Tools • Many vendors have off the shelf connector technology that allows connectivity and integration of existing software packages and data stores. • GUI Transformation tools allow for visual creation and maintenance and versioning of data transformations. • Many MOMs (Message Oriented Middleware) already support automatic SOAP transformations • Many software packages are committed to SOAP as a standard which would make the EAI piece transparent

18. EAI: The Next Level Business Webs While EAI focuses on connecting data and applications internally, Business Webs focus on connecting applications and business flows across organizational boundries

19. Distribution/Warehousing Raw Materials Manufacturing Current Condition Transportation • Paperwork intensive • Human Intervention • Inherent inefficiencies

20. Your Close Affiliates Your Company The Challenge The Future Distribution Channels 1000’s 100’s 10’s 1 1 10’s 100’s 1000’s 10,000’s Suppliers

21. Relationships Are Dynamic. Each eBusiness relationship is unique. Every eBusiness initiative integrates disparate systems. Every change requires a skilled programmer. Time & Cost 1 5 20 100 1,000 # of unique eBusiness relationships

22. What are Web Services? • A Web service is a collection of functions that are packaged as a single entity and published to the network for use by other programs. • Web services are building blocks for creating open distributed systems, and allow companies and individuals to quickly and cheaply make their digital assets available worldwide. • A Web service can aggregate other Web services to provide a higher-level set of features

23. What are Web Services? • Based on UDDI (Universal Description, Discovery and Integration) and XML • Platform-independent, open framework for describing services, discovering businesses, and integrating business services.

24. Web Services Strengths • Interoperability • Based on open standards, utilizes existing infrastructure • Ubiquity • Communicates through XML/HTTP – Any system that supports these standards, supports Web Services • Low barrier to entry • Concepts easy to understand, easy to implement • Toolkits allow COM, J2EE, and CORBA components to be exposed as WebServices • Industry Support

25. Web Services Layers Publish, Find, Use Services: UDDI Service Interactions: SOAP Universal Data Format: XML Communications Layer: HTTP

26. SOAP- Definition • Simple Object Access Protocol • W3C standard: Not proprietary to any one software vendor • Like CORBA and COM, designed as an interoperability protocol that allows objects to “talk” to each other • SOAP defines a simple mechanism for expressing application semantics by providing a modular packaging model and encoding mechanisms for encoding data within modules

27. SOAP- Structure • Objects can be functions, programs, or systems • Three parts to SOAP: • Envelope: Defines a framework for describing what is in a message and how to process it, • Encoding rules: Expressing instances of application-defined datatypes, • RPC Representation: Convention for representing remote procedure calls and responses.

28. SOAP Example • Example 1: Sample SOAP Message embedded in an HTTP Request POST /StockQuote HTTP/1.1 Host: www.stockquoteserver.com Content-Type: text/xml; charset="utf-8" Content-Length: nnnn SOAPAction: "http://example.org/2001/06/quotes" <env:Envelope xmlns:env="http://www.w3.org/2001/06/soap-envelope" > <env:Body> <m:GetLastTradePrice env:encodingStyle="http://www.w3.org/2001/06/soap-encoding" xmlns:m="http://example.org/2001/06/quotes"> <symbol>DIS</symbol> </m:GetLastTradePrice> </env:Body> </env:Envelope>

29. Example, continued • Example 2 shows the SOAP message sent by the StockQuote service in the corresponding HTTP response to the request from Example 1. HTTP/1.1 200 OK Content-Type: text/xml; charset="utf-8" Content-Length: nnnn <env:Envelope xmlns:env="http://www.w3.org/2001/06/soap-envelope" > <env:Body> <m:GetLastTradePriceResponse env:encodingStyle="http://www.w3.org/2001/06/soap-encoding" xmlns:m="http://example.org/2001/06/quotes"> <Price>34.5</Price> </m:GetLastTradePriceResponse> </env:Body> </env:Envelope>

30. SOAP- Advantage • Once applications are wrapped with SOAP – they are in effect a web service • A web service can be invoked directly through an HTTP call • A web service can be “discovered” utilizing a directory service

31. What’s underneath SOAP? • Soap is the abstraction layer – not the integration tool • EAI techniques are used to connect between the SOAP wrapper and legacy systems • Once EAI objects are wrapped in SOAP they can easily communicate with each other thus creating a cross enterprise integration architecture

32. Web Services- Layers Publish, Find, Use Services: UDDI Service Interactions: SOAP Universal Data Format: XML Communications Layer: HTTP

33. UDDI • Universal Description, Discovery and Integration • Platform-independent, open framework for describing services, discovering businesses, and integrating business services. • A layer above SOAP, it is the glue that connects all the SOAP objects into a Business Web • DNS like model implementation of the directory

34. 4. 1. SW companies, standards bodies, and programmers populate the registry with descriptions of different types of services Marketplaces, search engines, and business apps query the registry to discover services at other companies 2. 5. Businesses populate the registry with descriptions of the services they support Business uses this data to facilitate easier integration with each other over the Web UBR assigns a programmatically unique identifier to each service and business registration 3. UDDI UDDI Business Registry BusinessRegistrations Service Type Registrations

35. Registry Data • Businesses register public informationabout themselves • Standards bodies, Programmers, Businesses register information about their Service Types

36. Business name • Text description • List of multi-language text strings • Contact info • Names, phone numbers, fax numbers, web sites… • Known identifiers • List of identifiers that a business may be known by - DUNS, Thomas, other

37. Business categories • 3 standard taxonomies in V1 • Industry: NAICS (Industry codes - US Govt.) • Product/Services: UN/SPSC (ECMA) • Location: Geographical taxonomy • Implemented as name-value pairs to allow any valid taxonomy identifier to be attached to the business white page

38. New set of information businesses use to describe how to “do e-commerce” with them • Nested model • Business processes • Service descriptions • Binding information • Programming/platform/implementation agnostic • Services can also be categorized

39. Service Type Registration • Pointer to the namespace where service type is described • What programmers read to understand how to use the service • Identifier for who published the service • Identifier for the service type registration • called a tModelKey • Used as a signature by web sites that implement those services

40. XML document Created by end-user company (or on their behalf) Can have multiple service listings Can have multiple taxonomy listings Contact Contact Phone Address Email Phone Address Email keyedReference keyedReference keyedReference keyedReference tModelKey keyName keyValue tModelKey keyName keyValue tModelKey keyName keyValue tModelKey keyName keyValue Business Registration businessEntity businessKey name URL description contacts businessServices identifierBag categoryBag businessService businessService serviceKey tModelKey Name Description BindingTemplates Key Name Description BindingTemplates

44. Talk Overview • What’s really new about Web Services • What a SOAP Router is • What a SOAP Router can do for you • How an SOAP Router works • ALIN in theory • ALIN in practice

45. Web Services are NOT New • There is nothing new under the sun in distributed (or parallel , or grid, …) computing • These are all efforts to lash many systems together to act as one • SOAP today is a distributed computing technology • ‘Sprinkling angle-bracket pixie dust’ over plain ol’ client/server RPC • Sure, it works across the public Internet, but only by treating it as a very slow LAN • Trade rags already lament the ‘gap’ at this boundary: real security, reliability, and performance problems reveal the fallacy of assuming TCP/IP ‘just works’

46. Web Services ARE new • Decentralizationdoes pose brand-new challenges • Latency • Services are no longer mere milliseconds away. • We need middleware for seconds up to weeks • Agency • Services can be owned by other organizations • We need middleware for translating semantics • Because SOAP messages are self-contained, they can be stored, analyzed, and replayed at will

47. SOAP is Simpler Than You Think • Starts w/point-to-point, synchronous invocations • just as Web page servers are accessed today • Main virtue is that it is not DCOM, IIOP, or RMI • Encapsulates the messy details of cross-platform, cross-language binding using standard marshalling formats • Getting started requires very little re-education. • On the other hand, straight RPC message exchange patterns (MEPs) don’t inherently reduce the tight coupling between client and server semantics.

48. SOAP is Cooler Than You Think: • The vastly underestimated power of SOAP is that its model is genuinely asynchronous • Today, we use it as a form of XML RPC • Underneath, that’s merely a pair 1-way messages • Furthermore, SOAP Actors are not hosts • An actor is a much more general concept than a fixed service provider machine and port • One actor may be a proxy for many others • Ideally, actors represent actual business models

49. What is ‘SOAP Routing’? • Today, the canonical Web Services diagram directly connects SOAP endpoints • There will also be a complementary technology that sits in the middle: • Store-and-forward reliable messaging • Filtering, transformation, & analysis of flows • Intelligent “active proxy chaining” of actors

50. What Good is ‘SOAP Routing’? 1/2 • Internet-scale Middleware • Just as distributed objects needed message-oriented-middleware (MOM)… • … decentralized services will need mediators • Wrappers: SOAP brokers to legacy systems • Monitors: SOAP transaction correlation & audit • Queues: decouples SOAP provider from requestor • Pub/Sub: decouples SOAP actors entirely • The difference is coping with firewalls and other kinds of agency boundaries