Ace104 Lecture 5

1. Ace104Lecture 5 The Flavor of SOAP: A study of the key concepts without all of the detail

2. Some facts about SOAP • SOAP no longer stands for Simple Object Access Protocol • This was dropped in 2003 with publication of v1.2 • Considered to be a misleading name • Originally was XML-RPC • Created in 1998 as a very lightweight protocol for rpc • Was basis for development of SOAP several years later • SOAP currently maintained by W3C XML working group • SOAP and XML-RPC are based entirely on XML

3. SOAP as a messaging protocol • SOAP is fundamentally a stateless, one-way message exchange paradigm • However, applications can create more complex interaction patterns (e.g., request/response, request/multiple responses, etc.) by combining such one-way exchanges with features provided by an underlying protocol and/or application-specific information. • SOAP is silent on the semantics of any application-specific data it conveys, as it is on issues such as the routing of SOAP messages, reliable data transfer, firewall traversal, etc. • However, SOAP provides the framework by which application-specific information may be conveyed in an extensible manner. Also, SOAP provides a full description of the required actions taken by a SOAP node on receiving a SOAP message.

4. SOAP • SOAP is made up of three major parts • A generic XML messaging framework • An data encoding standard • An RPC (remote procedure call) framework • It is possible to use just the messaging framework or messaging framework/encoding standard without using the RPC mechanism (though latter is where much of power/usefulness lies).

5. Web Services • Note that classic Web Services are made up of three parts • SOAP • WSDL (Web Services Descriptor Language) • UDDI (Universal Description, Discovery, and Integration) • All three are based on XML • SOAP simply defines the structure of the XML document used to transfer the message • WSDL and UDDI are covered next

6. SOAP: Messaging framework • Just defines a generic message XML schema • Virtually any type of message you can think of can be packaged as a SOAP message. • However, doing so without RPC mechanisms takes only very small advantage of the features defined in the SOAP standard

7. General (Basic) Structure SOAP Message • Envelope • Defines the content of the message • Header (optional) • Contains destination information, versioning, extensions • Good place for security • Body • Contains payload SOAP Envelope SOAP Header SOAP Body Payload Document(s) SOAP Fault

8. General (Basic) Structure SOAP Message <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/encoding/" soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"> <soap:Header> ... ... </soap:Header> <soap:Body> <!-- User request code here --> <soap:Fault> ... ... </soap:Fault> </soap:Body> </soap:Envelope>

9. SOAP encoding • In order to build SOAP messages from our language of choice, we need to know how to serialize data -- ie the rules for representing an integer, string, or floating point number. The serialization of data inside a SOAP message is referred to as encoding • The encodingStyle attribute defined by the SOAP specification is used to identify the encoding rules used in a particular message. • SOAP does this in a language agnostic way, much like CORBA (but not in binary form) • If the encodingStyle attribute does not appear in the message, the receiver cannot make any assumptions about how data will be represented within the message

10. SOAP Encoding • You may be wondering about the relationship between encoding and XML Schemas • Encoding can make use of XML Schemas. • The SOAP Specification defines a single set of (recommended) encoding rules call SOAP encoding. SOAP encoding is based on XML Schemas and as such it closely models many of the standard types and constructs. The value is http://schemas.xmlsoap.org/soap/encoding/, which points to the XML Schema that defines the encoding rules. • SOAP encoding rules use XML Schemas heavily, relying on the XML Schema datatypes namespace and the type attribute. • The key difference is that encoding does not mandate XML Schemas. • Encoding rules are simply identified by a URI. The rules implied by that URI could be backed up by nothing more than a verbal agreement, or possibly some written documentation. • This allows developers who do not necessarily need the capabilities of XML Schemas to forego their use and start sending messages with encoding rules based on an accepted URI.

11. SOAP Encoding • For example, SOAP stipulates that an array of three integers be represented as: <SOAP-ENC:Array SOAP-ENC:arrayType="xsd:int[3]"><SOAP-ENC:int>8</SOAP-ENC:int><SOAP-ENC:int>5</SOAP-ENC:int><SOAP-ENC:int>9</SOAP-ENC:int> </SOAP-ENC:Array> • SOAP also provides a type for representing binary data One approach for working with binary data is to use the base64 type. We can represent binary data, such as an image file, as an array of bytes in the message. The base64 type converts binary data to text using the base64-encoding algorithm of XML Schemas. There is no relationship between SOAP and base64-encoding; If we use it, our application (or implementation of SOAP for your platform) must be able to understand and work with base64-encoding.

12. SOAP RPC • The third part of SOAP is an RPC mechanism that turns messages into method calls • We have a generic message structure + data. It requires just a little more work to turn the message into a function call. • Must be a standard way to represent parameters and return values, exceptions, etc. • Note that the encoding and rpc mechanisms are only important if SOAP is being automatically generated/read from the application level (see next slide) in a general way

13. SOAP RPC cartoon VB application Java application InvoiceVB-Structure InvoiceJava-Structure SOAP client SOAP Server SOAP Message The client application thinks its making a procedure call to a remote module

14. Sample RPC rules • This is intended just to give you a flavor. Best to allow • applications to do this automatically: • Consider the following three method signatures • // Reverse the string, s, and return the new string. • string ReverseString ( [in] string s ); • // Reverse the string, s, and return the new string. • void ReverseString ( [in] string s, [out] string sRev ); • // Reverse the string, s, passed in by reference. • void ReverseString ( [in, out] string s ); • See next slide for SOAP rpc encoding

15. Sample rpc rules The first version reverses the string and returns the result as the return value of the method <x:ReverseStringResponse xmlns:x="http://www.wrox.com/"> <x:ret xsi:type="xsd:string">THOR</x:ret> </x:ReverseString> The second version has no return value, but instead uses an out parameter called sRev: <x:ReverseStringResponse xmlns:x="http://www.wrox.com/"> <x:sRev xsi:type="xsd:string">THOR</x:sRev> </x:ReverseString> The final version reverses the string after passing it by reference, so the parameter s is both an in and out parameter: <x:ReverseStringResponse xmlns:x="http://www.wrox.com/"> <x:s xsi:type="xsd:string">THOR</x:s> </x:ReverseString>

16. SOAP transport • Recall that SOAP is just a generic message envelope • Augmented by encodingstyle and simple rpc rules, it becomes a powerful middleware layer for remote procedure calls, if one chooses to use it that way • Now we must consider how to transport SOAP messages -- this is the final ingredient in making it something useful

17. SOAP protocol bindings • Question:how are SOAP messages transmitted? • Answer: using existing protocols (http, SMTP, etc.) • This has some obvious advantages vs. defining its own protocol • Piggybacks on security model, general robustness • This has some disadvantages also • What are these? • SOAP defines bindings to different protocols that specify how SOAP is used with that protocol to send messages. • http is most popular

18. Inside http • http is a simple, flexible protocol • Some examples GET http://people.cs.uchicago.edu/~asiegel/lottery/lotto.html POST /path/script.cgi HTTP/1.0 From: frog@jmarshall.com User-Agent: HTTPTool/1.0 Content-Type: application/x-www-form-urlencoded Content-Length: 32 home=Cosby&favorite+flavor=flies POST /path/script.cgi HTTP/1.0 From: frog@jmarshall.com User-Agent: HTTPTool/1.0 Content-Type: text/xml Content-Length: 32 <greeting>hello world</greeting>

19. Testing http • Good idea to play around with http by connecting to server and issuing http commands • There are a two typical ways to do this: • Using telnet, which allows arbitrary commands to be passed to a server • telnet people.cs.uchicago.edu 80 • Note that expect can be useful in automating this • Using a socket library in a programming language (see sock.py on website) • Question: how does the server obtain the uploaded data in each case?

20. Role of SOAP • Note that the http + XML is the important thing here • SOAP only helps standardize the meaning of the messages that are sent • In terms of datatypes for rpc • In terms of headers, faults, etc. • Note that it is still possible to bypass SOAP and define your own xml-based protocol, retaining many of the advantages of SOAP.

21. Sorting out the API’s • In Java the following directly related API’s are available: • SAAJ • SOAP with Attachments API for Java • Provides a relatively low-level interface that allows one to programmatically construct/decompose SOAP messages and send to web server • Intended more tool writers. Good for learning. • JAX-RPC • Java API form XML-based RPC • Java’s rmi framework over SOAP • Compare RMI, CORBA, etc. • Makes developer unaware of SOAP internals • Apache XML-RPC for Java • A framework for remote procedure calls using XML-RPC • Recall that XML-RPC is an alternative protocol to SOAP

22. Looking deeper into SOAP

23. Envelope • MUST be the root element of the SOAP message • MUST be associated with SOAP envelope namespace • http://schemas.xmlsoap.org/soap/envelope • http://www.w3.org/2001/06/soap-envelope in SOAP 1.2 (Oct 15, 2002) • SOAP serialization namespace • Encoding Style attributes can contain a URI describing how the data should be serialized. • Two usual styles (more on this later) • "SOAP Section 5" encoding: http://www.w3.org/2001/06/soap-encoding • Literal encoding: (no namespace used – or set to empty string) • SOAP message MUST NOT contain • DTD • Processing Instructions.

24. Envelope versioning • Version determined by the namespace associated with the Envelope element • SOAP 1.1 Envelope version: http://schemas.xmlsoap.org/soap/envelope • If any other namespace used, assume it's a version problem • Versioning problems must generate a SOAP Fault • Example SOAP fault: HTTP/1.0 500 Internal Server Error Content-Type: text/html; charset="utf-8" Content-length: 311 <env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"> <env:Body> <env:Fault> <faultcode>env:VersionMismatch</faultcode> <faultstring>SOAP Envelope Version Mismatch</faultstring> </env:Fault> </env:Body> </env:Envelope>

25. Envelope Versioning Fault in SOAP 1.2 Note that 1.2 Envelope Version Fault Response is versioned 1.1 (or whatever incoming request is) • SOAP 1.2 (Oct 15, 2002) has defined an Upgrade element in the header for the versioning fault: <?xml version="1.0" ?> <env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"> <env:Header> <upg:Upgrade xmlns:upg="http://www.w3.org/2002/06/soap-upgrade" > <envelope qname="ns1:Envelope" xmlns:ns1="http://www.w3.org/2002/06/soap-envelope" /> </upg:Upgrade> </env:Header> <env:Body> <env:Fault> <faultcode>env:VersionMismatch</faultcode> <faultstring>Version Mismatch</faultstring> </env:Fault> </env:Body> </env:Envelope>

26. Header • Optional • If present, must immediately follow the SOAP Envelope XML element followed by any header entries • Uses same namespace as Envelope • Often contains meta-information regarding the method call. • Examples: • Security • No security mechanisms yet, but soon • Transaction IDs

27. Header • actor attribute  defines the URI for which the header elements are intended (i.e. who should process a Header element) • mustUnderstand attribute  how to process (default is “0” if not present) • encodingStyle attribute  used to describe how data (such as binary integers) are marshaled into characters in the XML document <env:Header> <t:TransactionID xmlns:t="http://www.cs.uchicago.edu/dangulo/transact" env:mustUnderstand="1" env:actor="http://www.cs.uchicago.edu/dangulo/transact" > 42 </t:TransactionID> <m:localizations xmlns:m="http://www.cs.uchicago.edu/dangulo/localize/" env:actor="http://www.cs.uchicago.edu/dangulo/currency" > <m:language>en</m:language> <m:currency>USD</m:currency> </m:localizations> </env:Header>

28. actor Attribute • The SOAP message often gets passed through several intermediaries before being processed • For example, a SOAP proxy service might get the message before the target SOAP service • Header may contain information for both • intermediary service • target service • actor attribute specifies which service should process a specific Header element • actor attribute is replaced by role attribute in SOAP 1.2

29. Intermediary Services • SOAP requires that an intermediary strip off Header elements specified for that intermediary before passing the message to the next service in the chain. • If information in a Header element targeted for an intermediary is also needed by another service in the chain • The intermediary service may insert additional Header elements with an actor attribute that specifies the downstream service • In fact, any service may insert any Header elements that it deems necessary • If a Header element has no actor attribute • It is assumed to be destined for the final recipient • This is equivalent to adding an actor attribute with the URL of the final recipient

30. mustUnderstand Attribute • Also put on a Header element • If its value is "1" • recipient is required to understand and make proper use of the information supplied by that element • intended for situations where recipient can't do its job unless it knows what to do with the specific information supplied by this particular element • Examples of use • Client is upgraded to a new version which includes extra information • username • security

31. mustUnderstand Attribute • If the recipient does not understand this element • Must respond with a SOAP Fault HTTP/1.0 500 Internal Server Error Content-Type: text/xml; charst="utf-8" Content-length: 287 <env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"> <env:Body> <env:Fault> <faultcode>env:MustUnderstand</faultcode> <faultstring>SOAP Must Understand Error</faultcode> <faultactor>http://www.cs.uchicago.edu/dangulo/transact</faultactor> </env:Fault> </env:Body> </env:Envelope> • faultactor indicates where fault took place • We'll look at Faults in more detail later • Attribute values change to "true" / "false" in SOAP 1.2

32. Some examples Taken from W3C primer

33. Sample SOAP message for travel reservation <?xml version='1.0' ?> <env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope"> <env:Header> <m:reservation xmlns:m="http://travelcompany.example.org/reservation" env:role="http://www.w3.org/2003/05/soap-envelope/role/next" env:mustUnderstand="true"> <m:reference>uuid:093a2da1-q345-739r-ba5d-pqff98fe8j7d</m:reference> <m:dateAndTime>2001-11-29T13:20:00.000-05:00</m:dateAndTime> </m:reservation> <n:passenger xmlns:n="http://mycompany.example.com/employees" env:role="http://www.w3.org/2003/05/soap-envelope/role/next" env:mustUnderstand="true"> <n:name>Andrew Siegel</n:name> </n:passenger> </env:Header> Next slide ..

34. <env:Body> <p:itinerary xmlns:p="http://travelcompany.example.org/reservation/travel"> <p:departure> <p:departing>New York</p:departing> <p:arriving>Los Angeles</p:arriving> <p:departureDate>2001-12-14</p:departureDate> <p:departureTime>late afternoon</p:departureTime> <p:seatPreference>aisle</p:seatPreference> </p:departure> <p:return> <p:departing>Los Angeles</p:departing> <p:arriving>New York</p:arriving> <p:departureDate>2001-12-20</p:departureDate> <p:departureTime>mid-morning</p:departureTime> <p:seatPreference/> </p:return> </p:itinerary> <q:lodging xmlns:q="http://travelcompany.example.org/reservation/hotels"> <q:preference>none</q:preference> </q:lodging> </env:Body> </env:Envelope>

35. Sample SOAP reply <?xml version='1.0' ?> <env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope"> <env:Header> <m:reservation xmlns:m="http://travelcompany.example.org/reservation" env:role="http://www.w3.org/2003/05/soap-envelope/role/next" env:mustUnderstand="true"> <m:reference>uuid:093a2da1-q345-739r-ba5d-pqff98fe8j7d</m:reference> <m:dateAndTime>2001-11-29T13:35:00.000-05:00</m:dateAndTime> </m:reservation> <n:passenger xmlns:n="http://mycompany.example.com/employees" env:role="http://www.w3.org/2003/05/soap-envelope/role/next" env:mustUnderstand="true"> <n:name>Andrew Siegel</n:name> </n:passenger> </env:Header> Next slide …

36. <env:Body> <p:itineraryClarification xmlns:p="http://travelcompany.example.org/reservation/travel"> <p:departure> <p:departing> <p:airportChoices> JFK LGA EWR </p:airportChoices> </p:departing> </p:departure> <p:return> <p:arriving> <p:airportChoices> JFK LGA EWR </p:airportChoices> </p:arriving> </p:return> </p:itineraryClarification> </env:Body> </env:Envelope>

37. Reply continuing conversational exchange <?xml version='1.0' ?> <env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope"> <env:Header> <m:reservation xmlns:m="http://travelcompany.example.org/reservation" env:role="http://www.w3.org/2003/05/soap-envelope/role/next" env:mustUnderstand="true"> <m:reference>uuid:093a2da1-q345-739r-ba5d-pqff98fe8j7d</m:reference> <m:dateAndTime>2001-11-29T13:36:50.000-05:00</m:dateAndTime> </m:reservation> <n:passenger xmlns:n="http://mycompany.example.com/employees" env:role="http://www.w3.org/2003/05/soap-envelope/role/next" env:mustUnderstand="true"> <n:name>Andrew Siegel</n:name> </n:passenger> </env:Header> <env:Body> <p:itinerary xmlns:p="http://travelcompany.example.org/reservation/travel"> <p:departure> <p:departing>LGA</p:departing> </p:departure> <p:return> <p:arriving>EWR</p:arriving> </p:return> </p:itinerary> </env:Body> </env:Envelope>

38. RPC Notice that previous did not include rpc capability To invoke a SOAP RPC, the following information is needed: The address of the target SOAP node. The procedure or method name. The identities and values of any arguments to be passed to the procedure or method together with any output parameters and return value. 4. A clear separation of the arguments used to identify the Web resource which is the actual target for the RPC, as contrasted with those that convey data or control information used for processing the call by the target resource. The message exchange pattern which will be employed to convey the RPC, together with an identification of the so-called "Web Method” (on which more later) to be used. 6. Optionally, data which may be carried as a part of SOAP header blocks.

39. SOAP RPC request with a mandatory header and two input (or "in") parameters <?xml version='1.0' ?> <env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope" > <env:Header> <t:transaction xmlns:t="http://thirdparty.example.org/transaction" env:encodingStyle="http://example.com/encoding" env:mustUnderstand="true" >5</t:transaction> </env:Header> <env:Body> <m:chargeReservation env:encodingStyle="http://www.w3.org/2003/05/soap-encoding" xmlns:m="http://travelcompany.example.org/"> <m:reservation xmlns:m="http://travelcompany.example.org/reservation"> <m:code>FT35ZBQ</m:code> </m:reservation> <o:creditCard xmlns:o="http://mycompany.example.com/financial"> <n:name xmlns:n="http://mycompany.example.com/employees">Andrew Siegel</n:name> <o:number>123456789099999</o:number> <o:expiration>2005-02</o:expiration> </o:creditCard> </m:chargeReservation> </env:Body> </env:Envelope>

40. RPC response with two output (or "out") parameters ?xml version='1.0' ?> <env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope" > <env:Header> <t:transaction xmlns:t="http://thirdparty.example.org/transaction" env:encodingStyle="http://example.com/encoding" env:mustUnderstand="true">5</t:transaction> </env:Header> <env:Body> <m:chargeReservationResponse env:encodingStyle="http://www.w3.org/2003/05/soap-encoding" xmlns:m="http://travelcompany.example.org/"> <m:code>FT35ZBQ</m:code> <m:viewAt> http://travelcompany.example.org/reservations?code=FT35ZBQ </m:viewAt> </m:chargeReservationResponse> </env:Body> </env:Envelope>

41. RPC response with a "return" value and two "out" parameters <?xml version='1.0' ?> <env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope" > <env:Header> <t:transaction xmlns:t="http://thirdparty.example.org/transaction" env:encodingStyle="http://example.com/encoding" env:mustUnderstand="true">5</t:transaction> </env:Header> <env:Body> <m:chargeReservationResponse env:encodingStyle="http://www.w3.org/2003/05/soap-encoding" xmlns:rpc="http://www.w3.org/2003/05/soap-rpc" xmlns:m="http://travelcompany.example.org/"> <rpc:result>m:status</rpc:result> <m:status>confirmed</m:status> <m:code>FT35ZBQ</m:code> <m:viewAt> http://travelcompany.example.org/reservations?code=FT35ZBQ </m:viewAt> </m:chargeReservationResponse> </env:Body> </env:Envelope>

42. Extra Slides In progress, read to get a sense of underlying details

43. Marshalling / Serialization VB application Java application Data here is binary Data here is binary Data here is ASCII InvoiceVB-Structure InvoiceJava-Structure Must Marshall or Serialize Must UnMarshall or DeSerialize SOAP client SOAP Server SOAP Message • To be interoperable, we use XML • XML is ASCII, not binary • End points use binary • Must Marshall (Serialize) and UnMarshall (DeSerialize) on the ends

44. Body • Message to exchange. • Most often for RPC calls and error reporting. • Immediate child element of SOAP Envelope XML element • follows Header, if present • Uses same namespace as Envelope and Header • Contains serialized method arguments. • Remote method name • Used to name the method call’s XML element • Must immediately follow the SOAP body opening XML tag. • SOAP Fault goes in the Body (of a response) too • The only Body elements actually defined in the SOAP specification are the SOAP Fault elements • Other elements are user defined

45. Example • A simple SOAP XML document requesting the price of soap (leaving off the required namespaces declarations) <env:Envelope> <env:Body> <m:GetPrice> <Item>Lever2000</Item></m:GetPrice> </env:Body> </env:Envelope> • Note that namespaces qualifiers are not required on elements in the Body.

46. Client/Server… • In order for SOAP to work • Client must have code running that is responsible for building the SOAP request. • Server must also be responsible for • Understanding the SOAP request • Invoking the specified method • Building the response message • Returning it to the client. • These details are up to you. • There already exist SOAP implementations for languages such as C++, Perl, VB, and Java.

47. Binding • SOAP is transport independent • SOAP usually transported over HTTP • SOAP can be transported over any protocol • e.g. SMTP (e-mail) • GSI (Globus Secure Transport) • HTTPS • pure sockets • HTTP is the default binding

48. SOAPAction HTTP header • When using SOAP over HTTP, must include SOAPAction header • SOAPAction HTTP request header field indicates that it is a SOAP HTTP request (contains a SOAP message) • The value • Indicates the intent of the request in a manner readily accessible to the HTTP server. • Is a URI • Is up to the application – not specified by SOAP specs • Doesn't have to be resolvable • An HTTP client must use SOAPAction header field when issuing a SOAP HTTP Request. • An HTTP server must not process an HTTP request as a SOAP HTTP request if it does not contain a SOAPAction header field. • It may be used by firewalls to filter request messages • It may be used by servers to facilitate dispatching of SOAP messages to internal message handlers • It should not be used as an insecure form of access authorization.

49. SOAPAction HTTP header • Example POST /xt/services/ColorRequest HTTP/1.0 Content Length: 442 Host: localhost Content-type: text/xml; charset=utf-8 SOAPAction: "/getColor" <!?xml version="1.0" encoding="UFT.8"?> <env:Envelope env:encodingStyle="http://schemas.xmlsoap.org/SOAP/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:env="http://schemas.xmlsoap.org/SOAP/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> ...

50. SOAP Messages with Attachments • SOAP messages often have attachments, such as pictures • The attachments don't have to be XML encoded, but may be binary • The SOAP message becomes the root of a Multipart/Related MIME structure • The SOAP message refers to the attachment using a URI with the cid: protocol • cid = "content ID"