1 / 48

Part I: Quick and Dirty

T HE US N ATIONAL V IRTUAL O BSERVATORY. Web Service Technologies. Or How the Magic Happens. Matthew J. Graham CACR/Caltech. Part I: Quick and Dirty. No SOAP. Overview. Web services and SOA RESTful services Rich clients SOAP and WSDL Attachments Security State Asynchrony.

gaetane-graves
Download Presentation

Part I: Quick and Dirty

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. THE US NATIONAL VIRTUAL OBSERVATORY Web Service Technologies Or How the Magic Happens Matthew J. Graham CACR/Caltech Part I: Quick and Dirty No SOAP NVO Summer School 2005

  2. Overview • Web services and SOA • RESTful services • Rich clients • SOAP and WSDL • Attachments • Security • State • Asynchrony NVO Summer School 2005

  3. Web services (oc)cult • Invocations • Strange languages • Action at a distance • High priesthood NVO Summer School 2005

  4. What is a web service? • “A software system designed to support interoperable machine-to-machine interaction over a network. It has an interface described in a machine-processable format.” - W3C • Not a new idea: RPC, RMI NVO Summer School 2005

  5. Service Oriented Architecture • An application architecture within which all functions are defined as independent services with well-defined invocable interfaces which can be called in defined sequences to form scientific processes. • Not a new idea: CORBA, ICE • Principles: • Service reusability • Service contract • Service loose coupling • Service abstraction • Virtual Observatory = Service Oriented Astronomy • Service composability • Service autonomy • Service statelessness • Service discoverability NVO Summer School 2005

  6. How’s it done in the real world? • “Things should be made as simple as possible, but no simpler” - Albert Einstein • A web service is just a machine-readable web app and a web app is just a dynamic web site • WWW is the largest, most distributed and scalable application on the planet  HTTP and HTML (XML) NVO Summer School 2005

  7. REST: The formal approach • Architectural style not an implementation(http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm) • Each resource has a URI • Exchange resource representations (XML) • Uniform CRUD API: HTTP protocolCRUD actionDescription POST CREATE Create a new resource GET RETRIEVE Retrieve a resource representation PUT UPDATE Update a resource DELETE DELETE Delete a resource NVO Summer School 2005

  8. Accidentally RESTful • HTTP GET/POST + XML (POX/HTTP) • Verbs allowed in URIs • Requires little new infrastructure - just HTTP and XML processing technologies • Simple clients, e.g. wget or xsltproc • Commercially popular (85% of traffic, 6x faster): • Amazon (http://www.amazon.com/gp/aws/landing.html) • Yahoo (http://developer.yahoo.net) • eBay (http://developer.ebay.com/rest) • Flickr (http://www.flickr.com/services/) • Del.icio.us (http://del.icio.us/doc/api) NVO Summer School 2005

  9. What do RESTful services lack? • Reliable messaging • Digital signatures • Message routing • Resource life cycle management • Asynchronous event notification NVO Summer School 2005

  10. Is this a problem? • Security • transport level (HTTPS): point-to-point • fast and well supported • State • sessions • Asynchrony • easy to implement NVO Summer School 2005

  11. Client technologies: fat web pages • XForms (http://www.w3.org/Markup) • MVC approach to web forms separates content (XML model/instance) from presentation (XSLT/XHTML) • Server-side implementation: • Orbeon (http://www.orbeon.org) • Chiba (http://chiba.sourceforge.net) • Client-side implementation: • FormsPlayer (http://www.formsplayer.com) • FormFaces (http://www.formfaces.com) • Browsers (e.g. http://www.mozilla.org/projects/xforms) NVO Summer School 2005

  12. AJAX (Asynchronous Javascript + XML) • Uses browser’s XML support: DOM, XSLT • XMLHttpRequest • Google Maps, Google Suggest NVO Summer School 2005

  13. The bleeding edge • AFLAX (Asynchronous Flash + XML): • http://www.xamlon.com/software/xamlonpro/flash/aflax.aspx • OpenLaszlo (http://openlaszlo.org) • Mashups • On July 19, Google registered: googlesolarsystem.*, googlegalaxy.*, and googleuniverse.*, NVO Summer School 2005

  14. AJAX SOA NVO Summer School 2005

  15. THE US NATIONAL VIRTUAL OBSERVATORY Web Service Technologies Or How the Magic Happens Matthew J. Graham CACR/Caltech Part II: Standards SOAP rules! NVO Summer School 2005

  16. Recap • Client technologies: XForms, AJAX • RESTful approach: • Distributed hypermedia model (WWW) is key architecture for loosely coupled distributed systems • Each URL is a representation of some object, e.g. http://us-vo.org/conesearch?POS=123,34&SR=0.5 • Manipulate representation with CRUD interface • Application understands that resource access method is semantically significant • Low REST vs. high REST NVO Summer School 2005

  17. What is SOAP? • Originally meant Simple Object Access Protocol • An XML-based communication protocol and encoding format for exchanging structured information in a decentralized, distributed environment • W3C specification (http://www.w3.org/TR/soap) NVO Summer School 2005

  18. Anatomy of a SOAP message • An envelope to encapsulate data which defines formatting conventions for describing the message contents and routing directions: header and body • A message exchange pattern: request/response (RPC mechanism), fire-and-forget • A transport or binding protocol • Data encoding rules for describing the mapping of application-defined datatypes into an XML tag-based representation NVO Summer School 2005

  19. SOAP example Request: <soap:Envelope xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance xmlns:xsd=http://www.w3.org/2001/XMLSchema xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:Body> <ComovingLineOfSight xmlns="http://skyservice.pha.jhu.edu"> <z>float</z> <hubble>float</hubble> <omega>float</omega> <lambda>float</lambda> </ComovingLineOfSight> </soap:Body> </soap:Envelope> Response: <soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:Body> <ComovingLineOfSightResponse xmlns="http://skyservice.pha.jhu.edu"> <ComovingLineOfSightResult>float</ComovingLineOfSightResult> </ComovingLineOfSightResponse> </soap:Body> </soap:Envelope> NVO Summer School 2005

  20. Why is SOAP better? • Asynchrony • Routing • Reliable messaging (e.g. once-and-only delivery, guaranteed or exact execution) • Send and receive complex datatypes to invoke a particular method not just key-value pairs • Security • Binds to other protocols • Service description NVO Summer School 2005

  21. What is WSDL? • Web Services Description Language • An XML grammar for describing a web service as a collection of endpoints capable of exchanging messages in a particular fashion • W3C specification (http://www.w3.org/TR/wsdl) • http://www.xmethods.net NVO Summer School 2005

  22. Anatomy of a WSDL file <definitions> </definitions> <import>* - include other WSDLs <types> - define datatypes used in <message> <schema></schema>* </types> <message>* - model data exchanged <part></part>* </message> <porttype>* - a subset of operation supported for <operation>* an endpoint <input></input> - define input and output parameters <output></output> <fault></fault>* </operation> </porttype> <binding>* - formatting and representation of SOAP <operation>* message on the wire <input></input> <output></output> </operation> </binding> <service>* - identifies actual endpoint for WS <port></port>* </service> NVO Summer School 2005

  23. WSDL example • <definitions xmlns:http=http://schemas.xmlsoap.org/wsdl/http/ xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:s=http://www.w3.org/2001/XMLSchema xmlns:s0=http://skyservice.pha.jhu.edu xmlns:soapenc=http://schemas.xmlsoap.org/soap/encoding/ targetNamespace="http://skyservice.pha.jhu.edu" xmlns="http://schemas.xmlsoap.org/wsdl/"> • <types><s:schema elementFormDefault="qualified" targetNamespace="http://skyservice.pha.jhu.edu"> • <s:element name="ComovingLineOfSight">… • <s:element minOccurs="1" maxOccurs="1" name="z" type="s:float" /> • <s:element minOccurs="1" maxOccurs="1" name="hubble" type="s:float" /> • <s:element minOccurs="1" maxOccurs="1" name="omega" type="s:float" /> • <s:element minOccurs="1" maxOccurs="1" name="lambda" type="s:float" /> • …</s:element> • <s:element name="ComovingLineOfSightResponse”>… • <s:element minOccurs="1" maxOccurs="1" name="ComovingLineOfSightResult" type="s:float" /> • …</s:element> • </s:schema></types> • <message name="ComovingLineOfSightSoapIn"> • <part name="parameters" element="s0:ComovingLineOfSight" /> • </message> • <message name="ComovingLineOfSightSoapOut"> • <part name="parameters" element="s0:ComovingLineOfSightResponse" /> • </message> • <portType name="DistanceSoap"> • <operation name="ComovingLineOfSight"> • <documentation>Return the comoving line of sight distance...</documentation> • <input message="s0:ComovingLineOfSightSoapIn" /> • <output message="s0:ComovingLineOfSightSoapOut" /> • </operation> • </portType> • <service name="Distance"> • <port name="DistanceSoap" binding="s0:DistanceSoap"> • <soap:address location="http://voservices.net/Cosmology/ws_v1_0/Distance.asmx" /> • </port> • </service> • </definitions> NVO Summer School 2005

  24. What about the binding? <binding name="DistanceSoap" type="s0:DistanceSoap"> <soap:binding transport="http://schemas.xmlsoap.org/soap/http" style="document" /> <operation name="ComovingLineOfSight”> <soap:operation soapAction="http://skyservice.pha.jhu.edu/ComovingLineOfSight" style="document" /> <input> <soap:body use="literal" /> </input> <output> <soap:body use="literal" /> </output> </operation> </binding> NVO Summer School 2005

  25. Binding attributes • Style (representation on the wire) • rpc: the endpoint treats child elements in the body as XML representation of method call (SOAP 1.1, sec. 7) • document: the body can contain arbitrary XML • Use (how data is serialized across the wire) • encoded: rules in a URL specified by encodingStyle attribute • literal: rules specified by XML schema NVO Summer School 2005

  26. WSDL binding flavours (I) RPC Document • <message name=“Request”> • <part name=“x” type=“xs:int”/> • </message> • <message name=“empty”/> • <portType name=“foo”> • <operation name=“method”> • <input message=“Request”/> • <output message=“empty”/> • </operation> • </portType> • <types> • <schema> • <element name=“xElement” type=“xs:int”/> • </schema> • </types> • <message name=“Request”> • <part name=“x” element=“xElement”/> • </message> • <message name=“empty”/> • <portType name=“foo”> • <operation name=“method”> • <input message=“Request”/> • <output message=“empty”/> • </operation> • </portType> Literal • <message name=“Request”> • <part name=“x” type=“xs:int”/> • </message> • <message name=“empty”/> • <portType name=“foo”> • <operation name=“method”> • <input message=“Request”/> • <output message=“empty”/> • </operation> • </portType> Encoding NVO Summer School 2005

  27. WSDL binding flavours (II) RPC Document • <soap:envelope> • <soap:body> • <method> • <x>5</x> • </method> • </soap:body> • </soap:envelope> • <soap:envelope> • <soap:body> • <xElement>5</xElement> • </soap:body> • </soap:envelope> Literal • <soap:envelope> • <soap:body> • <method> • <x xsi:type=“xs:int”>5</x> • </method> • </soap:body> • </soap:envelope> Encoding NVO Summer School 2005

  28. WSDL binding flavours (III) Document/literal wrapped • <types> • <schema> • <element name=“method”> • <complexType> • <sequence> • <element name=“x” type=“xs:int”/> • </sequence> • </complexType> • </element> • </schema> • </types> • <message name=“Request”> • <part name=“parameters” element=“method”/> • </message> • <message name=“empty”/> • <portType name=“foo”> • <operation name=“method”> • <input message=“Request”/> • <output message=“empty”/> • </operation> • </portType • <soap:envelope> • <soap:body> • <method> • <x>5</x> • </method>> • </soap:body> • </soap:envelope> NVO Summer School 2005

  29. Which flavour to use? • Doc style can pass entire transaction as an XML document (state) • Doc style not constrained by RPC-oriented encoding • Doc style can be validated at call time • Processing overhead in encoding payloads with RPC • Doc style can use low memory parsers such as SAX and StAX • RPC’s natural tendency to expose programming language object structures  doc/literal wrapped (95%) NVO Summer School 2005

  30. Why not doc/literal wrapped? • Overloaded operations: public void myMethod (int x, float y); public void myMethod (int x); • Number of parameters: public void someOtherMethod(int x, float y); • Data graphs: <complexType name=“Node”> <sequence> <element name=“name” type=“string”/> <element name=“left” type=“Node”/> <element name=“right” type=“Node”/> </sequence> </complexType> RPC/encoding: <A> Literal: <A> <name>A</name> <name>A</name> <left href=“9999”/> <left> <right href=“9999”/> <name>B</name> </A> </left> <B id=“9999”> <right> <name>B</name> <name>B</name> </B> </right> </A> A Left Right B Left Right NVO Summer School 2005

  31. Client Invocation Models • Static: use generated stubs (wsdl2*) • Dynamic: • no generated code • a proxy dynamically generates a class at runtime that conforms to a particular interface, proxying all invocations to a single ‘generic’ method • Examples: • Java : use javax.xml.rpc.Service.getPort() and createCall() • .NET : use RealProxy class (must extend ContextBound) or Reflection.Emit • Generic SOAP client: http://soapclient.com/soaptest.html NVO Summer School 2005

  32. Interoperability • Suitable for and capable of being implemented in a neutral manner on multiple operating systems and in multiple programming languages • Not all web services are interoperable! • Web Services Interoperability Organisation (http://www.ws-i.org) • WS-I Testing Tools NVO Summer School 2005

  33. Attachments: opaque data • “By value” • XML representation or • use xs:hexBinary or xs:base64Binary within the body • data expansion by a factor of ~1.33 - 4 • anything within SOAP body gets parsed • processing costs of encoding/decoding • “By reference” • attach pure binary data as external unparsed entity outside SOAP message • use reference URI within the body NVO Summer School 2005

  34. Reference solutions • SwA (SOAP with Attachments) • Multipart MIME message: SOAP (0), data (1-n) • Use Content-Id as reference in body • Lack of length header on message sections • No recommendation just W3C Note • DIME (Direct Internet Message Encapsulation) • Uses faster and more efficient binary encoding • No standard, disowned by Microsoft • Both introduce a data structure outside realm of XML data model: no rules to specify how attachment content related to SOAP envelope so incompatible with WS-* NVO Summer School 2005

  35. MTOM • Message Transmission Optimization Mechanism • Uses MIME - backwards compatible with SwA • Uses XOP:Include as reference mechanism (XOP = XML Binary Optimized Packaging) • Conceptually binary data is base64-encoded in SOAP XML document  compatible with WS-* • Implementations: • Axis2 (http://ws.apache.org/axis2) • WSE 3.0 (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnwse/html/newwse3.asp) NVO Summer School 2005

  36. Security • Transport level (https) • Message level: • End-to-end: allows for unlimited intermediaries • Data origin authentication • Different types of security tokens/credentials: • unsigned (username/password) • binary (X.509 certificate) • XML (SAML token) • Multiple credentials NVO Summer School 2005

  37. WS-Security • OASIS standard (http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wss)) • Security token validation (authentication): • validate authentication assertions made by principals • Message integrity (signing): • verify message origin • validate encryption keys • confirm security token claims • Message confidentiality (encryption) • Introduces extra XML into SOAP header NVO Summer School 2005

  38. WSS Implementations • Java: • WSS4J (http://ws.apache.org/wss4j) • C#: • WSE 2.0 (http://msdn.microsoft.com/webservices/webservices/building/wse/default.aspx) • WSRF.Net (http://www.cs.virginia.edu/~gsw2c/wsrf.net.html) • Perl : • WS-Security will be supported by WSRF::Lite (but not yet) (http://www.sve.man.ac.uk/Research/AtoZ/ILCT) • Python: • pyGridWare (http://dsd.lbl.gov/gtg/projects/pyGridWare/) NVO Summer School 2005

  39. State • Stateless is good: • In case of failure, just restart without concern of previous interactions (reliability) • New service instances can be created/destroyed in response to system load (scalability) • How to handle state? • Separate web service and state information (resource) • Identify resource with a unique key • Use message exchanges with the service to interact with the resource (manipulate state) NVO Summer School 2005

  40. WS-Resource • An entity composed of a web service and a stateful resource • The address is called an endpoint reference (WS-Addressing) • ACID: • Updates made in all-or-nothing fashion (atomicity) • Consistent state even after failure (consistency) • Updates isolated within a given work unit (isolation) • Permanence of updates (durability) NVO Summer School 2005

  41. WS-RF: the nuts and bolts • WSDL for a stateful service: <definitions> <import>* <types> <xs:schema> <xs:element name=“StatefulResourceProperties”>…</xs:element> <xs:schema> <types> <porttype wsdlpp:extends=“…” wsrp:ResourceProperties=“tns:StatefulResourceProperties”> • Implementations: • Java: GT4 (htttp://www.globus.org); Apache WSRF (http://ws.apache.org/wsrf) • .NET: WSRF.Net (http://www.cs.virginia.edu/~gsw2c/wsrf.net.html) • Python: pyGridWare (http://dsd.lbl.gov/gtg/projects/pyGridWare/) • Perl: WSRF::Lite (http://www.sve.man.ac.uk/Research/AtoZ/ILCT) NVO Summer School 2005

  42. Asynchrony • Real world is asynchronous • No current standards for asynchronous services but most promising is OASIS ASAP (http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=asap) • Toolkits exist which facilitate asynchronous activities: • WS-RF (see above) • Axis2 (http://ws.apache.org/axis2) • JMS (http://java.sun.com/products/jms) / Caffeine (http://caffeine.berlios.de/site/) • WSIF (http://www.apache.org/wsif) NVO Summer School 2005

  43. Messaging operations • WSDL defines four types of messaging operation that an endpoint can support: • One-way: endpoint receives a message • Request/response: endpoint receives a message and sends a correlated message • Solicit/response: endpoint sends a message and receives a correlated message • Notification: endpoint sends a message • One-way/two-way transport behaviour NVO Summer School 2005

  44. Patterns for asynchrony (I) • Fire and Forget: • Request/response (Transport timeout) C S C S C S NVO Summer School 2005

  45. Patterns for asynchrony (II) • Polling: • Callback: C S C S NVO Summer School 2005

  46. WS-Addressing • No standard SOAP way to specify: • where a message is going • how to return a response • where to report an error • WS-Addressing provides: • To • ReplyTo • FaultsTo • Anonymous • MessageId / RelatesTo • Standard for including service-specific attributes NVO Summer School 2005

  47. MEST (MESsage Transfer) • Messaging: • No notion of client/server: just peers • Largely time independent: messages delivered when peer is available • Messages can be duplicated and delivered to multiple peers • Messages and services are first class abstractions (no interfaces, data and operations) • SSDL (http://www.ssdl.org) • Indigo: dual contracts are beyond WSDL NVO Summer School 2005

  48. Conclusions • Pick the right approach: do you need SOAP? • Location of functionality: fat web pages • Think about interactions: WSDL • Use your favourite language and platform • Interoperability is a great idea • Be aware of emerging and converging technologies and new paradigms NVO Summer School 2005

More Related