XL – An XML Programming Language for Web Services

1. XL – An XML Programming Languagefor Web Services Daniela Florescu Andreas Grünhagen Donald Kossmann Presented by Yury Berezansky

2. Agenda • Web services development - state of the art • Requirements for a Web Service Programming Language • Web services in XL • XL statements • Implementation • Example

3. Agenda • Web services development - state of the art • Requirements for a Web Service Programming Language • Web services in XL • XL statements • Implementation • Example

4. Web service - definition • Web service is an autonomous piece of software uniquely identified by an URI and that can interact with peer Web services via messages using Internet protocols like XML Protocol or HTTP • Web services can preserve an internal state • Web services can participate in complex conversations (exchange of correlated messages among a certain number of participant Web services)

5. Current approach • Most Web services are built using classic programming languages (Java, VB, C#) • Most Web services are built using SQL-based RDBMS

6. Drawbacks of the current approach • Costly and error-prone data transformations are involved: • XML - language mismatch • Language - RDBMS mismatch • XML, existing languages and relational databases are too different and incompatible for building large scale applications • Often the application tire mixes very different semantic actions (e.g. low level protocols manipulation, performance optimizations, data validation, high level application logic) in the same imperative language • WSDL interfaces must be constructed manually

7. Agenda • Web services development - state of the art • Requirements for a Web Service Programming Language • Web services in XL • XL statements • Implementation • Example

8. High-level programming andservice composition • The language must be high level and use declarative constructs whenever possible • The language must support high-level exception handling and other special constructs to easily implement more complex services like logging, data lineage, time-triggered actions, etc. • The language must allow the construction of high-level services out of the composition of more basic services which are not necessarily written in this language

9. Business processes, Web conversations • The language must provide constructs to implement business processes and, more generally, it must support conversations between two or more Web services

10. Unique XML-based data model and type system • The data manipulated must be modeled by the standard XML data model and type system • No other data models and type systems are allowed

11. Location independent invocation • Web services must be uniquely identified using URIs • The invocation of a Web service must use the respective URI and be independent from the location where its code is stored or executed

12. Compliance with the W3C standards • XL must be compliant with the XML W3C standards such as: • XML Schema • XQuery • XPath • XSLT • XML Forms • XML Protocol • WSDL

13. Agenda • Web services development - state of the art • Requirements for a Web Service Programming Language • Web services in XL • XL statements • Implementation • Example

14. Design philosophy • Provide the missing link between the various XMS standards set by W3C • By combining the type system, expressions and the service interfaces into a single language, take advantage of existing standards without abandoning possible optimizations • Extend the existing W3C standards in order to obtain a complete and uniform programming model for Web services

15. Web service declarations • A Web Service in XL: • generalizes the notion of an XQuery entity • is identified by a URI • can contain: • a set of local function declarations • a set of type definitions and schema and namespace imports • local data declarations • declarative clauses • specifications of the Web service operations service <uri> < FUNCTIONS & LOCAL OPERATIONS > < LOCAL DECLARATIONS > < DECLARATIVE WEB SERVICE CLAUSES > < PUBLIC OPERATION SPECIFICATIONS > endservice

16. Functions • The same as in XQuery

17. Variable declarations • Local variables can hold only XML data and their values can be constrained by the XML type system • Service-wide variables • represent the internal state of the whole Web service • instantiated once when the Web service is installed and persist the whole life time of the Web service • the scope of these variables is the whole Web service. • example: the warehouse database of an online shop • Conversation variables • represent the internal state of a particular conversation • instantiated when the Web service joins a new conversation and persist until the conversation termination • multiple instances can exist • example: session id, maximum bid for an item

18. Variable declarations (cont.) service <uri> ! ! function definitions, local types, schema imports . . . ! ! state of the Web service (let [<type>]<variablename> [:= <expression>];)* ! ! state of a conversation of the service (context let [<type>]<variablename> [ := <expression> ];)* ! ! declarative Web services clauses and operations . . . endservice • “type” is an optional type constraining the type of the variable’s value. If not specified, the variable can be bound to any valid instance of the XML data model. • “expression” is an XQuery expression describing the initial value of the variable. If not specified, the variable is initialized to the empty sequence.

19. Declarative Web service clauses • A set of high level declarations that control the Web services global state, how the Web service operations are executed, and how the Web services interacts with other Web services. • HISTORY • Automatically records all calls to operations of the Web service in an implicitly declared read only $history variable • This includes: operation, caller’s UIR, input and output messages, timestamp • DEFAULT & UNKNOWNOPERATION • DEFAULT operation is executed whenever a message with no operation specification is received • UNKNOWN operation is executed whenever a message with undefined (in this Web service) operation specification is received

20. Declarative Web service clauses (cont.) • INIT & CLOSE • Specify a pair of operations that are automatically invoked when the Web service is created and destroyed, respectively • These operations can only be invoked once and they take no input • INVARIANTS • Defines global Web services integrity constraints (or invariants) • An arbitrary number of invariants can be defined • If (at any time) an invariant is violated, the statement that caused the violation is undone, and an exception is raised • Typically, invariants are defined for stateful services and constrain the value of user-defined variables, the $history variable and contexts of conversations

21. Declarative Web service clauses (cont.) • ON CHANGE • If the value of a certain variable changes, the specified operation is called with an empty input • Can monitor changes to any variable declared in the Web services local declarations • ON EVENT • Whenever a given boolean expression evaluates to true, a specified operation is invoked • If an INPUT is specified, the corresponding expression is evaluated and passed to the operation as input

22. Declarative Web service clauses (cont.) • ON ERROR INVOKE • Specifies an operation to be invoked for unhandled exceptions • An exception is passed as input to the operation • The output of this operation is returned to the client of the Web service

23. Declarative Web service clauses (cont.) • CONVERSATIONPATTERN • Specifies how the Web service interacts with other services as part of conversations • If a pattern is specified, then the URI of the conversation is set implicitly whenever the Web service sends a message to another Web service • Each single operation can overwrite the default pattern

24. Declarative Web service clauses (cont.) • Required pattern • If the Web service receives a message that has no conversation URI (i.e., is not part of a conversation), then the Web service will generate a new conversation URI and all other Web services it calls as part of processing the input message will be called using this new conversation URI (C2). • If the Web service receives a message with a conversation URI, then all other Web services it calls as part of processing the input message will be called using the conversation URI of the input message. • Mandatory pattern • First operation should establish a conversation • All other operations must be called as part of the same conversation

25. Agenda • Web services development - state of the art • Requirements for a Web Service Programming Language • Web services in XL • XL statements and statement combinators • Implementation • Example

26. Variable assignments • Syntax: • let [type] variable := expression • Local variables need not be declared before being used • (XML schema) type of a variable can optionally be set as part of the first assignment to this variable • The scope of a variable is the block where the variable is defined • Expressions can be any expression defined by the W3C XQuery proposal

27. Update Statements • W3C has not yet released a recommendation to manipulate XML data (there is only an initial proposal). Once a recommendation has been released by the W3C, XL is going to adopt the syntax and semantics of that recommendation. In the meantime, the following statements are used to manipulate XML data: • insert in order to add new nodes to the XML hierarchy • insert <creditcard> . . .</creditcard> into $customer • delete in order to delete nodes from the XML hierarchy • delete $customer/creditcard[type=”Visa”]

28. Update Statements (cont.) • replace in order to adjust elements • replace $customer/phone with <phone>…</phone> • rename in order to rename certain nodes (element or attributes) • rename $customer/name as ”fullname” • move in order to move some XML nodes to a different location in the XML tree, while still preserving the internal structure and the node identifiers • move $customer/telephone after $customer/city

29. Service Invocation Statements • Web services are invoked independently of the specific way they are implemented by a SOAP message • The message is sent exactly once and in a best effort way • A result is returned as a SOAP message • Synchronous invocation: • <expression> --> <uri> [::<operation>] [--> <variable>] • Asynchronous invocation: • <expression> ==> <uri> [::<operation>] [==><operation>]

30. if (<booleanExpression>) then <statement> endif else <statement> endelse switch if (<booleanExpression>) then <statement> end if (<booleanExpression>) then <statement> end ! ! . . . [default <statement> end] endswitch Conditional statements • IF and SWITCH statements similar to Java

31. Iteration statements • WHILE and DO-WHILE loops are the same as in Java while (<booleanExpression>) do <statement> endwhile • FOR-LET-WHERE-DO loop corresponds to FLWOR expression in XQuery for <variable> in <expression> let <variable> in <expression> where <booleanExpression> do <statement>

32. Exception handling statements • Errors are reported by throwing exceptions • Syntax: • throw <expression> • Expression can be any kind of XQuery expression • An unhandled exception is returned as a message to the caller of the service (instead of the value of the $output variable) • In order to handle exceptions locally XL adopts the try-catch syntax from Java.

33. Statement combinators • Sequence – statement1 is executed before statement2 • <statement1> ; <statement2> • Failure – if statement1 fails, statement2 is executed • <statement1> ? <statement2> • Choice – Either statement1 or statement2 is executed, but not both • <statement1> | <statement2>

34. Statement combinators (cont.) • Parallel – statement1 and statement2 are executed in parallel • <statement1> || <statement2> • Dataflow – if there is a data dependency, the dependent statement is executed last. If there are not dependencies, the statements are executed in any order (or in parallel). If there is a cyclic dependency the combination is illegal. • <statement1> & <statement2>

35. Agenda • Web services development - state of the art • Requirements for a Web Service Programming Language • Web services in XL • XL statements and statement combinators • Implementation • Example

36. Design goals • Distribution • It should be possible to distribute the execution of a Web service over different computers • Platform independence • It should be possible to execute XL-Web services on different platforms and operating systems • Expressions • An XML-Query engine has to be integrated into the XL runtime • Optimization • A basis for powerful and automatic tuning and optimization should be provided

37. Implementation highlights • XL virtual machine exploits implicit or explicit given parallelism. Synchronization points are set automatically • XL virtual machine handles the context of conversations • XL virtual machine handles variables which do not fit in main memory (e.g., the customer database of an online shop). • XL virtual machine handles persistence and supports transactions

38. Statement graphs • Each node represents a statement • Nodes are connected by directed edges (combinators) optionally attributed by a (boolean) variable or a negation of a variable • If at the end of a node execution an outgoing combinator evaluates to TRUE the succeeding node is also executed • The statement graph does not imply any ordering between succeeding statements

39. let $a := {0} let $_p := ($a=0) ($_p) not ($_p) let $a := {1, 3, 5} let $a := {2, 4, 6} let $b := $ a let $a := {1, 2, 3} let $num := count ($a) let $sum := sum($a) sync let $avg := $sum/$num Statement graphs (cont.) • if-then-else let $a := {0}; i f ($a = 0) then let $a := {1, 3, 5} endif else let $a := {2, 4, 6} endelse let $b := $ a; • parallel statements let $a := {1, 2, 3}; let $num := count ($a) || let $sum := sum($a); let $avg := $sum/$num;

40. Architecture of XL runtime

41. Message types • XL communicates by sending XML SOAP messages over HTTP • operation call - the message handler initiates an appropriate interpreter which in turn initializes the input variable $input, starts the execution, waits for the execution to finish and returns the result – either an error message or the content of the variable $output. • debug messages - a special debugging interpreter provides an interface for a remote debugging client

42. Context information • Context information is contained in control blocks organized in a tree structure • At the execution-time each statement can access all variables which are contained in its control block or any parent control block. • The root control block holds all global variables • The auxiliary statements BeginBlock and EndBlock in the statement graph indicate where a new control block has to be used or an old one can be disposed

43. Handling conversations • XL runtime maps URIs contained in SOAP messages to existing control-blocks • If an operation call occurs as part of an existing conversation the associated control-block is used as parent control block for this execution • If the service is not part of the conversation and the adequate conversation pattern is specified a new context is created and added to the URI context map

44. Interpreter • Each time an operation is to be executed the interpreter • Fetches the associated statement graph for this operation • Creates a new context representation which inherits the global and conversation variables • Generates a dynamic binding between statement graph and context representation

45. XML Query Expressions • XML Query expressions are executed by an external query engine for XQuery • XL treats expressions as simple character-strings which are passed to the query engine along with the context information • Pros • Can be easily replaced by another implementation. • Cons • Degradation of performance and lose of optimization possibilities

46. Dimensions of the execution model

47. Agenda • Web services development - state of the art • Requirements for a Web Service Programming Language • Web services in XL • XL statements and statement combinators • Implementation • Example

48. Highlights • Partial implementation of an online shop • A client Web Service starts a conversation by calling the method login which has the conversation pattern requiredNew and starts therefore a new conversation in any case • conversationpattern mandatory in the service header defines that for example the operation buy can only be called if a conversation has been started already by calling login • For each new conversation a new instance of the context variable $customer is created and initialized by default. In every further operation call in the conversation the same content is bound to the variable $customer

49. Global definitions service HTTP://www.shop.com namespace xf = ”http://www.w3.org/2001/08/xquery-operators” !! Web service internal data let $warehouseDB := <warehouseDB />; let $customerDB := <customerDB />; context let $customer := <customer />; !! entire Web service activity is monitored history ; !! default operation is unknownOP defaultoperation unknownOP; !! any operation call must belong to a conversation conversationpattern mandatory; !! a conversation cannot last more than 10 days conversationtimeout xf:duration(”P10D”) logout; !! if an error occurs , logout is called on error invoke logout;

50. Operation login operation login !! input variable is valid precondition $input validates as customerLoginSchema; !! customer needs a valid login precondition xf:exists( for $c in $input/customer, $db in $customerDB/customer where $c/id eq $db/id and $c/passwd eq $db/passwd return $c) ; !! a new conversation is started conversationpattern requiredNew; body !! initialize the conversation variable let $customer := <customer> <id>$input/customer/id</id> <orders/> </customer>; !! return value let $output := ’Login successful’ endbody endoperation