June 2013

1. Mediator GE Webinar June 2013

2. Agenda • Mediator GE Overview • Mediator GE: Mediation Engine (TI) • Mediator GE: Dynamic Mediation API (Thales) • Demo: Event Brokering • Questions & Answers 2

3. Overview FI-WARE platform should be able to support services exposed through different protocols and technologies and enable the creation of new composite services. The mediator shall provide the glue between the service layer and the composition layer in order to enhance the composition capabilities of the compositor GEs.

4. Capabilities • The main capability of the Mediator GE is to build an interoperability layer providing implementation for various enterprise integration patterns and data and protocol transformations • It is a kind of broker between the service consumer and the service provider • Particularly it provides Data Mediation (Transformation), Protocol Mediation and routing capabilities using various mechanisms: • Using a library of built-in transformers in case of well-known formats • Using templates for more flexible data/protocol transformation capabilities • Using XML DSL (Domain Specific Language) in order to fulfill more complex protocol/data transformation requirements or complex integration patterns (split/aggregations) • Using custom code (various programming languages are supported) to reach maximum flexibility in most complex scenarios 4

5. Role in FIWARE • Basically the Mediator GE could play the role of centralizing and abstracting the access towards the heterogeneity of different devices or target applications exposing different capabilities or the same capabilities through different interfaces and data models. • In this scenario the mediator GE can be configured to implement different functionalities: • exposing a uniform interface and data model towards a plethora of heterogeneous devices and sensors • routing incoming messages towards the right target device or service based on the content of the message or other criteria • performing the role of splitting a coarse grained task into many specific items that shall be executed by different actors (device or applications), dispatching the split tasks and aggregating all results in order to return the whole result to the caller. • performing the role of event broker. A client application interested in events from a particular set of devices or other applications can subscribe to them using a particular service exposed by the mediator where all relevant event sources are configured. • providing an API for automatically solving particular mediation issues (based on semantic meta-data) 5

6. Architecture The mediator offers a set of available mediation tasks: the set of mediation capabilities that can be used via the mediator. The mediator allows users to create and manage their mediation services: a mediation service is a virtual proxy towards a web service that executes a mediation task between the caller and the target service. The mediation task must be chosen from the set of available mediation tasks. The mediator GE is used mainly by the Compositor Editor and Compositor Engine GEs within the FI-WARE platform. It provides a layer of virtual proxies to be used by the compositor instead of the target services in order to make the compositor able to support various kind of target services. Besides the FI-WARE platform also Future Internet applications or composite services on top of the FI-WARE platform can use the mediator as a service for their own purpose.

7. Mediation Engine (TI) • The reference implementation of GE-Mediator provided by Telecom Italia is based on two open source frameworks: • WSO2 ESB (http://wso2.com/products/enterprise-service-bus) • Apache Camel (http://camel.apache.org) • Particularly the value added by Telecom Italia is related to: • A custom mediation module based on Apache Camel in order to enhance the mediation capabilities of WSO2 ESB. • Basically it is possible to add Camel route to a running instance of WSO2 ESB through a customization of the WSO2 ESB Web Admin GUI • An enhanced per user logging capability • Basically it provides data related the who is using what inside the ESB • The integration with a monitoring and disaster recovery tool based on JMX

8. WSO2 Enterprise Bus Overview The mediation built-in capabilities are provided through: Service Virtualization Message Mediation The main mediation capabilities are: Data Transformation Protocol mediation Load Balancing and Failover Manipulating SOAP Headers (WS-Security) Iterate/clone messages Policy enforcement

9. How to configure Mediators • WSO2 ESB provide a built in Mediator Catalog • Usually a mediator is configured using XML • WSO2 ESB provides a Web GUI to help to configure Mediators without or less XML • ESB mediation framework is extensible. It is possible to write custom mediators using: • Pure Java • Spring Bean • Scripting Languages: javascript, ruby • Examples of Mediator functionalities: • Drop Mediator : Drops a message, stopping further processing of the message • Send Mediator :Send the message out, using information inside the message or according to some external configuration • Script Mediator : Executes a scripting language with the message • Filter Mediator : Filter messages using XPath • XSLT Mediator : Do XSLT transformations on messages

10. Mediators catalog: core mediators

11. Mediators catalog: filter mediators

12. Mediators catalog: transform mediators

13. Mediators catalog: advanced mediators

14. Mediators catalog: extensions

15. Example 1: managing WS-Security <proxy xmlns="http://ws.apache.org/ns/synapse" name="smsService_ws_security" Transports="https" statistics="disable" trace="disable" startOnLoad="true"> <target> <outSequence> <send /> </outSequence> <endpoint> <address uri="http://localhost:9000/services/SmsMessageAdapter" /> </endpoint> </target> <enableSec /> <policy key="conf:/repository/axis2/service-groups/smsService_ws_security/ services/smsService_ws_security/policies/UTOverTransport" /> </proxy>

16. Example 2: Data Transformation SOAP2REST <proxy xmlns="http://ws.apache.org/ns/synapse" name="SoapToRest_Sms" transports="http" statistics="enable" trace="enable" startOnLoad="true"> <target> <inSequence> <property name="Accept" value="application/xml" scope="transport" /> <log level="full" /> <router> <route xmlns:soaptorest="http://telecomitalia.it/ictservice/smsService/soaptorest" expression="$body/soaptorest:sendSms"> <target sequence="anon"> <sequence> <xslt key="gov:/transformations/trasformSOAP2RESTPayload" /> <property name="HTTP_METHOD" value="POST" scope="axis2" type="STRING" /> <log level="full" /> <send> <endpoint> <address uri="http://polaris.cselt.it:9763/SmsServiceRest/rest/sms" format="rest" /> </endpoint> …......................................... </proxy>

17. Example 2: Data Transformation SOAP2REST XSLT <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <!-- Our SoapToRest service, built to expose an existing Rest service as a Soap service, for each method uses as payload the Rest service payload incapsulated in an element whoose name is that we gave to the soap method. Here we remove this incapsulation, to get payload that we have to send to the Rest service. --> <xsl:output method="xml" indent="yes"/> <xsl:template match="/node()/node()"> <xsl:copy> <xsl:copy-of select="./*"/> </xsl:copy> </xsl:template> </xsl:stylesheet>

18. Example 3: Protocol Transformation TCP2HTTP <proxy xmlns="http://ws.apache.org/ns/synapse" name="TCPtoHTTP_mail" transports="tcp" statistics="enable" trace="enable" startOnLoad="true"> <target endpoint="fiwareMailproxy"> <inSequence> <log level="full" /> </inSequence> <outSequence> <send /> </outSequence> </target> </proxy>

19. Example 4: Throttling capability The throttling capability means that it is possible to restrict the usage of a mediation service using different parameters. Range - The IP address range or the domain is restricted from accessing the service. Requests from such clients will be restricted based on the specified values. The restriction can be : Deny or Control Control means that it is possible to set usage limitation based on: - Maximum Request Count (MRC) - Prohibit Time Period (PTP)

20. Example 5: Loadbalancing capability The loadbalancing capability means that it is possible to configure a mediation service to use different service endpoints in order to balance the calls using a round robin algorithm. More complex loadbalancing algorithms currently can be configured only using a custom mediator. <session type="http|simpleClientSession"/>? <loadBalance [policy="roundRobin"]> <endpoint .../>+ </loadBalance>

21. Dynamic Mediation API (DMA)Optional Library provided by Thales

22. DynamicMediation API (DMA) • Main features and assumptions: • Optional library delivered as part of the Mediator GE • Can be used “internally” by the Mediator as part of a created “Mediation Service” • API can also be used directly by third-parties for solving particular mediation issues (as shown in this presentation) • FI-PPP projects: use under the conditions established in the FI-PPP Collaboration Agreement. Distributed under Thales license as a binary-only closed-source library • Useful when syntactic mismatch occurs (at design or runtime) between Service consumer’s needs and Target Service offer. • Additional semantic information (meta-data) + domain ontology is used automatically by the DMA for bridging the syntactic gap between a Service Consumer and a Target service => automatic data conversion

23. SSL - Smart Systems Lab DMA Overview 1 3 Mismatch Ontology DMA features TargetService is_equivalent Service Request • (1) Automatically deal with service (request / target) syntactic mismatch • Use semantic-based characterization of service features (methods, parameters,…) • Supports direct target service invocation through library API • Communication protocol agnostic (SOAP, REST) • (2) Automatic data mediation between services • Data must be described using XSD and semantically-annotated • (3)Use of ontology as business model reference Data Target Data Request DMA Library Automatic conversion 2

24. DMA API typical use cases • Consumer needs to invoke a specific operation on an available target service and already has a structured payload he wants to send • Potential mismatch at data-model level  payload needs to be automatically converted by the DMA • Semantic annotation of consumer-side payload and target service offer mandatory • DMA will find the semantically matching operation in the target service • Consumer needs to invoke a specific operation on an available target service and doesn’t have an imposed structured payload to send • Consumer will provide DMA with a list of parameters with simple data type • Consumer-side data-model undefined payload will be automatically created by the DMA • Semantic annotation of parameters and target service offer still mandatory • DMA will find the semantically matching operation in the target service • In both cases, DMA can be used either to build the right payload (and stop), or to also deal with target service invocation

25. DMA API (1/4) • DMA provides 4 Dynamic Mediation Tasks that are accessible as Java methods or SOAP endpoints: • String dynamicallybuildServicePayloadNoOPWithParameters( • String usdlURI • , String serviceID • , String operationConcept • , List<SemanticParamItem> inputs) • throws MediatorException; • Method description: • Purpose: using a list of unstructured data as input parameters, dynamically building an XML payload adapted to a given service and linked to an operation concept.

26. DMA API (2/4) • String dynamicallybuildServicePayloadNoOPWithPayload( • String usdlURI • , String serviceID • , String operationConcept • , String saxd_payload • , String payload ) • throws MediatorException; • Method description: • Purpose: using a structured payload (with its associated data schema) as input parameter, dynamically building an XML payload adapted to a given service and linked to an operation concept.

27. DMA API (3/4) • String dynamicDataMediationInvocationNoOPWithParameters ( • String usdlURI • , String serviceID • , String operationConcept • , List<SemanticParamItem> inputs • , List<SemanticParamItem> awaitedOutput) • throws MediatorException; • Prototype description: • Purpose: dynamically searches and invokes a service capability based on the given operation concept. The operation of the service, if found, is called with a dynamically built payload based on the list of unstructured data passed as input parameters. • Note: the expected data format of the result(s) is defined by the last input parameter awaitedOutput. The result is then adapted before being returned.

28. DMA API (4/4) • String dynamicDataMediationInvocationNoOPWithPayload ( • String saxsd_payload • , String serviceID • , String payload • , String wanted_response • , String operationConcept) • throws MediatorException; • Prototype description: • Purpose: dynamically searches and invokes a service capability based on the given operation concept. The operation of the service, if found, is executed using the input payload, first adapted to the service. • Note: the data format of both input payload and expected result(s) is defined by the input parameter saxd_payload. The result is then adapted before being returned.

29. Use case: simple calculator exemple – Case 1 • A: Consumer needs to invoke a specific operation on an available target service and doesn’t have an imposed structured payload to send • IMediatormediator = MediatorEntry.getInstance();; • String operationConcept = "http://www.example.com/Calculator.owl#Addition"; • SemanticParamItem param1 = new SemanticParamItem(); • param1.setConcept("http://www.example.com/Calculator.owl#OperandeGauche"); • param1.setValue("12"); • SemanticParamItem param2 = new SemanticParamItem(); • param2.setConcept("http://www.example.com/Calculator.owl#OperandeDroite"); • param2.setValue("5"); • SemanticParamItem param3 = new SemanticParamItem(); • param3.setConcept("http://www.example.com/Calculator.owl#Somme"); • List<SemanticParamItem> result = null; • result = mediator.dynamicDataMediationInvocationNoOPWithParameters(usdlURI, serviceID, operationConcept, inputs, outputs); • for (SemanticParamItem item : result){ • assertTrue(item.getValue().equals("17")); • System.out.println(item.getConcept()); • System.out.println(item.getValue()); • }

30. Use case: simple calculator exemple – Case 2 • B: Consumer needs to invoke a specific operation on an available target service and already has a structured payload he wants to send • IMediatormediator = MediatorEntry.getInstance();; • String wanted_response = "AdditionResponse"; • String payload = Util.getLocalPathFromResource("fiware/payload.xml"); • SemanticParamItem param3 = new SemanticParamItem(); • param3.setConcept("http://www.example.com/Calculator.owl#Somme"); • String saxsd_payload = Util.getLocalPathFromResource("fiware/saxsd_inputs_outputs.xml"); • System.out.println(saxsd_payload); • String operationConcept = "http://www.example.com/Calculator.owl#Addition"; • String result = null; • result = mediator.dynamicDataMediationInvocationNoOPWithPayload( • saxsd_payload, usdlURI, serviceID, payload, wanted_response, operationConcept); • assertTrue(result != null); • assertTrue(result.contains("15"));

31. Use case: simple calculator exemple – Case 2 • Hypothesis of work: • Payload: <Addition> <operandeg>12</operandeg> <operanded>3</operanded> </Addition> • Annotated schema for the payload and/or request/response using SAWSDL: <xs:elementname="Addition"> <xs:complexType> <xs:sequence> <xs:elementname="operandeg" sawsdl:modelReference="http://www.example.com/Calculator.owl#OperandeGauche" type="xs:string"/> <xs:elementname="operanded" sawsdl:modelReference="http://www.example.com/Calculator.owl#OperandeDroite" type="xs:string"/> </xs:sequence> </xs:complexType> </xs:element> <xs:elementname="AdditionResponse"> <xs:complexType> <xs:sequence> <xs:elementname="resultat" sawsdl:modelReference="http://www.example.com/Calculator.owl#Somme" type="xs:string"/> </xs:sequence> </xs:complexType> </xs:element> </xs:schema>

32. DEMO

33. Example Scenario: Task Dispatcher and Event Brokering 33

34. Example Scenario: Task Dispatcher and Event Brokering • In this scenario the mediator can play the roles of task dispatcher using the split/aggregator pattern and event broker. • As a task dispatcher the mediator receive a coarse grained task from the task manager and , based on some criteria related to the content of the task, split it into specific items and dispatch them to the right target (human or application) • As an event broker the mediator exposes a service in order to receive events from one or more event sources and allows applications to subscribe to an event topic in order to get related events when they are received by the broker. In this way the event broker collect events from devices, optionally filter them and send them to the current subscribers. • Moreover, in this demo scenario the mediator play also the role of centralizing the calls to third party services used by the client (subscribed) applications (i.e. email and sms services). • In order to show the load-balancing capability of the mediator GE, the email service mediation can be configured to use two different endpoints balanced through a simple round robin algorithm 34

35. References • Fiware Apps Wiki Architecture • https://forge.fi-ware.eu/plugins/mediawiki/wiki/fi-ware-private/index.php/FIWARE.OpenSpecification.Apps.Mediator • Fiware Apps Wik Open Spec: • https://forge.fi-ware.eu/plugins/mediawiki/wiki/fi-ware-private/index.php/FIWARE.OpenSpecification.Apps.Mediator • WSO2 ESB docs site: http://wso2.com/products/enterprise-service-bus