Holding slide prior to starting show

1. Holding slide prior to starting show

2. Portals for Workflow Composition and Service Management David W. Walker School of Computer Science Cardiff University http://www.cs.cf.ac.uk/User/David.W.Walker/

3. System Portals • We need portals to be able to handle tasks such as: : • Problem specification, resource discovery and job composition • Service deployment and lifecycle management • Submission, execution, monitoring, and (maybe) interaction • These tasks are quite generic, rather than application-specific. • Portal can behave rather like a “Grid Shell”

4. Web Services • Everything is a (Web/Grid) service. • This includes: • Computation routines • Access to files and databases • Components of the Grid infrastructure, such as workflow enactment engines, resource monitors, etc.

5. A Common Approach to Workflow • Visual service composition. • Service interfaces and other metadata expressed in an XML-based service description document. • Services registered with, and discovered through, a registry.

6. Visual Service Composition Portlet • Applications and high-level services are composed from interacting services. • Composition can be done graphically using a “drag and drop” style of interface to draw a workflow. • In a workflow graph the nodes represent services and the edges represent data flow or control flow. • Need special nodes to represent loop and conditional constructs. • Need query mechanism to discover services

7. Workflow Description • A workflow can be represented with an XML-based workflow description language such as WSFL, SWFL, SCUFL, BPEL4WS, etc. • A workflow description document can be passed to a workflow enactment engine, which provides an execution environment for the workflow.

8. Two Hard Problems • Semantic specification of applications • Scheduling of workflow nodes on distributed resources. • Early binding model: bind to specific service/platform at composition time (“validation”). • Intermediate binding model: bind at “compile” time (when converting from XML to executable form). • Late binding model: bind dynamically at runtime. • Later binding allows the use of more up-to-date information to make scheduling decisions.

9. Mobility • Should (some) services be mobile and/or clonable? • Allows the automatic creation of new services that are heavily used. • May be important for autonomic grids. • Generalises current data migration and replication concepts.

10. Communication and Notification • Another important issue is how to carry out the data flow and control flow in a workflow. • Need an asynchronous messaging system to support interoperability and to handle connection problems.

11. Portlets • Portlet technology endorsed by GCE working group of GGF, e.g., JetSpeed • Web services have user-facing ports that provide content to users. • Portlets can be regarded as interfaces to that content, specified via an extension to WSDL. • Portals can be built from collections of portlets, e.g., as a set of panels for job submission, job status, visualisation, etc. • Portlets provide as component model for user interfaces in the same way that web services provide a component model to middleware.

12. Grid-Oriented Portal Work at Cardiff • Visual Service Composition Environment (VSCE) for building applications based on service workflow. • Service Workflow Language (SWFL) for describing composite applications. • Tools for transforming between workflow graph  SWFL  executable code. • Jini Service-oriented Grid Architecture (JISGA). An execution environment taking SWFL as input. • Java-C Automatic Wrapper (JACAW). Wraps C code as Java using JNI for deployment as Web service in JISGA. • Grid-Service-in-a-Box (GSiB). Supports service creation, deployment, management, and monitoring.

13. Motivation and Targets • Easy-to-use tools are needed to allow scientists to take full advantage of the new Grid without them needing detailed knowledge of the underlying infrastructure. • GSiB project is developing a visual problem-solving environment/portal for all service users (clients and providers).

14. Service Deployment • A web service • Its public interfaces and bindings are described in an XML document. • It is registered in a service registry to allow service discovery and query. • Usually additional code or wrapper is needed to make existing software available as a Web service – a lot of repeated work. • Legacy problems exists.

15. Service Deployment • JACAW (Java-C Automatic Wrapper) automatically generates the JNI wrapping code for a legacy code. • Axis Java2WSDL utility generates WSDL document from the Java code. • UDDI registry server. • UDDI4J provides a Java API for interacting with a UDDI registry.

16. Service Deployment Legacy code JACAW WSDL Document JAVA2WSDL JNI Wrapper UDDI Registry UDDI4J

17. Service Composition • Graphical symbols for constructing a flow model. Data input port Normal and assignment activity Data output port Control input port Control output port Control activity Data link Control link

18. An Example retA = activityA(..); for( i=0; i<100; i++){ activityB(retA, i); } retC = activityC(); A retA For NO retA YES i B C

19. Service Workflow Language (SWFL) <activity name="for"> <for setParallel="no"> <input message=“ForInput”/> <expression><![CDATA[index=0;index<100;index++]]></expression> </for> </activity> <dataLink name=“A_For” source=“A” target=“for”> <swflMap sourceMessage=“AOutput” targetMessage=“ForInput”> <part> <sourcePart name=“retA”/><targetPart name=“retA”/> </part> </swflMap> </dataLink> To be continued

20. Service Workflow Language (SWFL) <dataLink name=“For_B” source=“for” target=“B”> <swflMap sourceMessage=“ForOutput” targetMessage=“BInput”> <part> <sourcePart name=“retA”/><targetPart name=“retA”/> <sourcePart name=“i”/><targetPart name=“i”/> </part> </swflMap> </dataLink <controlLink name=“A_For” source=“A” target=“for” /> <controlLink name=“For_B” source=“for” target=“B” controlPort=“YES”/> <controlLink name=“For_C” source=“for” target=“C” controlPort=“NO”/> <controlLink name=“B_For” source=“B” target=“for” />

21. Service Workflow Language (SWFL) • SWFL is XML based for describing interacting Web services. • Extends Web Service Flow Language (WSFL) by • supporting the application of all the conditional and loop control constructs of the Java language to the composition of Web services. • allowing more general data mappings.

22. Another Picture swfl Job Workflow Engine Drawing Screen Create SWFL Validator Execution Environment Visual Service Composition Environment

23. SWFL2Java • It converts the description of a job in SWFL into executable Java code. SWFL Description Document Java Executable Code Dataflow Graph ControlFlow Graph FlowModel

24. Visual Service Composition Environment • Collaborative code development environment using a visual programming tool for seamlessly integrating services into higher-level services or complete applications. • Applications are created by plugging together services to form a workflow graph. Service metadata is used to check if interfaces are compatible. • Different developers can place their components in a shared repository, e.g, UDDI or Jini lookup service. • Output is in the form of SWFL.

25. JISGA • JISGA consists of two main parts: • A WorkflowEngine service • A JobProcessor service • Grid application is submitted to a WorkflowEngine service as SWFL. • Sequential jobs are handled directly by the WorkflowEngine service. • Parallel jobs involve multiple JobProcessor services.

26. What the Workflow Engine Is • Processes service-based applications described in an XML-based workflow language. • Determines the order of execution and generates the harness code. • Executes the code which includes discovering services, invoking services, receiving results and sending results to clients.

27. SWFL2Java <?xml…> <JFlowModel …………… …………. ………. …. Intermediate FlowModel object SWFL2Graph Graph2Java Java Executable Code SWFL description What the Workflow Engine Does • Converts a SWFL description of a composite service-based job into an executable Java code, and executes it. To be continued…

28. Future Research • Further development of GSiB for creating, deploying, managing and monitoring Grid applications. • More experiments in integration of Jini services and Web services in JISGA. • Demo applications to illustrate use of JISGA on UK e-Science Grid. • Integration of better scheduling in JISGA.

29. Final Comments • Service-oriented approach to Grid computing is generally agreed on • But there are many options for how such Grids might be build (GT3, Jini, JXTA, etc) • Ability to interoperate between different Grids is important. • Need to get more experience with actually using services for e-Science.