Workflow Optimisation Services for e-Science Applications

1. Workflow Optimisation Services for e-Science Applications David W. Walker Cardiff University

2. WOSE Overview • Draws together JISGA and Triana work at CU, with ICENI at IC, and portal expertise at DL. • Topics addressed • Service aggregation and deployment • Runtime discovery and late binding of services • Service discovery and selection from multiple semantically equivalent services

3. Workflow Optimisation • Types of workflow optimisation • Through service selection • Through workflow re-ordering • Through exploitation of parallelism • When is optimisation performed? • At design time (early binding) • Upon submission (intermediate binding) • At runtime (late binding)

4. Service Binding Models • Late binding of abstract service to concrete service instance means: • We use up-to-date information to decide which service to use when there are. multiple semantically equivalent services • We are less likely to try to use a service that is unavailable.

5. Late Binding Case • Search registry for all services that are consistent with abstract service description. • Select optimal service based on current information, e.g, host load, etc. • Execute this service. • Doesn’t take into account time to transfer inputs to the service. • In early and late binding cases we can optimise overall workflow.

6. WOSE Architecture Work at Cardiff has focused on implementing a late binding model for dynamic service discovery, based on a generic service proxy, and service discovery and optimisation services. User Configuration script Web service instance Workflow script Converter ActiveBPEL workflow engine Discovery Service Proxy Optimization Service History database Registry services (such as UDDI) Performance Monitor Service

7. Service Discovery Issues • Service discovery and optimisation is based on service metadata. • Could store in a database. • Could obtain by interrogating service.

8. Optimisation by Re-Ordering • Work at Imperial has looked at static optimisation • Optimise the runtime execution of workflow before it is executed • Achieves the goal through: • Re-ordering of components • Addition of components • Substitution of components • Pruning of the workflow • Performance and workflow aware Scheduling • Runtime Optimisation • through monitoring, check-pointing and migration

9. Matrix Gen Matrix Gen multiply Matrix Gen multiply Vector Gen multiply Matrix Gen multiply Vector Gen Component Manipulation • Re-ordering: Workflows (often composed from composite workflows) may contain non-optimal ordering of components • Use re-ordering to improve performance

10. Component Addition • Addition: For a component requiring a specific format of data as input, a transformer component could be added to achieve the desired format. • Allows more optimal components to be used together Input required in MPS format Output in LP format C 1 LP to MPS C 2

11. Component Substitution • Substitution: • A Jacobi Iteration linear solver replaced by Conjugate Gradient linear solver according to the output of the Discretizer (FEM) • Based on observing the meta-data associated with previous components A (sparse and diagonally dominant) JI linear solver FEM b

12. a b d c  f e  h g Not needed Not needed Pruning • Workflow Pruning: • Workflows may contain unused components. Especially when composed from other sub-workflows • Remove redundant components

13. Work Done at Daresbury • Daresbury started working with evaluating different BPEL engines to have proper understanding of BPEL specification i.e. ActiveBPEL, Oracle BPEL engine initially Collaxa BPEL engine. • Most projects at CCLRC are portal based, so we studied the relevance of Portal based workflow monitoring and controlling solution. • Web Services for Remote Portlets (WSRP) is an emerging specification, with limited support - seems to bridge Portal and Web Services and a lot of work has been done, with demo prototype. • WSRF is extension of existing Web Services and provides extra layer of functionality without changing the Web Service implementation. • WSRF extension for Web Service starts from modification of existing WSDL file of specific Web Service. • WSRF extension of WSDL is not straight forward and needs a thorough understanding of WSDL and WSRF. • Daresbury has built working prototype for testing Web Services based on WSDL.

14. Future Work • Complete WOSE prototype with dynamic scheduling. • Compare dynamic and static approaches at Cardiff and Imperial. • Improve Discovery Service mechanisms. • Investigate different approaches to optimisation.