Workflow Optimisation Services for e-Science Applications

1. Workflow Optimisation Services for e-Science Applications David W. Walker Cardiff University

2. WOSE Overview • Draws together JISGA and Triana work at Cardiff University, with ICENI at Imperial College, and portal expertise at Daresbury Laboratory. • Topics addressed • Service aggregation and deployment • Runtime discovery and late binding of services • Service discovery and selection from multiple semantically equivalent services

3. Workflow Optimisation • Types of workflow optimisation • Through service selection • Through workflow re-ordering • Through exploitation of parallelism • When is optimisation performed? • At design time (early binding) • Upon submission (intermediate binding) • At runtime (late binding)

4. Service Binding Models • Late binding of abstract service to concrete service instance means: • We use up-to-date information to decide which service to use when there are. multiple semantically equivalent services • We are less likely to try to use a service that is unavailable.

5. Late Binding Case • Search registry for all services that are consistent with abstract service description. • Select optimal service based on current information, e.g, host load, etc. • Execute this service. • Doesn’t take into account time to transfer inputs to the service. • In early and late binding cases we can optimise overall workflow.

6. User Configuration script Web service instance Workflow script Converter ActiveBPEL workflow engine Discovery Service Proxy Optimization Service Registry services (such as UDDI) WOSE Architecture Work at Cardiff has focused on implementing a late binding model for dynamic service discovery, based on a generic service proxy, and service discovery and optimisation services.

7. Workflow script Optimisation service XSLT converter 6. Selected service 5. List of services Workflow deploy WOSE client Workflow engine Web service Web service proxy Discovery service 2A. Direct invocation 1. Request 3A. Direct result 3. Service query 2. Dynamic invocation through proxy 7. Invoke service 4. List of services 8. Result 10. Result 9. Result through proxy WOSE Sequence Diagram Optimisation criteria

8. Service Selection Issues • Service discovery and selection is based on service metadata. • Could store in a database. • Could obtain by interrogating service. • Could select service based on previous history. • Could select service based on current (or recent) data.

9. Optimisation by Re-Ordering • Work at Imperial has looked at static optimisation • Optimise the runtime execution of workflow before it is executed • Achieves the goal through: • Re-ordering of components • Addition of components • Substitution of components • Pruning of the workflow • Performance and workflow aware Scheduling • Runtime Optimisation • through monitoring, check-pointing and migration

10. Matrix Gen Matrix Gen multiply Matrix Gen multiply Vector Gen multiply Matrix Gen multiply Vector Gen Component Manipulation • Re-ordering: Workflows (often composed from composite workflows) may contain non-optimal ordering of components • Use re-ordering to improve performance

11. Component Addition • Addition: For a component requiring a specific format of data as input, a transformer component could be added to achieve the desired format. • Allows more optimal components to be used together Input required in MPS format Output in LP format C 1 LP to MPS C 2

12. Component Substitution • Substitution: • A Jacobi Iteration linear solver replaced by Conjugate Gradient linear solver according to the output of the Discretizer (FEM) • Based on observing the meta-data associated with previous components A (sparse and diagonally dominant) JI linear solver FEM b

13. a b d c  f e  h g Not needed Not needed Pruning • Workflow Pruning: • Workflows may contain unused components. Especially when composed from other sub-workflows • Remove redundant components

14. Future Work • Complete WOSE prototype with dynamic scheduling. • Compare dynamic and static approaches at Cardiff and Imperial. • Improve Discovery Service mechanisms. • Investigate different approaches to optimisation.