SchedScripter : Workflows for Grid-based Human Resources Scheduling Applications

1. SchedScripter: Workflows for Grid-based Human Resources Scheduling Applications George Goulas, Christos Gogos, Panayiotis Alefragis, Efthymios Housos Computer Systems Laboratory, Electrical & Computer Engineering Dept., University of Patras

2. Scheduling Human Resources • Scheduling Problem: Assign a set of ACTIVITIES to a set of RESOURCES given a set of DEMANDS and a set of CONSTRAINTS to produce a set of PRODUCTS(Goods or services) • Resources include humans (opposed to computer process scheduling) • Upfront knowledge of task details • NP-hard/NP-Complete problem, Combinatorial Explosion • Problem Domains: Transportation, bus/driver scheduling, airline crew pairing, course /exams timetabling, Production, job-shop scheduling, ..

3. SchedScripter Support the creation of • Distributed Human Resources Scheduling Applications • On a grid infrastructure like the EGEE • Using a distributed scheduling application model • Supports dynamic component composition guided by the solution process and progress • Using open, widely used protocols and implementations for communications and coordination

4. Distributed Scheduling App Pre Processing Legality / Regulation Evaluation Sub Problem Selection Strategy Selector Alternative Schedule Generators Cost Calculation Feedback Finish Subproblem Optimization / Solution Selection Finish Post Processing Problem Solver Different Problem Solvers Scheduling Coordinator

5. SchedScripter Features • Scheduling Application Templates based on Distributed Design Patterns • Application Coordination using Application Templates or BPEL or ad-hoc Web Services usage • Submit web services container as grid jobs on the worker nodes

6. Application Structure WORKER NODE WORKER NODE WORKER NODE WORKER NODE WORKER NODE WORKER NODE WORKER NODE Application Code Application Code Application Code Application Code Application Code Application Code Application Code Grid Application Coordination Bulletin Workflow Mgmt WN Matcher Registry

7. Application Coordination 1/3 • Master / Worker Coordination • Java implementation of the master/worker distributed design pattern • For developers not familiar with WS-*/BPEL Bulletin Coordinator Improver Host Improver Host Improver Improver Improver Improver

8. Application Coordination 2/3 • Swarm Pattern (peer-to-peer) • Java implementation of p2p design pattern • “Master” receives messages and collects final result when ready • For developers not familiar with WS-*/BPEL

9. Application Coordination 3/3 • BPEL based Coordination • WN Matcher • Hierarchical Pattern proposed • Master BPEL invoking worker BPEL processes • Worker BPEL process on single WN • Web Services ad-hoc integration • Toolkit to assign workload

10. Master Node User Applications and User Services BPEL Composite Services Workflow Mgmt Swarm Master Scheduling Application Coordinator It can optionally live inside Bulletin or it can use its web services interface Registry WS WN Matcher WS Bulletin WS Master File Management WS Port Forwarding WS WS-Security using Grid X509 Certificates Simple WS Container Master Node

11. Worker Node File Mgmt WS WN Mgmt WS Job Host WS User Defined WS Job Mgmt WS Improver Improver Improver Improver Improver Improver MesageBox Improver Host • WS-Security using Grid X509 Certificates Simple WS Container Worker Node

12. Exams Timetabling Case Study* • Based on an already available examination timetabling application from our team • Initial Tests • Independent Improvers Parameter Sweep • Tournament between Improvers, 50% chance to use best available solution or one of the 2-10 top, synchronous • asynchronous • Modifiable parameters were the Simulated Annealing meta-heuristic parameters, used in the application *Gogos C., Goulas G., Alefragis P. and Housos E. Pursuit of Better Results for the Examination Timetabling Problem Using Grid Resources, 2009, IEEE Symposium on Computational Intelligence in Scheduling (CI-Sched), Nashville, Tennessee, USA, 30 Mar-2 Apr 2009 (to appear).

13. Results *Best of 100 sequential runs, ~10 min runtime ** Same as sequential, more independent parallel runs *** 1200 sec total runtime, 15 synchronous steps, 30 WNs

14. Case Study Experience • Easy to use environment • 1 week effort to adapt application to SchedScripter • Application refactoring to break it into components • Create initial solution • Improve using parameters, initial solution and specific configuration of meta-heuristics • Restart with different initial solution but keep problem in memory (problem loading up to a minute long process) • Significant solution quality improvements with grid

15. Solution Quality (Cost function) Progress Sample

16. Conclusions • Easy to use framework • Java templates for non- BPEL/WS-* -savvy developer • Developer provides “strategy” or “coordinator” and “improver” classes • EGEE compatible but not limited, runs even on cluster • BPEL and web-services support for application coordination • BPEL and web services abstractions and “details” could be hard to understand for developers ! • EPRs, namespaces, deploy.xml, WSDL files, BPEL engine artifacts, packaging, deployment, partner links, ..

17. Future Work • Examination timetabling using scatter search is almost ready • Develop /adapt various scheduling applications from different scheduling domains • Seeking non-scheduling scientific computing applications • Adapting SchedScripter for GT4/WSRF is interesting but EGEE middleware compliance is a major choice • largest infrastructure we could have access to at the moment !