Inter- A pplication C ontrol T hrough G lobal S tates M onitoring O n a Grid

1. Inter-Application Control Through Global States Monitoring On a Grid M. Tudruj, J. Borkowski, D. Kopanski Polish-Japanese Institute of Information Technology, Koszykowa 86, 02-008 Warsaw, Poland

2. Monitoring global states control Processes can communicate with a number of Synchronizers. Synchronizers learn state information from processes and send back control information. S a synchronizers processes state information P1 qArrows represent reliable, asynchronouscommunication channels P3 P2 P4 S b

3. Monitoring consistent global states • There is no global clock, no shared memory • Synchronizer must be able to order properly incoming events to build Strongly Consistent Global States (SCGS) • SCGS is a combination of process local states, one state from each process, such that the local states are pairwise concurrent. E.g. <s1,t1> is a SCGS, <s1,t2> is not. s1 e1 e2 P1 t1 t2 P2 f1 f2 m2 m1 sync

4. Strongly Consistent Global States • Events must have timestamps to be able to order messages correctly. Logical vector clocks or real time intervals based on roughly synchronized local clockscan be used • If process local clocks are synchronized with a known accuracy, then real time interval timestamps can be used to identify SCGS

5. Computation activation and cancellation caused by predicate evaluation

6. GRADE system • A complete graphical programming environment for developing message passing applications designed at Parallel and Distributed Systems Laboratory of the SZTAKI Institute of Hungarian Academy of Sciences • Application level specifies processes • andtheir interconnections • Process level defines control flow diagram of a process • Text level is used to enter sequential C code into elements of a flow diagram

7. GRADE extension – PS-GRADEState information monitoring local state info transfer channels signal transfer channels standard message passing channels

8. PS-GRADE - synchronizer

9. PS-GRADE– synchronizer reception of state variables condition send signal condition window control flow window

10. PS-GRADE– Process -control flow window Start signal - sensitive region "watching - signal" Start signal - insensitive region Resume interrupted computations Send state Cancel computations End signal - insensitive region End signal - sensitive region "endwatching - signal"

11. PS-GRADE– synchronizer hierarchy

12. The principles of Grid application control • Control of Grid application by: • Data control flow (similarly to P-GRADE Workflow implemented by SZTAKI) , based on input and output files for cluster application • Grid Synchronizer : • Collects information (vector of state) about application state • Detects SCGS or OGS • Computes conditions • Sends signals to the application

13. A Grid-level synchronizer inserted into a workflow graph Synch1 3 4 3 3 2 1 2 2 5 6 2 2 A1 1 1 1 A2 A3 4 5 4 5 1 3 1 A4 A5 3 4 1 A6

14. A Grid-level synchronizer and an application Application A2 GRID Synchronizer Application A3 Application A5 Application A4

15. Grid PS-GRADE environment general structure

16. Simple workflow. A selected application starts executing after a set of selected applications is completed. Example: complicated scientific computations performed layer by layer in different computer networks. Organizing Grid-level program execution control The following co-operation schemes included into the proposed Grid environment will be discussed:

17. Alternative workflow.One of several applications is selected for execution depending on the results (state) of former applications. Example: one of two available program packages is run depending on computation results performed so far. Partial canceling of workflow: Applications that become superfluous from the point of view of the general purpose of computation are stopped. Example: The exhaustive parallel search on a Grid for the optimal solution in a solution space is stopped or restricted when the search provides a satisfactory solution.

18. Supporting workflow: A set of currently executed applications require activation of auxiliary applications, which will provide useful results. Example: In a coarse grain simulation of a system of moving objects a collision that appears, stimulates a change in the Grid application configuration. An application which models the collision in a detailed way (with a fine granularity of events) is activated. After detailed simulation of the collision the coarse grain of the simulation process is restored. Workflow coupling: A common (global) status of many applications is monitored and control directives are distributed to particular applications of needed. Applications compute parameters that are subject to mutual exchange. Some parameters in meta-level applications are updated with the use of results of some auxiliary computations

19. Example – A Grid TSP application structure

20. The TSP application Synch1 conditions: DataRequest, MinDist, FinishCond (left to right)

21. The TSP application B&B part: condition window B&B part: communication diagram Heuristic search Application structure search process application structure

22. Conclusions • The paper has presented how the synchronization-based parallel application control can be extended and ported onto the GRID level. With the use of the proposed method we can create an advanced control of many applications running in the GRID environment. Inter–application coordination between programs, which are executed on different GRID sites, is supported. • We identify five types of Grid-level program execution control • The presented example shows that the new programming environment provides convenient means for designing complicated Grid applications control. Being on the Grid, we can extend the time consuming parts of the applications and run it on any available clusters during the middle stage of the algorithm. The best results from the heuristic part of the application obtained in a shorter time than by the B&B computations can support faster finding of the exact solution.