SoRTGrid: A Framework compliant with Soft Real Time requirements

1. SoRTGrid: A Framework compliant with Soft Real Time requirements A. Merlo1, V. Gianuzzi1, A. Corana2, A. Clematis3, D. D’Agostino3, A Quarati3 1 DISI – Università di Genova, Italy. 2 IEIIT-CNR, Genova, Italy 3 IMATI-CNR, Genova, Italy.

2. Agenda • Grid, Qos and Soft Real Time • QoS over the Grid • Grid and Time Constraints • Presenting SoRTGrid • Architecture • “Physiology” • Discovery phase • Benefits of SoRTGrid • Conclusions and future works

3. QoS over Grid • Grid was born as a paradigm for sharing huge amount of resources for the execution of large batch job. • Requests of huge amount of resources to use for large periods of time • Sufficient support for discovery and selection of resources is provided by metascheduler systems that do not provide guarantees on: • Length of discovery phase • Success in finding and select resource • However, since the first dawn of the paradigm, the demand of QoS for execution of different classes of applications has been noticed.

4. QoS over Grid /2 • QoS: “harder issue” than in other distributed context: • High Dynamism • constant change in the set of resource • A certain resource can be available when discovered and no more available when selected. • This produced unpredictable latency. • Virtualization • There is a gap between virtual and real state of a resource • A free selected resource can have a state different than expected and be not suitable for the job requirements. • This provides no guarantees on successful job completion.

5. QoS over Grid /3 • Currently, there are some proposals (Rana, Manascè et al.) for providing guarantees to the Grid User on the availability and quantity of resources. Such approaches underline that: • QoS is a cross-layers issue. • QoS can be granted only if some “guarantees” are provided at lower level. • Unfortunately, it is still not possible to provide guarantees on time constraints from the discovery of resources to execution of a job within a deadline.

6. Grid and Time constraints • The set of timed-based applications from the best effort to the Soft Real Time ones could take many advantages in running on the Grid. • Cheapness (less expensive than buying proper machine for managing peaks of computation)‏ • Availability (resource availability due to pre-reservation)‏ • Fault-tolerance (redundancy due to the high number of possible available resources)‏

7. Time-oriented application • In our work, we considered job that are: • Deterministic • Time-constrained (till Soft Real Time) • Our SoRTGrid aims to implement strategies to satisfy the “missing requirements” and make Grid accessible for tasks like: • Coordination of swarms of robot • Real-time management of sensor networks • Advanced streaming applications over Grid (effective TV-on-demand, on-the-fly conversions, …)‏

8. Presenting SoRTGrid • A Grid framework able to manage time constraints must have: • Resource discovery “in time” • Pre-reservation mechanisms to avoid inconsistencies during the acquisition phase. • Mechanism to assure the effective availability of selected resources • The achievement of the above conditions depends strictly on a automatic and proficient interaction between Grid Users and Resource Owners without any centralized metascheduler.

9. Presenting SoRTGrid /2 • User and Owner knows better than a metascheduler their respective realities. • SortGrid requires that both parts possess some “abilities” • User side: • Evaluation of execution time • Definition of execution deadline • Owner side: • Direct control of resources (up-to-date information on their state) • Resolution of semantic gap typical of metaschedulers

10. Architecture of SoRTGrid • Interaction between User and Owner are provided by a couple of autonomic agent classes, the User and the Owner one. • User Agent: It uses SoRTGrid to retrieve resource and executes User jobs within their deadline. It provides information on job requirement to SoRTGrid thrugh a proper document, called JRM (Job Requirement Manifest). • Owner Agent: It uses SoRTGrid to “offer” the resources possessed by the Owner to the User Agents under certain QoS level and time conditions. It produces set of SoRTBids that are documents containing a “proposal” on a set of resources.

11. Architecture of SoRTGrid /2 • SoRTGrid act as a framework and provides basical service for sets of User’s and Owner’s Agents. • It is composed by a set of independent Facilitator Agents, everyone operating on a certain subpart of the Grid.

12. Facilitator Agent • Facilitator Agent is composed by: • Singleton BidMan Grid Service • Factory DiPe Service • A local SoRTBid repository • User and Owner Agents (and their respective resources) are registered on a certain Facilitator Agent and interact with it only. • Operations (discovery, negotiation, etc) are executed by interaction of Facilitator Agent components. • FA interacts with Grid middleware to obtain basic funcitonalities.

13. Services provided by SoRTGrid • Owner Agent side: • Publication of SoRTBids through the Local Repository. • Offering of SoRTBids to User Agents • Feedbacks on SoRTBids. • User Agent side: • Time-constrained discovery of SoRTBids • Pre-selection of SoRTBids.

14. Owner Agent: SoRTBids and BidMan • A SoRTBid contains: • Amount of resources shared • Minimum level of QoS offered • Period of time in which the resource is offered (P). • Effective execution time T • Expiration time. • A SoRTBid is published through the BidMan service: • It allows upload/removal of SoRTBids. • It performs maintenance operations and provide statistics.

15. User Agent: JRM and DiPe • A User Agent realizes for Grid User: • Job requirements appraisal • Job discovery, selection, megotiation and execution within the deadline. • A JRM contains information on the job: • Resource requirements • QoS class • Expiration Threeshold • Appraisal Value • Utility function parameters • Number of required SoRTBids • The JRM is provided to the DiPe Grid Service to start a discovery and pre-selection of SoRTBids.

16. SoRTGrid and QoS • SoRTGrid (and related SoRTBids) supports three classes of QoS: • Soft Real Time Class. The higher level of QoS; it guarantees the total availability of the resources for T. • High Level Class. The Texecution time is granted in P but the job can be interrupted. • Standard Class. The Texecution time is granted only if jobs with higher QoS level do not run on the same resources.

17. “Physiology” of SoRTGrid

18. Discovery phase in SoRTGrid • It is composed by a Local and a Remote Discovery. • Local Discovery provides a selection of SoRTBid on the Local Repository. • Remote Discovery is performed only if enough “suitable” bids have not been retrieved and ends when the time allocated for discovery phase has passed. • DiPe service provides the following services for the discovery phase: • Local Discovery and PRE-RESERVATION • Starts Remote discovery • Delivery of suitable SoRTBids to UA.

19. Discovery phase in SoRTGrid /2

20. Benefits of SoRTGrid • Autonomy. The two parts involved work independently for different aims and iteract to maximize the respective results and there is no centralization of any kind. • Adaptability. SoRTGrid is defined by a set of parameters, so it it adabptable to different underlying Grid context. • Consistency. Interaction between User and Owner Agent grants a flow of up-to-date information. A pre-reservation system assures that a resource remains effectively available for at least a certain time. • Automation. The use of agents excludes human part to be involved directly, avoiding undefined latencies, accordingly with the time critical goal fo SoRTGrid.

21. Conclusions and future works • A simulation of SoRTGrid has been implemented through GridSim. • We’re simulating different application scenarios. • Our research is oriented to the definition of classes of application suitable for the execution in SoRTGrid. • Further implementation of negotiation and execution phase.