An Economy Driven Resource Management Architecture for Global Computational Power Grids

1. An Economy Driven Resource Management Architecture for GlobalComputational Power Grids Rajkumar Buyya, David Abramson, and Jonathan Giddy. Presented By : Vikas

2. The EcoGrid Project Supported by Collaboration

3. Presentation Layout • Issues • Features Expected of a distributed resource environment • Resource Management Structures • Need For Computational Economy • Economy driven Grid Resource Management Architecture • GRACE • A new Nimrod/G Resource Broker • Conclusions and Future Work

4. Issues • The growing computational requirements • Availability of Internet as a ubiquitous commodity communication media • Low cost high performance machines such as clusters across multiple organizations • Rise of scientific problems of multi organizational interest

5. 2100 2100 2100 2100 2100 2100 2100 2100 2100 Computing Platforms PERFORMANCE Administrative Barriers Individual Group Department Campus State National Globe Inter Planet Universe Inter Planet Cluster/Grid ?? Single Processor Shared Memory Local Cluster Global Cluster/Grid

6. Features Expected of a geographically distributed Multi-Organizational Resources • Flexibility and extensibility • Domain autonomy • Scalability • Single global name space • Ease of use and transparent access • High performance • Security • Management and exploitation of resource • Heterogeneity • Interoperability with multiple systems • Resource allocation or co-allocation • Fault-tolerance • Dynamic adaptability • Economy of computation

7. Resource Management Structures Scheduler Model : Centralized (Single Resource) Scheduler Architecture : Resource Example System : Unix, Linux, Windows

8. Resource Management Structures Scheduler Model : Centralized (Multiple Resources) (Single/Multiple Domains) Scheduler Architecture : Resource 1 Resource 2 Jobs Queue Resource n Example System : Unix, Linux, Windows

9. Resource Management Structures Scheduler Model : Decentralized (Self coordinated or Job Pool) (Single/Multiple Domains) Scheduler Architecture : Resource 1 Resource 2 (Job Pool) Resource n Cluster systems like MOSIX follows simple model. Grid systems can follow but it is complex to realize Example System :

10. Resource Management Structures Scheduler Model : Hierarchical (Multiple Domains) Scheduler Architecture : Resource 1 Resource 2 (Resource Broker or super scheduler) (Local Scheduler) Resource Resource n Example System : Cluster systems or Grid Systems such as AppLes, Nimrod/G,HCondor

11. Some Resource Management Models

12. Why Computational Economy ? • What comprises the Grid? • What motivates one to contribute their resource to the Grid? • Is it possible to have access to all resources in the Grid by contributing our resource? • If not, how do we have access to all Grid resources? • If we have access to resources through collaboration, are we allowed to solve commercial problems? • If we gain access to Grid resources by paying money, do resource owners need to charge the same or different price for other users? • Is access cost the same for peak and off-peak hours? • How can resource owners maximize their profit? • How can users solve their problems within a minimum cost? • If the user relaxes the deadline by which results are required, can solution cost be reduced?

13. Benefits of economy based resource Management • It helps in building large-scale computational grid as it motivates resource owners to contribute their idle resources for others to use and profit from it. • It provides fair basis for access to grid resources for everyone. • It helps in regulating the demand and supply. • It offers an incentive for users to back off when solving low priority problems and thus encourages the solution of time critical problems first. • It removes the need for a central coordinator (during negotiation). • It offers uniform treatment to all resources. That is,it allows trading of everything including computational power, memory, storage, network bandwidth/latency, and devices or instruments. • It helps in developing scheduling policies that are user centric rather than system centric

14. Benefits of economy based resource Management(Contd..) • It offers an efficient mechanism for allocation and management of resources. • It helps in building a highly scalable system as decision-making process is distributed across all users and resource owners. • Finally, it places the power in the hand of both resource owners and users—they can make their own decisions to maximize the utility and profit.

15. Economy driven Grid Resource Management Architecture Key Components • User Applications • Grid Resource Broker • Grid Middleware • Local Resource Manager

16. Grid Components

17. Economy driven Grid Resource Management Architecture

18. GRACE(Grid Architecture for computational Economy) Components in the infrastructure • A Trade Manager • Trading Protocols and APIs • A Trade server

19. Grid Trade Manager and Trade Server • It offers uniform treatment to all resources. That is,it allows trading of everything including computational power, memory, storage, network bandwidth/latency, and devices or instruments. • It helps in developing scheduling policies that are user centric rather than system centric. • It offers an efficient mechanism for allocation and management of resources. • It helps in building a highly scalable system as decision-making process is distributed across all users and resource owners. • Finally, it places the power in the hand of both resource owners and users—they can make their own decisions to maximize the utility and profit.

20. Grid Open Trading APIs • grid_request_quote(tid, DT) • grid_trade_negotiate (tid, DT) • grid_trade_confirm(tid, DT) • grid_trade_cancel(tid, DT) • grid_trade_change( tid, DT) • grid_trade_reconnect(tid, resource_id) • grid_trade_disconnect(tid) • where • tid = Trade Identification code • DT = Deal Template

21. Call for Bid(DT) Grid Open Trading Protocols Bid Manager Bid Server Get Connected Pricing Rules API Reply to Bid(DT) Negotiate Deal(DT) …. Confirm Deal(DT, Y/N) DT - Deal Template - resource requirements (BM) - resource profile (BS) - price (any one can set) - status - change the above values - negotiation can continue - accept/decline - validity period Cancel Deal(DT) Change Deal(DT) Get Disconnected

22. Open Trading Finite State Machine DT <BM, Request for Resource > <BM, Ask Price > <<BS, Update >> <<BM, Update >> ND <BS, Bid > <BS, Final Offer > <BM, Final Offer > Offer BS Offer BM <BM, Rej.> <BM, Accept > <BS, Reject > D NA ND-Negotiate Deal

23. A New Nimrod/G Resource Broker

24. Conclusions and Future Work • Discussed a new middle service infrastructure - GRACE • Realization of various scheduling models driven by computational economy and incorporation of into Nimrod/C resource broker • Explore scheduling algorithms based on resource reservation and dynamic computational economy • Plan to drive the scheduling work based on fuzzy logic and genetic algorithms

25. Grid Projects Around The World • USA: Globus, Legion, WebFlow, NetSolve, and NASA IPG. • Asia/Japan: Ninf and Bricks • Australia: Nimrod/G and DISCWorld. • Europe: UNICORE, CERN Data Grid, MOL, Globe,DAS, MetaMPI.