Design and Evaluation of a Resource Selection Framework for Grid Applications

1. Design and Evaluationof a Resource Selection Frameworkfor Grid Applications Chuang Liu, Lingyun Yang, Dave Angulo, Ian Foster Distributed System Lab University of Chicago http://dsl.cs.uchicago.edu Work performed within the Grid Application Development Software (GrADS) Project of the NSF Next Generation Software Program

2. Problem • Co-selection of resources, which involves • Select a set resources for an Grid application that meet application requirements • Map workload to resources • Investigate feasibility of declarative language for expressing application requirements and resource capabilities • Set matching as an extension of matchmaking

3. Outline • Problem • Set matching • Resource selection service • Experiments • Summary

4. ClassAds Language(U.Wisconsin Condor Group) • ClassAds is a language for expressing and evaluating a named expression • A ClassAd is a set of named expressions • Attributes: describe characteristics • Requirements: identify matched ClassAds • Rank: ranks matched ClassAds

5. Set Match • Set extended ClassAds language • Represent characteristics of resources • Represent requests for a resource set • Matchmaker • Construct candidate resource sets • Identify compatible matches between requests and candidate resource sets • Rank compatible matches

6. Set Match:Set-extended ClassAds Language • Aggregation functions for set attributes • Max(expression), Min(expression), Sum(expression), SetSize(set) • Other functions • Suffix(S, <string List>), Prefix(S, <string List>) • Example:

7. Set Match: Matchmaker requests Set-extended Condor matchmaking engine Set Resource ClassAd 1 Constructor Resource ClassAd 2 resources evaluate Request ClassAd Resource ClassAd 3 {Res2} {Res1,Res2} Resource ClassAd 4 {Res1,Res3} match, or fail

8. Set Match: An Example

9. Outline • Problem • Set matching • Resource selection service • Experiments • Summary

10. Resource Selection Service (RSS) • Help applications to choose a good resource set in Grid environment • Synchronous and asynchronous service • Mapping application workload to resources if needed

11. Resource Selection Service: Framework RSS GIIS Resource Information Resource Request MDS Set GRISes Resource App Matcher Monitor NWS Result Mapper

12. RSS: Resource Request • Owner: The sender of this request • Type of Service: Synchronous or asynchronous • Job description: The characteristics of the job to be run, for example, problem size, and the performance model • Job resource requirements: User resource requirements, for example, memory capability, type of operating system, software packages installed, etc. • Mapper: The mapper algorithm to be used • Rank: Criteria to rank the matched resources

13. Resource Request: An Example 1. [ Service = “SynService"; 2. iter=100; alpha=100; x=100; y=100; z=100; 3. computetime = x*y*alpha/other.cpuspeed*370; 4. comtime= ( other.RLatency+ y*x*254/other.RBandwidth +other.LLatency+y*x*254/other.Lbandwidth); 5. exectime=(computetime+comtime)*iter+startup; 6. Mapper = [type ="dll"; libraryname="cactus"; function="mapper"]; 7. requirements = Sum(other.MemorySize) >= (1.757 + 0.0000138*z*x*y) && suffix(other.machine, domains); 8. domains={ “cs.utk.edu”, “ucsd.edu”}; 9. rank=Min(1/exectime) ] Type of Service Job description Mapper Resource Requirements Rank

14. Resource Request: Result <virtualMachine> <result statusCode="200" statusMessage="OK"/> <machineList> <machine dns="torc2.cs.utk.edu" processor= 2 x= 20> <machine dns="torc3.cs.utk.edu" processor= 2 x= 15> <machine dns="torc6.cs.utk.edu" processor= 2 x= 15> </machineList> </virtualMachine>

15. Outline • Problem • Set Match • Resource Selection Service • Experiments • Summary

16. Experiments: Application & Testbed • Cactus code that simulates the 3D scalar field produced by two orbiting sources • Application performance model • ExecTime= Comm. time + Computation time+ start-up • Application mapping algorithm • 1 Dimension mapping • Experiment environment • GrADS testbed • Comprises workstation clusters at universities across the United States (including the University of Chicago, UIUC, UTK, UCSD, Rice University, and USC/ISI)

17. Experiments: Mapping Result • Machine1: 450 MHz, no CPU load • Machine2: 500 MHz, CPU load=2

18. 1. o.ucsd.edu, mystere.ucsd.edu, saltimbanco.ucsd.edu 2: mystere.ucsd.edu, o.ucsd.edu 3: o.ucsd.edu, Saltimbanco.ucsd.edu 4: o.ucsd.edu 5: saltimbanco.ucsd.edu 6: mystere.ucsd.edu Experiments: Resource Selection (One Site)

19. 1: torc6.cs.utk.edu 2: o.ucsd.edu 3: Saltimbanco.ucsd.edu 4: torc6.cs.utk.edu + o.ucsd.edu 5: o.ucsd.edu + saltimbanco.ucsd.edu 6: o.ucsd.edu, mystere.ucsd.edu, torc 6: cs.utk.edu Experiments: Resource Selection (Two Sites)

20. Summary • Extended the ClassAds language to describe set-based requirement for a resource set • Implemented a set matchmaker & created a resource selection service framework • Validation with Cactus application • Future: Extend semantics, implementation, application of set matching framework • Thanks to • NSF Next Generation Software Program • Alain Roy, Jennifer Schopf, GrADS colleagues • http://dsl.cs.uchicago.edu