1 / 32

Design and Evaluation of a Resource Selection Framework for Grid Applications

Design and Evaluation of a Resource Selection Framework for Grid Applications. University of Chicago. Problem. Co-selection of resources Resource Selection Resource configuration Mapping workload to resources. Outline. Problem > Related work Set Match Resource Selection Service

Download Presentation

Design and Evaluation of a Resource Selection Framework for Grid Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Design and Evaluationof a Resource Selection Frameworkfor Grid Applications University of Chicago

  2. Problem • Co-selection of resources • Resource Selection • Resource configuration • Mapping workload to resources

  3. Outline • Problem • > Related work • Set Match • Resource Selection Service • Experiments • Summary • Future work

  4. Related work • Networked Batch Queue System • PBS, LSF, NQE • Application-specific scheduler • AppLeS, etc. • Condor

  5. Outline • Problem • Related work • > Set Match • Resource Selection Service • Experiments • Summary • Future work

  6. Set Match • Resource Set: • Several resources with similar characteristics that may be located in different sites. • Set Match • A Set Match occurs between a resource request and a resource set. If the resource set fulfill the resource request and the resource request fulfill resource’s management policy, they match each other.

  7. Set Match • Matchmaker

  8. Set Match • ClassAds Language The classads language is a language for expressing and evaluating attributes. An attribute is a named expression. A classad is a set of named expressions. Expression similar to those found in C/C++. B=[ name=”foo”; type="machine"; cpuspeed=800M; memory=512M] A=[ Requirements = other.type = = "machine" && other.cpuspeed > 500M; Rank = other.cpuspeed ]

  9. Set Match • Set-Extended ClassAds language • Aggregation functions • Max(), Min(), Sum(), SizeOfSet() • Other functions • suffix(S, L), Prefix(S, L)

  10. Set Match An Example of the Set Extended ClassAds [ Type=”Set”; Domainlist={ucsd.edu, cs.uiuc.edu}; requirements = Sum(other.memory) > 1G && other.cpuspeed > 200M && suffix(other.Hostname, Domainlist); rank = Min(other.cpuspeed)*SizeOfSet() ]

  11. Resource Selection mechanism

  12. Resource Selection mechanism

  13. Outline • Problem • Related work • Set Match • > Resource Selection Service • Experiments • Summary • Future work

  14. Resource Selection Service • Help applications to choose a good Resource Set in Grid environment. • Synchronous and asynchronous service • Mapping application workload to resources if needed.

  15. Resource Selection Service • Framework

  16. Resource Request • Owner: The sender of this request. • Type of Service: Synchronous or asynchronous. • Job description: The characteristics of the job to be run, for example, problem size, problem specification and the performance model. • Job resource requirements: User resource requirements, for example, memory capability, type of operating system, software packages installed, etc. • Mapper: The kind of mapper algorithm to be used. • Rank: The criteria to rank the matched resources.

  17. Resource Request: An example Type of Service 1. [ Service = "InstantService"; 2. iter=100; alpha=100; x=100; y=100; z=100; 3. computetime = x*y*alpha/other.cpuspeed*370; 4. comtime= ( other.RLatency+ y*x*254/other.RBandwidth +other.LLatency+y*x*254/other.Lbandwidth); 5. exectime=(computetime+comtime)*iter+startup; • Mapper = [type ="dll"; libraryname="cactus"; function="mapper"]; 7. requirements = Sum(other.MemorySize) >= (1.757 + 0.0000138*z*x*y) && suffix(other.machine, domains); 8. domains={ cs.utk.edu, ucsd.edu}; 9. rank=Min(1/exectime) ] Job description Mapper Resource Requirements Rank

  18. Resource Selection Result <virtualMachine> <result statusCode="200" statusMessage="OK"/> <machineList> <machine dns="torc2.cs.utk.edu" processor= 2 x= 20> <machine dns="torc3.cs.utk.edu" processor= 2 x= 15> <machine dns="torc6.cs.utk.edu" processor= 2 x= 15> </machineList> </virtualMachine>

  19. Outline • Problem • Related work • Set Match • Resource Selection Service • > Experiments • Summary • Future work

  20. Cactus application • Application performance model • ExecTime= (Communication time + Computation time) * slowdown • Application mapping algorithm • 1 Dimension mapping R 1

  21. Cactus application • Application performance model • ExecTime= (Communication time + Computation time) * slowdown • Application mapping algorithm • 1 Dimension mapping R 1 R 2

  22. Cactus application • Application performance model • ExecTime= (Communication time + Computation time) * slowdown • Application mapping algorithm • 1 Dimension mapping R 1 R 2 R 3

  23. Cactus application • Application performance model • ExecTime= (Communication time + Computation time) * slowdown • Application mapping algorithm • 1 Dimension mapping R 1 R 2 R 3

  24. Execution Time Prediction Time (S) Torc1,3, 5, o.ucsd.edu Torc1,3, 5 Torc1 Torc5 Torc1,3, 5 Torc1 Torc5 torc1 Torc1 Torc5 Problem Size

  25. Resource Selection Algorithm Test • 1. o.ucsd.edu, mystere.ucsd.edu, saltimbanco.ucsd.edu • 2: mystere.ucsd.edu, o.ucsd.edu • 3: o.ucsd.edu, Saltimbanco.ucsd.edu 4. o.ucsd.edu • 5: saltimbanco.ucsd.edu 6: mystere.ucsd.edu

  26. Resource Selection Algorithm Test • 1: torc6.cs.utk.edu 2: o.ucsd.edu 3: Saltimbanco.ucsd.edu • 4. Torc6.cs.utk.edu + o.ucsd.edu 5: o.ucsd.edu + saltimbanco.ucsd.edu • 6. o.ucsd.edu, mystere.ucsd.edu, torc6.cs.utk.edu

  27. Outline • Problem • Related work • Set Match • Resource Selection Service • Evaluation of the implementation • > Summary • Future work

  28. Summary • Extended the ClassAds language to describe the requirement for a Resource Set • Implement a Set Matchmaker • Create a general framework for the Resource Selection Service • Validate our implementation by the Cactus application

  29. Future Work • Extend the resource selection algorithm to support other applications. • Gang match • Resource with particular topology • Provide fault tolerance to incomplete or error information • A general matchmaking mechanism

  30. Welcome comments and suggestions Thank you!

  31. Problem • Co-selection of resources • Dynamic nature of Grid resources • Heterogeneity of resources • The autonomy of resources • The quality of resources depends on the application’s run characteristics

More Related