310 likes | 457 Views
Resource Selector. Chuang Liu. What do we want to do?. A smart Resource Selector. Resource requirement. App. R S. What do we want to do?. App. R S. These Resources seem fit your requirement best. Goal. A Resource Selector for general purpose
E N D
Resource Selector Chuang Liu
What do we want to do? • A smart Resource Selector Resource requirement App R S
What do we want to do? App R S These Resources seem fit your requirement best
Goal • A Resource Selector for general purpose • Matching between application’s requirement and a set of resources • Adaptability to dynamic status of distributed environment • Support ownership of resource • Support performance model of application • The interface between application and RS should be simple
RS--Structure App RS A subset of resource GIIS Resource Selector App MDS MatchMaker GRIS Requirement Resource
Challenges • How to specify resource and request and how to match request with resource. • Consistency of data in RS and system status. • How to choose N resources from M available resources. (N <=M)
Challenges • How to specify the request and resource and how to match the request with resource • Consistency of data in RS and system status. • How to choose N resources from M available resources. (N <=M)
ClassAd—Mechanism to specify resources and request • The classad mechanism is a language for expressing and evaluating attribute • A classad is a set of named expressions • Each named expression is called an attribute • Expression similar to those found in C/C++
ClassAd : example(resource) • [ • OpSys=“LINUX”; • Name= “trapezius.cs.uchicago.edu”; • LoadAvg= 0.03; • Friends = (“foster”, “dave”); • Untrusted = (“evil”, “rival”); • Constraints= !member(other.Owner, Untrusted) && (LoadAvg < 0.3); • ]
ClassAd : example(request) • [ • Owner=“chliu”; • Requirements= other.LoadAvg < 0.3 && other.opSys=“LINUX”; • Rank = 1/other.LoadAvg; • ]
Challenges • How to specify the resource and request and how to match the request with resource. • Consistency of data in RS and system status. • How to choose N resources from M available resources. (N <=M)
Consistency • Several threads in RS update information about system status based TTL value. • Update information about available resource by access GIIS • Update information about status of every resource by access GRIS or GIIS • Tradeoff between Performance and Consistency
Challenges • How to specify the resource and request and how to match the request with resource. • Consistency of data in RS and system status. • How to choose N resources from M available resources which fit application’s requirement best. (N <=M)
Resource Selection • How to select N resource from M available resources efficiently. • How to judge which one is best among several matched results
Resource Selection • How to select N resource from M available resources efficiently. • How to judge which one is best among several matched results
Criteria to judge the desirability of resource • Performance model • F(resource Info, application info) • [ minCPUSpeed > 10 MIPS • minMemSize > 100 MB • Rank= minCPUSpeed * NumOfResource • ] • Embed a program or function call in Classad • ? Classad don’t support function call in expression.
Resource Selection • How to Select N resources from M available resources • How to judge which one is best among several matched results
Bilateral match- Clique • Organize all available resources into several cliques. Match maker Classads for clique Classads for requirement Classads for resource
Clique- How to organize clique Methods to organize clique • Manually • Automatically • Pros: • Easy and useful • Cons: • Not flexibility
Clique- Naive • Naive method • For example: • Resource: { a, b, c} • Cliques {a}, {b}, {c}, {a, b}, {b,c}, {a,c}, {a, b, c} • Cons • The number of clique is 2 to N
Gang Match • Gang Matching 1 N Match maker Classads for requirement Classads for resource
Gang match- Greedy Algorithm • Greedy Algorithm • Clique, candidate = null • Match the requirement with every resource • Choose resource with highest ranking as the first number of clique • For(;;) { • If (clique match requirement) and (performance of application increase) • Candidate=clique • Match the requirement with resource which is not in clique • Add node with highest ranking in the left nodes to clique • Else • Return Candidate • }
Gang match- Greedy Algorithm • Pros: • High performance • Give pretty good optimal result to loosely coupled application • Cons: • Locally optimal choice does not always lead to globally optimal solution
Gang match- Port and docking • Ports and docking • [ Ports = { • [ Label = Host1; • Requirements= MemorySize > 17.2M; • Constraint = Host1.Arch == "INTEL" && Host1.OpSys == "LINUX"; • Rank = Host1.MIPS • ], • [ Label = Host2; • Requirements= MemoryReqs > 18 M; • Constraint = Host2.Arch == "INTEL" && Host2.OpSys == "LINUX" && • Host1.Subnet == Host2.Subnet • Rank = Host2.KFlops • ] • Rank = 1 * Host1.MIPS + 8 * Host2.Kflops; • } ]
Gang match- Port and docking • Pros: • Internal mechanism provided by Classad • ? Available in Classad package • Cons: • Match performance • Application need to specify how many nodes it wanted
Gang match- Dynamic programming • Limited resource use • System administrator control how many resources user can use • User’s requirement • [ performance > threadhold value && number of resource is as little as possible] • Application’s requirement • App specify how many CPU it wanted in request • A 4 X 4 X 4 Grid calculation, numOfCPU < 64
Gang match Dynamic programming • Multi-dimensions knapsack problem • Knapsack problem : “There are M items, every item has a Weight and a Value. Try to choose items from these items such that their value is maximum and their total weight is less than W.”
Gang match- Dynamic programming • Pros: • More flexible • Always provide good resource to application • polynomial time algorithm o(N) • Cons: • Performance is bad than greedy algorithm
Project Status • A RS is running, but not smart enough. • Two match strateges have been implemented • Clique • Greedy algorithm
Open problem • How well does RS work? • Performance of application on selected resource • Cost of RS • Evaluate different strategy in match making
Welcome comment and suggestion Thank you!