1 / 31

Resource Selector

Resource Selector. Chuang Liu. What do we want to do?. A smart Resource Selector. Resource requirement. App. R S. What do we want to do?. App. R S. These Resources seem fit your requirement best. Goal. A Resource Selector for general purpose

wynter-fitzgerald
Download Presentation

Resource Selector

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Resource Selector Chuang Liu

  2. What do we want to do? • A smart Resource Selector Resource requirement App R S

  3. What do we want to do? App R S These Resources seem fit your requirement best

  4. Goal • A Resource Selector for general purpose • Matching between application’s requirement and a set of resources • Adaptability to dynamic status of distributed environment • Support ownership of resource • Support performance model of application • The interface between application and RS should be simple

  5. RS--Structure App RS A subset of resource GIIS Resource Selector App MDS MatchMaker GRIS Requirement Resource

  6. Challenges • How to specify resource and request and how to match request with resource. • Consistency of data in RS and system status. • How to choose N resources from M available resources. (N <=M)

  7. Challenges • How to specify the request and resource and how to match the request with resource • Consistency of data in RS and system status. • How to choose N resources from M available resources. (N <=M)

  8. ClassAd—Mechanism to specify resources and request • The classad mechanism is a language for expressing and evaluating attribute • A classad is a set of named expressions • Each named expression is called an attribute • Expression similar to those found in C/C++

  9. ClassAd : example(resource) • [ • OpSys=“LINUX”; • Name= “trapezius.cs.uchicago.edu”; • LoadAvg= 0.03; • Friends = (“foster”, “dave”); • Untrusted = (“evil”, “rival”); • Constraints= !member(other.Owner, Untrusted) && (LoadAvg < 0.3); • ]

  10. ClassAd : example(request) • [ • Owner=“chliu”; • Requirements= other.LoadAvg < 0.3 && other.opSys=“LINUX”; • Rank = 1/other.LoadAvg; • ]

  11. Challenges • How to specify the resource and request and how to match the request with resource. • Consistency of data in RS and system status. • How to choose N resources from M available resources. (N <=M)

  12. Consistency • Several threads in RS update information about system status based TTL value. • Update information about available resource by access GIIS • Update information about status of every resource by access GRIS or GIIS • Tradeoff between Performance and Consistency

  13. Challenges • How to specify the resource and request and how to match the request with resource. • Consistency of data in RS and system status. • How to choose N resources from M available resources which fit application’s requirement best. (N <=M)

  14. Resource Selection • How to select N resource from M available resources efficiently. • How to judge which one is best among several matched results

  15. Resource Selection • How to select N resource from M available resources efficiently. • How to judge which one is best among several matched results

  16. Criteria to judge the desirability of resource • Performance model • F(resource Info, application info) • [ minCPUSpeed > 10 MIPS • minMemSize > 100 MB • Rank= minCPUSpeed * NumOfResource • ] • Embed a program or function call in Classad • ? Classad don’t support function call in expression.

  17. Resource Selection • How to Select N resources from M available resources • How to judge which one is best among several matched results

  18. Bilateral match- Clique • Organize all available resources into several cliques. Match maker Classads for clique Classads for requirement Classads for resource

  19. Clique- How to organize clique Methods to organize clique • Manually • Automatically • Pros: • Easy and useful • Cons: • Not flexibility

  20. Clique- Naive • Naive method • For example: • Resource: { a, b, c} • Cliques {a}, {b}, {c}, {a, b}, {b,c}, {a,c}, {a, b, c} • Cons • The number of clique is 2 to N

  21. Gang Match • Gang Matching 1 N Match maker Classads for requirement Classads for resource

  22. Gang match- Greedy Algorithm • Greedy Algorithm • Clique, candidate = null • Match the requirement with every resource • Choose resource with highest ranking as the first number of clique • For(;;) { • If (clique match requirement) and (performance of application increase) • Candidate=clique • Match the requirement with resource which is not in clique • Add node with highest ranking in the left nodes to clique • Else • Return Candidate • }

  23. Gang match- Greedy Algorithm • Pros: • High performance • Give pretty good optimal result to loosely coupled application • Cons: • Locally optimal choice does not always lead to globally optimal solution

  24. Gang match- Port and docking • Ports and docking • [ Ports = { • [ Label = Host1; • Requirements= MemorySize > 17.2M; • Constraint = Host1.Arch == "INTEL" && Host1.OpSys == "LINUX"; • Rank = Host1.MIPS • ], • [ Label = Host2; • Requirements= MemoryReqs > 18 M; • Constraint = Host2.Arch == "INTEL" && Host2.OpSys == "LINUX" && • Host1.Subnet == Host2.Subnet • Rank = Host2.KFlops • ] • Rank = 1 * Host1.MIPS + 8 * Host2.Kflops; • } ]

  25. Gang match- Port and docking • Pros: • Internal mechanism provided by Classad • ? Available in Classad package • Cons: • Match performance • Application need to specify how many nodes it wanted

  26. Gang match- Dynamic programming • Limited resource use • System administrator control how many resources user can use • User’s requirement • [ performance > threadhold value && number of resource is as little as possible] • Application’s requirement • App specify how many CPU it wanted in request • A 4 X 4 X 4 Grid calculation, numOfCPU < 64

  27. Gang match Dynamic programming • Multi-dimensions knapsack problem • Knapsack problem : “There are M items, every item has a Weight and a Value. Try to choose items from these items such that their value is maximum and their total weight is less than W.”

  28. Gang match- Dynamic programming • Pros: • More flexible • Always provide good resource to application • polynomial time algorithm o(N) • Cons: • Performance is bad than greedy algorithm

  29. Project Status • A RS is running, but not smart enough. • Two match strateges have been implemented • Clique • Greedy algorithm

  30. Open problem • How well does RS work? • Performance of application on selected resource • Cost of RS • Evaluate different strategy in match making

  31. Welcome comment and suggestion Thank you!

More Related