1 / 22

Policy Driven Heterogeneous Resource Co-Allocation with Gangmatching

Policy Driven Heterogeneous Resource Co-Allocation with Gangmatching. Rajesh Raman, Miron Livny, Marvin Solomon University of Wisconsin Presented by: Kashmira Phalak. Key Ideas.

eleanorhill
Download Presentation

Policy Driven Heterogeneous Resource Co-Allocation with Gangmatching

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Policy Driven Heterogeneous Resource Co-Allocation with Gangmatching Rajesh Raman, Miron Livny, Marvin Solomon University of Wisconsin Presented by: Kashmira Phalak

  2. Key Ideas • Dynamic, heterogeneous and distributively owned resource environments pose the challenge of resource representation, allocation and management • Conventional resource management methods fail to address these issues • Matchmaking matches each customer with a single resource: Inadequate • Gangmatching satisfies applications which require a collection of heterogeneous resources to be matched and allocated to perform a task • Gangmatching particularly focuses on the license management co-allocation problem

  3. What is Matchmaking? • Matchmaking is a methodology for Distributed Resource Management • Actions involved in Matchmaking: Matchmaker (Match Algorithm(2)) Advertisement(1) Advertisement(1) Match Notification(3) Match Notification(3) Entity (Requester) Entity (Provider) Claiming(4)

  4. Motivation for Gangmatching • The License Management Problem • Limited number of licenses and validity on only a few workstations

  5. The Gangmatching Model • Multilateral Gangmatching extends the bilateral matchmaking scheme • Matchmaking Framework: • ClassAd representation • Matchmaking Algorithm • Matchmaking (Notification) protocols • Claiming protocols

  6. The Gangmatching Model • ClassAd Representation Gangmatch Request:

  7. The Gangmatching Model Workstation Ad: License Ad:

  8. The Gangmatching Model • Matchmaking Algorithm • Matchmaking (Notification) Protocols • Involves sending copies of the matched gang to every agent • whose advertisement is involved in the gang.

  9. The Gangmatching Model • Claiming Protocols: • Proceed in a 2 phase manner: 1. VERIFY: • Root agent begins the phase by asking subordinates to verify if they want to join the gang • If reply is OK then COMMIT phase initiated • If reply is ABORT then root agent tells gang members to discard the match and start over • If root’s subordinates have children, then the phase is recursively performed 2. COMMIT • After root gets OK from all children, claims and allocations done

  10. Gangmatching Implementation Fill Direction • Naïve Gangmatching: Backtracking Direction ROOT AD Advertisement Filled Port Unfilled Port Marshalled sub-gang Already Filled Not yet Filled Under Consideration

  11. Indexed Matchmaking Algorithm Outline 1: O ← offer classads 2: R ← request classads 3: Orect ← classads in O converted to rectangles 4: Oindexed ← indexed intervals of rectangles in Orect 5: Rsorted ← R sorted by priority 6: for each C in Rsorted do 7: R ← convert C into a rectangle (Window query to probe the index) 8: M ← all rectangles in Oindexed intersecting R 9: B ← pick best match from M 10: output(B,R) 11: remove B from Oindexed 12: end for

  12. Intuitive Scheme for Rectangle Conversion Offers Requests [ x=10; Constraint=other.y>=7 ] [ y=8; Constraint=other.x>=8 ] {(x, y) | x = 10, y >= 7} {(x, y) | y = 10, x >= 8},

  13. Gangmatching Implementation • LR: Indexed In-Order Gangmatching • Overcomes inefficiency of Naïve algorithm due to exclusive reliance on constraint expression evaluation mechanism • Use of index probes before filling docks reduces the number of candidates to be considered to fill the port • Indexes attributes using conventional search trees and constraint expressions using interval trees • Algorithm always picks the machine first and then finds the compatible license • Inherits the naive algorithm’s weakness when operating in workloads composed of scarce or absent resources.

  14. The Gangmatching Model • Dynamic Gangmatching • Solution: Dynamic Right-left operation • Algorithm picks the license first and then finds a machine compatible with it • Constraints are shunted from license classad to the cpu port of job classad • Uses index mechanism to identify compatible candidates and to dynamically determine the port fill order • Algorithm immediately aborts if desired resources are unavailable.

  15. Gangmatching Performance • Workloads are characterized by: N = Number of job and workstation classads L = License density, either 50% or 100% of N S = Selectivity index of license classads • Elapsed time performance of algorithms measured on Intel Pentium Pro Linux workstation with 256MB RAM • Extensive testing now done on Condor

  16. Naïve Algorithm Performance • Extremely inefficient when licenses are scarce • Not sensitive to license selectivity indexes • Expression evaluation and backtracking dominate cost of algorithm • Worst case: O(n3)

  17. LR: Indexed In-Order Gangmatching Performance

  18. Dynamic Algorithm Performance • Performance doesn’t significantly differ between the 50% and 100% regime • Scarcer resources are incorporated first, reducing the number of required index probes.

  19. Left-Right Vs. Dynamic Algorithm Performance • Dynamic Algorithm issues more queries under 100% license workload but shows marked improvement over Indexed In-order Algorithm under 50% license workload

  20. Conclusions & Future Work • Conclusions: • Gangmatching successfully addresses the problem of policy driven heterogeneous resource co-allocation • The data indexing method and heuristic-driven dynamic algorithm can provide efficient implementation • Performance evaluation of the three proposed algorithms demonstrates the feasibility of the Gangmatching solution • Future Work: • Incorporate preferences into gangmatching algorithm • Develop better algorithms for wider and deeper gangs • Overcome restriction of knowing number of resources in advance

  21. Analysis • The matchmaking process behaves like a black-box • Should it be made imperative to satisfy all constraints? • Performance evaluations are shown for just for a single platform and adequate information about it’s role is not provided • Reliability and scalability issues need to be evaluated to address the concerns of larger grids • An automatic mechanism should be provided to replace a classad with its residual resources after matching. • Paper seems a little unbalanced since it provides performance evaluations of implementations which are not completely explained

  22. Questions?

More Related