1 / 11

VGrADS Spring Workshop 2007

VGrADS Spring Workshop 2007. Grid Economics Progress Report. Andrew Mutz and Ryan Zhang April 18, 2007. Introduction. Project Goals Can we use economic mechanisms to tell us anything about the relative value of grid resources?

andrewsc
Download Presentation

VGrADS Spring Workshop 2007

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. VGrADS Spring Workshop 2007 Grid Economics Progress Report Andrew Mutz and Ryan Zhang April 18, 2007

  2. Introduction • Project Goals • Can we use economic mechanisms to tell us anything about the relative value of grid resources? • Can we use this exchange rate information to schedule jobs on computational grids? • Project History • Began with the idea of applying a market mechanism to log data, in order to find the relative value of grid resources • Has become a simulator that uses DAGs and performance models to compute the equilibrium prices for grid resources

  3. T’atonnement • Created by Leon Walras in 1874 • Literally translates to “groping” • Consumers and producers are told a set of possible prices for many goods • Each reports how much they would consume or produce at the given price level • The central auctioneer mechanism adjust the prices based on this information and repeats • The process iterates until a set of prices is found that, for all resources, the quantity demanded and the quantity supplied are equal

  4. T’atonnement • Set of k resources r = {r1, r2, …, rk} • Set of k prices p = {p1, p2, …, pk } • Prices are currency per hour for one node • The T’atonnement mechanism asks each consumer and producer how much they would consume or produce at the current price point and aggregates this information to form an Excess Demand vector, ED. • ED(p) = {z1, z2, … z3} • z1 = qc1 - qp1 • The mechanism adjusts each price pi based on zi and repeats the process, iteratively searching for prices that minimize ED.

  5. T’atonnement • The price adjustment rule is simple • pi = pi + c * zi • If zi is positive (demand exceeds supply), then c*zi is positive and the price rises • If zi is negative (supply exceeds demand), then c*zi is negative and the price falls • More sophisticated price adjustment schemes exist that take in to account the effect pi has on zj, i != j • Smale’s method

  6. Overall Model • At each step of price adjustment, each consumer and producer is asked how many resources they would like to consume or produce at the current price levels. • Currently, producers offer all available units at any price level • This causes prices to drop to zero in periods of under-demand • The current solution is set an arbitrarily minimum price • Consumer • Each consumer is represented as a DAG • Each DAG is assigned a budget, representing the value of the job to the user • Each DAG has a performance model that tells it how long each sub-task would take to execute on each of the resources available • Each DAG chooses the resource that is most cost effective for it and reports to the mechanism how many resources they would like to consume at the given prices.

  7. Experiment Settings • We use TeraGrid’s configuration as the simulation resource environment • There are totally 16 clusters (excluding SMP clusters) • There are totally 12381 individual computing nodes (many of them are multi-core but we didn’t use processor number) • We used both random generated DAGs and EMAN DAGs and can plug-in any type of DAG in the future • We used both artificial performance models and real performance data • We used different algorithms for DAGs to select the cluster and estimate the number of resources it will ask

  8. UML Class Diagram

  9. Pseudo-code for one consumer Precomputation: Levelize the DAG D For each cluster C in the environment For each level L in D calculate the computing time for level L Find the estimated makespan M for DAG D on C For each Cluster C in the pick the number of resources N need estimate the budget B as N*M*C.Price Sort the clusters by their cost effectiveness = M*B*Min(N/C.Size , 1) Choose the cluster with minimum CE N equals the width of the DAG for random generated DAGs N initially equals the width of the second widest level for EMAN and could be reduced by an integer factor as long as the deadline permits N depends on the structure of the DAG

  10. Results • We are able to consistantly find the price setting that matches the supply and demand within 2% of the total resources for various configurations

  11. Future Directions • Consumers could be a mix of individual jobs, MPI jobs and DAGs • Producers would provide different number of resources depends on the price • Provide an incentive towards the under demand pricing(air tickets?) • Manage to allocate resources to consumers in the under-demand situation • Introduce the notion of time to the price • create a economy based Grid queuing mechanism

More Related