GridG: Synthesizing Realistic Computational Grids
Download
1 / 39

Dong Lu , Peter A. Dinda Prescience Laboratory Department of Computer Science Northwestern University Evanston, IL 602 - PowerPoint PPT Presentation


  • 70 Views
  • Uploaded on

GridG: Synthesizing Realistic Computational Grids. Dong Lu , Peter A. Dinda Prescience Laboratory Department of Computer Science Northwestern University Evanston, IL 60201 . Outline. Why GridG? What is GridG? Topology generation Hierarchical vs. degree based?

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Dong Lu , Peter A. Dinda Prescience Laboratory Department of Computer Science Northwestern University Evanston, IL 602' - nan


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Dong lu peter a dinda prescience laboratory department of computer science northwestern university evanston il 6020

GridG: Synthesizing Realistic Computational Grids

Dong Lu, Peter A. Dinda

Prescience Laboratory

Department of Computer Science

Northwestern University

Evanston, IL 60201


Outline
Outline

  • Why GridG?

  • What is GridG?

  • Topology generation

    • Hierarchical vs. degree based?

    • What are the relationships among the power laws of Internet topology?

  • Annotation

    • What are the intra- and inter- correlations among the hosts and within a host?

    • How to build the correlations into GridG?

  • Conclusions and future work


Why gridg
Why GridG?

  • Synthetic Grids needed to evaluate Middleware

    • Existing physical grids too small

    • Can’t control parameters

  • Example: Evaluation of our RGIS system

  • Example: Grid simulation projects

    • GridSim and SimGrid

  • Example: overlay network simulations

    • Application level multicast


  • Gridg a synthetic grid generator
    GridG: A Synthetic Grid Generator

    • Output: Network topology annotated with the hardware and software available on each node and link.

      • Layer 3 network: hosts, routers, links

      • Hosts: memory, architecture, number of CPUs, disk, operating system, vendor, clock rate

      • Routers: switching capacity

      • Links: bandwidth and Latency


    Example 1
    Example 1

    Router (switching capacity)

    Link (bw, latency)

    Host (arch, numcpu, clock rate, osvendor, mem, disk,)


    Requirements
    Requirements

    • Realistic topologies

      • Connected

      • Hierarchical topology

      • Power laws of Internet topology

    • Realistic annotations

      • Distributions of attributes

      • Correlations of attributes

        • Intra-host

        • Inter-host


    Gridg architecture
    GridG architecture

    • A sequence of transformations on a text-based representation of an annotated graph.


    Outline1
    Outline

    • Why GridG?

    • What is GridG?

    • Topology generation

      • Hierarchical vs. degree based?

      • What are the relationships among the power laws of Internet topology?

    • Annotation

      • What are the intra- and inter- correlations among the hosts and within a host?

      • How to build the correlations into GridG?

    • Conclusions and future work



    Current graph generators
    Current Graph generators

    • Random (Waxman)

    • Hierarchical :

      • Tiers, Transit-Stub, etc. have clear network hierarchy, but don’t follow power laws

    • Degree based :

    • Inet, Brite, PLRG, etc. follow power

    • laws, but don’t have clear network

    • hierarchy


    Topology generation in gridg 1 2
    Topology Generation in GridG (1/2)

    • Generate a basic graph without any redundant links using Tiers

      • This is a hierarchical graph

  • Assign each node an outdegree randomly using the outdegree exponent power law as the distribution

    • This enforces all the power laws!

    • Scale-free

  • Determine the remaining outdegree of each node by taking original hierarchical links into consideration


  • Topology generation in gridg 2 2
    Topology Generation in GridG (2/2)

    • Add redundant links between randomly chosen pairs of nodes with sufficient remaining outdegree

      • Nodes at higher levels (e.g., WAN) are given priority over nodes at lower levels (e.g., MAN)

  • Repeat 4 until there is no pair of nodes with positive remaining outdegree


  • Evaluation topology obeys rank exponent law
    Evaluation: Topology Obeys Rank Exponent Law





    Comparing to the internet
    Comparing To The Internet

    Power Law Internet Routers GridG Tiers

    Rank -0.49 -0.51 -0.18

    R2 0.94 0.89

    Outdegree -2.49 -2.63 -3.4

    R2 0.97 0.55

    Eigen -0.18 -0.24 -0.23

    R2 0.97 0.97

    Hop-plot 2.84 2.88 1.64

    R2 0.99 0.99

    Notice Close Match


    Relationship among power laws 0
    Relationship among power laws (0)

    • An interesting phenomenon: GridG and several other graph generators generate graphs according to the outdegree law only. But the generated graphs follow all four power laws!

    • How is this possible?

    • The power laws are closely related

    • Can we deduce other power laws from the outdegree power law?


    Dong lu peter a dinda prescience laboratory department of computer science northwestern university evanston il 6020

    Relationship among power laws (1)

    • Eigenvalue law follows from the outdegree law [Mihail and Papadimitriou]

    • Hop-plot and Eigenvalue power laws are followed by many topologies [Medina, et al]

    • Outdegree law follows from the rank law

    • Rank law does not follow from outdegree law

    • Alternative rank law follows from outdegree law and fits data better

    Our Results


    Relationship among power laws 2
    Relationship among power laws (2)

    Rank law Outdegree law

    This is a power law


    Dong lu peter a dinda prescience laboratory department of computer science northwestern university evanston il 6020

    Relationship among power laws (3)

    Log-log plot of the derived Outdegree law. Perfect power law fit. So we can do Rank law Outdegree law.


    Relationship among power laws 4
    Relationship among power laws (4)

    Outdegree law Rank law

    This is NOT a power law


    Relationship among power laws 5
    Relationship among power laws (5)

    Log-log plot of the derived Rank law. Not power law! So we can NOT do Outdegree law Rank law.

    Corresponds well to the Faloutsos Internet data


    Relationship among power laws 6
    Relationship among power laws (6)

    • Log-log plot of derived Outdegree law using the new Rank law. It is perfect power law.


    Relationship among power laws 7
    Relationship among power laws (7)

    We propose the following as the relationships among Internet topology power laws

    New rank law

    Outdegree power law

    Eigenvalue law


    Outline2
    Outline

    • Why GridG?

    • What is GridG?

    • Topology generation

      • Hierarchical vs. degree based?

      • What are the relationships among the power laws of Internet topology?

    • Annotation

      • What are the intra- and inter- correlations among the hosts and within a host?

      • How to build the correlations into GridG?

    • Conclusions and future work


    Annotation generator
    Annotation Generator

    • Distributions for attributes

      • Example: Smith MDS trace for memory

    • Intra-host correlation of attributes

      • Example: Memory and CPU

    • Inter- host correlations of attributes

      • Example: cluster of identical machines


    Intra host correlations
    Intra-host correlations

    • The Memory size, Architecture, CPU clock rate, Number of CPUs, Disk size, etc, all have certain distributions. These distributions are not independent, however

      • Example: a host with 64 CPUs is likely to have very big memory. Similarly, a host with a 3Ghz processor is likely to have bigger memory than a host with 1Ghz processor

    • Many Intra-host correlations are unknown

    • GridG has heuristic rules and can be extended by the user


    Heuristic intra host rules
    Heuristic Intra-host rules

    • One processor will have memory between 64M and 4G

    • More CPUs, more likely to have bigger memory and disk

    • More memory, more likely to have bigger disk, and vice versa

    • Windows machines won’t have more than 4 processors

    • Machines with different architectures have different distributions of CPU clock rate

    • Host load is not correlated to other attributes.



    Inter host correlations
    Inter-host correlations

    • Hosts that are close to each other are likely to share some attributes.

    • For example: OS concentration

      • Every IP subnet we probed had a dominant OS

    • OS concentration rule built into GridG

      • User can disable


    Annotation algorithm basic
    Annotation Algorithm : Basic

    • Based on the dependence tree, make grid conform to correlations by applying conditional probability

      • Choosing the distribution of an attribute based on attribute picked before it.

    • For example: first choose architecture according to a distribution, then choose the number of CPUs based on it, finally, choose the size of memory based on the previous two choices.


    Annotation algorithm user rules
    Annotation Algorithm: user rules

    • User can add rules to GridG: for example, “all the hosts with N or above processors will have memory bigger than N*1024 MB”, etc.

    • User rules appear as perl functions.

    • User can also configure the distribution of host attributes in the config file.


    Examples silly hosts
    Examples: Silly hosts

    Hosts generated without considering Intra-host

    correlation, each attribute follows its own distribution.


    Examples sensible hosts
    Examples: Sensible hosts

    Hosts generated with considering Intra-host

    correlations.


    Open questions
    Open questions

    • What are the real distributions of host attributes?

    • What are the real intra- and inter-host correlations?

    Difficult to answer without measurement data

    Difficult to acquire measurement data (see paper)

    We would appreciate your help!


    Conclusions
    Conclusions

    • We have presented GridG, a tool kit for generating synthetic computational grids.

    • The topology generation component can produce structured network topologies that obey the power laws of Internet topology.

    • The annotation generation component of GridG is built upon Internet measurements and a set of heuristic rules.


    Conclusions1
    Conclusions

    • While developing GridG’s topology generator, we discovered an interesting relationship among the power laws, and proposed a new one that better fits the data.

    • While measuring the Internet, we found the OS concentration phenomenon and built it into GridG as an user option.


    For more information
    For MoreInformation

    GridG is released online at:

    • http://www.cs.northwestern.edu/~urgis/GridG

    • http://www.cs.northwestern.edu/~urgis

      Related RGIS project papers:

    • “Nondeterministic queries in a Relational Grid Information Service”, In proceedings of SC03.

    • “Scoped and Approximate queries in a Relational Grid Information Service”, In proceedings of Grid2003.