GridG: Synthesizing Realistic Computational Grids
Download
1 / 39

Dong Lu, Peter A. Dinda Prescience Laboratory Department of Computer Science Northwestern University Evanston, IL 6020 - PowerPoint PPT Presentation


  • 69 Views
  • Uploaded on

GridG: Synthesizing Realistic Computational Grids. Dong Lu , Peter A. Dinda Prescience Laboratory Department of Computer Science Northwestern University Evanston, IL 60201 . Outline. Why GridG? What is GridG? Topology generation Hierarchical vs. degree based?

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Dong Lu, Peter A. Dinda Prescience Laboratory Department of Computer Science Northwestern University Evanston, IL 6020' - nan


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

GridG: Synthesizing Realistic Computational Grids

Dong Lu, Peter A. Dinda

Prescience Laboratory

Department of Computer Science

Northwestern University

Evanston, IL 60201


Outline
Outline

  • Why GridG?

  • What is GridG?

  • Topology generation

    • Hierarchical vs. degree based?

    • What are the relationships among the power laws of Internet topology?

  • Annotation

    • What are the intra- and inter- correlations among the hosts and within a host?

    • How to build the correlations into GridG?

  • Conclusions and future work


Why gridg
Why GridG?

  • Synthetic Grids needed to evaluate Middleware

    • Existing physical grids too small

    • Can’t control parameters

  • Example: Evaluation of our RGIS system

  • Example: Grid simulation projects

    • GridSim and SimGrid

  • Example: overlay network simulations

    • Application level multicast


  • Gridg a synthetic grid generator
    GridG: A Synthetic Grid Generator

    • Output: Network topology annotated with the hardware and software available on each node and link.

      • Layer 3 network: hosts, routers, links

      • Hosts: memory, architecture, number of CPUs, disk, operating system, vendor, clock rate

      • Routers: switching capacity

      • Links: bandwidth and Latency


    Example 1
    Example 1

    Router (switching capacity)

    Link (bw, latency)

    Host (arch, numcpu, clock rate, osvendor, mem, disk,)


    Requirements
    Requirements

    • Realistic topologies

      • Connected

      • Hierarchical topology

      • Power laws of Internet topology

    • Realistic annotations

      • Distributions of attributes

      • Correlations of attributes

        • Intra-host

        • Inter-host


    Gridg architecture
    GridG architecture

    • A sequence of transformations on a text-based representation of an annotated graph.


    Outline1
    Outline

    • Why GridG?

    • What is GridG?

    • Topology generation

      • Hierarchical vs. degree based?

      • What are the relationships among the power laws of Internet topology?

    • Annotation

      • What are the intra- and inter- correlations among the hosts and within a host?

      • How to build the correlations into GridG?

    • Conclusions and future work



    Current graph generators
    Current Graph generators

    • Random (Waxman)

    • Hierarchical :

      • Tiers, Transit-Stub, etc. have clear network hierarchy, but don’t follow power laws

    • Degree based :

    • Inet, Brite, PLRG, etc. follow power

    • laws, but don’t have clear network

    • hierarchy


    Topology generation in gridg 1 2
    Topology Generation in GridG (1/2)

    • Generate a basic graph without any redundant links using Tiers

      • This is a hierarchical graph

  • Assign each node an outdegree randomly using the outdegree exponent power law as the distribution

    • This enforces all the power laws!

    • Scale-free

  • Determine the remaining outdegree of each node by taking original hierarchical links into consideration


  • Topology generation in gridg 2 2
    Topology Generation in GridG (2/2)

    • Add redundant links between randomly chosen pairs of nodes with sufficient remaining outdegree

      • Nodes at higher levels (e.g., WAN) are given priority over nodes at lower levels (e.g., MAN)

  • Repeat 4 until there is no pair of nodes with positive remaining outdegree


  • Evaluation topology obeys rank exponent law
    Evaluation: Topology Obeys Rank Exponent Law





    Comparing to the internet
    Comparing To The Internet

    Power Law Internet Routers GridG Tiers

    Rank -0.49 -0.51 -0.18

    R2 0.94 0.89

    Outdegree -2.49 -2.63 -3.4

    R2 0.97 0.55

    Eigen -0.18 -0.24 -0.23

    R2 0.97 0.97

    Hop-plot 2.84 2.88 1.64

    R2 0.99 0.99

    Notice Close Match


    Relationship among power laws 0
    Relationship among power laws (0)

    • An interesting phenomenon: GridG and several other graph generators generate graphs according to the outdegree law only. But the generated graphs follow all four power laws!

    • How is this possible?

    • The power laws are closely related

    • Can we deduce other power laws from the outdegree power law?


    Relationship among power laws (1)

    • Eigenvalue law follows from the outdegree law [Mihail and Papadimitriou]

    • Hop-plot and Eigenvalue power laws are followed by many topologies [Medina, et al]

    • Outdegree law follows from the rank law

    • Rank law does not follow from outdegree law

    • Alternative rank law follows from outdegree law and fits data better

    Our Results


    Relationship among power laws 2
    Relationship among power laws (2)

    Rank law Outdegree law

    This is a power law


    Relationship among power laws (3)

    Log-log plot of the derived Outdegree law. Perfect power law fit. So we can do Rank law Outdegree law.


    Relationship among power laws 4
    Relationship among power laws (4)

    Outdegree law Rank law

    This is NOT a power law


    Relationship among power laws 5
    Relationship among power laws (5)

    Log-log plot of the derived Rank law. Not power law! So we can NOT do Outdegree law Rank law.

    Corresponds well to the Faloutsos Internet data


    Relationship among power laws 6
    Relationship among power laws (6)

    • Log-log plot of derived Outdegree law using the new Rank law. It is perfect power law.


    Relationship among power laws 7
    Relationship among power laws (7)

    We propose the following as the relationships among Internet topology power laws

    New rank law

    Outdegree power law

    Eigenvalue law


    Outline2
    Outline

    • Why GridG?

    • What is GridG?

    • Topology generation

      • Hierarchical vs. degree based?

      • What are the relationships among the power laws of Internet topology?

    • Annotation

      • What are the intra- and inter- correlations among the hosts and within a host?

      • How to build the correlations into GridG?

    • Conclusions and future work


    Annotation generator
    Annotation Generator

    • Distributions for attributes

      • Example: Smith MDS trace for memory

    • Intra-host correlation of attributes

      • Example: Memory and CPU

    • Inter- host correlations of attributes

      • Example: cluster of identical machines


    Intra host correlations
    Intra-host correlations

    • The Memory size, Architecture, CPU clock rate, Number of CPUs, Disk size, etc, all have certain distributions. These distributions are not independent, however

      • Example: a host with 64 CPUs is likely to have very big memory. Similarly, a host with a 3Ghz processor is likely to have bigger memory than a host with 1Ghz processor

    • Many Intra-host correlations are unknown

    • GridG has heuristic rules and can be extended by the user


    Heuristic intra host rules
    Heuristic Intra-host rules

    • One processor will have memory between 64M and 4G

    • More CPUs, more likely to have bigger memory and disk

    • More memory, more likely to have bigger disk, and vice versa

    • Windows machines won’t have more than 4 processors

    • Machines with different architectures have different distributions of CPU clock rate

    • Host load is not correlated to other attributes.



    Inter host correlations
    Inter-host correlations

    • Hosts that are close to each other are likely to share some attributes.

    • For example: OS concentration

      • Every IP subnet we probed had a dominant OS

    • OS concentration rule built into GridG

      • User can disable


    Annotation algorithm basic
    Annotation Algorithm : Basic

    • Based on the dependence tree, make grid conform to correlations by applying conditional probability

      • Choosing the distribution of an attribute based on attribute picked before it.

    • For example: first choose architecture according to a distribution, then choose the number of CPUs based on it, finally, choose the size of memory based on the previous two choices.


    Annotation algorithm user rules
    Annotation Algorithm: user rules

    • User can add rules to GridG: for example, “all the hosts with N or above processors will have memory bigger than N*1024 MB”, etc.

    • User rules appear as perl functions.

    • User can also configure the distribution of host attributes in the config file.


    Examples silly hosts
    Examples: Silly hosts

    Hosts generated without considering Intra-host

    correlation, each attribute follows its own distribution.


    Examples sensible hosts
    Examples: Sensible hosts

    Hosts generated with considering Intra-host

    correlations.


    Open questions
    Open questions

    • What are the real distributions of host attributes?

    • What are the real intra- and inter-host correlations?

    Difficult to answer without measurement data

    Difficult to acquire measurement data (see paper)

    We would appreciate your help!


    Conclusions
    Conclusions

    • We have presented GridG, a tool kit for generating synthetic computational grids.

    • The topology generation component can produce structured network topologies that obey the power laws of Internet topology.

    • The annotation generation component of GridG is built upon Internet measurements and a set of heuristic rules.


    Conclusions1
    Conclusions

    • While developing GridG’s topology generator, we discovered an interesting relationship among the power laws, and proposed a new one that better fits the data.

    • While measuring the Internet, we found the OS concentration phenomenon and built it into GridG as an user option.


    For more information
    For MoreInformation

    GridG is released online at:

    • http://www.cs.northwestern.edu/~urgis/GridG

    • http://www.cs.northwestern.edu/~urgis

      Related RGIS project papers:

    • “Nondeterministic queries in a Relational Grid Information Service”, In proceedings of SC03.

    • “Scoped and Approximate queries in a Relational Grid Information Service”, In proceedings of Grid2003.


    ad