slide1
Download
Skip this Video
Download Presentation
Dong Lu , Peter A. Dinda Prescience Laboratory Department of Computer Science Northwestern University Evanston, IL 60201

Loading in 2 Seconds...

play fullscreen
1 / 39

Dong Lu, Peter A. Dinda Prescience Laboratory Department of Computer Science Northwestern University Evanston, IL 6020 - PowerPoint PPT Presentation


  • 69 Views
  • Uploaded on

GridG: Synthesizing Realistic Computational Grids. Dong Lu , Peter A. Dinda Prescience Laboratory Department of Computer Science Northwestern University Evanston, IL 60201 . Outline. Why GridG? What is GridG? Topology generation Hierarchical vs. degree based?

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Dong Lu, Peter A. Dinda Prescience Laboratory Department of Computer Science Northwestern University Evanston, IL 6020' - nan


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

GridG: Synthesizing Realistic Computational Grids

Dong Lu, Peter A. Dinda

Prescience Laboratory

Department of Computer Science

Northwestern University

Evanston, IL 60201

outline
Outline
  • Why GridG?
  • What is GridG?
  • Topology generation
    • Hierarchical vs. degree based?
    • What are the relationships among the power laws of Internet topology?
  • Annotation
    • What are the intra- and inter- correlations among the hosts and within a host?
    • How to build the correlations into GridG?
  • Conclusions and future work
why gridg
Why GridG?
  • Synthetic Grids needed to evaluate Middleware
      • Existing physical grids too small
      • Can’t control parameters
  • Example: Evaluation of our RGIS system
  • Example: Grid simulation projects
    • GridSim and SimGrid
  • Example: overlay network simulations
    • Application level multicast
gridg a synthetic grid generator
GridG: A Synthetic Grid Generator
  • Output: Network topology annotated with the hardware and software available on each node and link.
    • Layer 3 network: hosts, routers, links
    • Hosts: memory, architecture, number of CPUs, disk, operating system, vendor, clock rate
    • Routers: switching capacity
    • Links: bandwidth and Latency
example 1
Example 1

Router (switching capacity)

Link (bw, latency)

Host (arch, numcpu, clock rate, osvendor, mem, disk,)

requirements
Requirements
  • Realistic topologies
    • Connected
    • Hierarchical topology
    • Power laws of Internet topology
  • Realistic annotations
    • Distributions of attributes
    • Correlations of attributes
      • Intra-host
      • Inter-host
gridg architecture
GridG architecture
  • A sequence of transformations on a text-based representation of an annotated graph.
outline1
Outline
  • Why GridG?
  • What is GridG?
  • Topology generation
    • Hierarchical vs. degree based?
    • What are the relationships among the power laws of Internet topology?
  • Annotation
    • What are the intra- and inter- correlations among the hosts and within a host?
    • How to build the correlations into GridG?
  • Conclusions and future work
current graph generators
Current Graph generators
  • Random (Waxman)
  • Hierarchical :
    • Tiers, Transit-Stub, etc. have clear network hierarchy, but don’t follow power laws
  • Degree based :
  • Inet, Brite, PLRG, etc. follow power
  • laws, but don’t have clear network
  • hierarchy
topology generation in gridg 1 2
Topology Generation in GridG (1/2)
  • Generate a basic graph without any redundant links using Tiers
      • This is a hierarchical graph
  • Assign each node an outdegree randomly using the outdegree exponent power law as the distribution
      • This enforces all the power laws!
      • Scale-free
  • Determine the remaining outdegree of each node by taking original hierarchical links into consideration
topology generation in gridg 2 2
Topology Generation in GridG (2/2)
  • Add redundant links between randomly chosen pairs of nodes with sufficient remaining outdegree
      • Nodes at higher levels (e.g., WAN) are given priority over nodes at lower levels (e.g., MAN)
  • Repeat 4 until there is no pair of nodes with positive remaining outdegree
comparing to the internet
Comparing To The Internet

Power Law Internet Routers GridG Tiers

Rank -0.49 -0.51 -0.18

R2 0.94 0.89

Outdegree -2.49 -2.63 -3.4

R2 0.97 0.55

Eigen -0.18 -0.24 -0.23

R2 0.97 0.97

Hop-plot 2.84 2.88 1.64

R2 0.99 0.99

Notice Close Match

relationship among power laws 0
Relationship among power laws (0)
  • An interesting phenomenon: GridG and several other graph generators generate graphs according to the outdegree law only. But the generated graphs follow all four power laws!
  • How is this possible?
  • The power laws are closely related
  • Can we deduce other power laws from the outdegree power law?
slide19

Relationship among power laws (1)

  • Eigenvalue law follows from the outdegree law [Mihail and Papadimitriou]
  • Hop-plot and Eigenvalue power laws are followed by many topologies [Medina, et al]
  • Outdegree law follows from the rank law
  • Rank law does not follow from outdegree law
  • Alternative rank law follows from outdegree law and fits data better

Our Results

relationship among power laws 2
Relationship among power laws (2)

Rank law Outdegree law

This is a power law

slide21

Relationship among power laws (3)

Log-log plot of the derived Outdegree law. Perfect power law fit. So we can do Rank law Outdegree law.

relationship among power laws 4
Relationship among power laws (4)

Outdegree law Rank law

This is NOT a power law

relationship among power laws 5
Relationship among power laws (5)

Log-log plot of the derived Rank law. Not power law! So we can NOT do Outdegree law Rank law.

Corresponds well to the Faloutsos Internet data

relationship among power laws 6
Relationship among power laws (6)
  • Log-log plot of derived Outdegree law using the new Rank law. It is perfect power law.
relationship among power laws 7
Relationship among power laws (7)

We propose the following as the relationships among Internet topology power laws

New rank law

Outdegree power law

Eigenvalue law

outline2
Outline
  • Why GridG?
  • What is GridG?
  • Topology generation
    • Hierarchical vs. degree based?
    • What are the relationships among the power laws of Internet topology?
  • Annotation
    • What are the intra- and inter- correlations among the hosts and within a host?
    • How to build the correlations into GridG?
  • Conclusions and future work
annotation generator
Annotation Generator
  • Distributions for attributes
    • Example: Smith MDS trace for memory
  • Intra-host correlation of attributes
    • Example: Memory and CPU
  • Inter- host correlations of attributes
    • Example: cluster of identical machines
intra host correlations
Intra-host correlations
  • The Memory size, Architecture, CPU clock rate, Number of CPUs, Disk size, etc, all have certain distributions. These distributions are not independent, however
    • Example: a host with 64 CPUs is likely to have very big memory. Similarly, a host with a 3Ghz processor is likely to have bigger memory than a host with 1Ghz processor
  • Many Intra-host correlations are unknown
  • GridG has heuristic rules and can be extended by the user
heuristic intra host rules
Heuristic Intra-host rules
  • One processor will have memory between 64M and 4G
  • More CPUs, more likely to have bigger memory and disk
  • More memory, more likely to have bigger disk, and vice versa
  • Windows machines won’t have more than 4 processors
  • Machines with different architectures have different distributions of CPU clock rate
  • Host load is not correlated to other attributes.
inter host correlations
Inter-host correlations
  • Hosts that are close to each other are likely to share some attributes.
  • For example: OS concentration
    • Every IP subnet we probed had a dominant OS
  • OS concentration rule built into GridG
    • User can disable
annotation algorithm basic
Annotation Algorithm : Basic
  • Based on the dependence tree, make grid conform to correlations by applying conditional probability
    • Choosing the distribution of an attribute based on attribute picked before it.
  • For example: first choose architecture according to a distribution, then choose the number of CPUs based on it, finally, choose the size of memory based on the previous two choices.
annotation algorithm user rules
Annotation Algorithm: user rules
  • User can add rules to GridG: for example, “all the hosts with N or above processors will have memory bigger than N*1024 MB”, etc.
  • User rules appear as perl functions.
  • User can also configure the distribution of host attributes in the config file.
examples silly hosts
Examples: Silly hosts

Hosts generated without considering Intra-host

correlation, each attribute follows its own distribution.

examples sensible hosts
Examples: Sensible hosts

Hosts generated with considering Intra-host

correlations.

open questions
Open questions
  • What are the real distributions of host attributes?
  • What are the real intra- and inter-host correlations?

Difficult to answer without measurement data

Difficult to acquire measurement data (see paper)

We would appreciate your help!

conclusions
Conclusions
  • We have presented GridG, a tool kit for generating synthetic computational grids.
  • The topology generation component can produce structured network topologies that obey the power laws of Internet topology.
  • The annotation generation component of GridG is built upon Internet measurements and a set of heuristic rules.
conclusions1
Conclusions
  • While developing GridG’s topology generator, we discovered an interesting relationship among the power laws, and proposed a new one that better fits the data.
  • While measuring the Internet, we found the OS concentration phenomenon and built it into GridG as an user option.
for more information
For MoreInformation

GridG is released online at:

  • http://www.cs.northwestern.edu/~urgis/GridG
  • http://www.cs.northwestern.edu/~urgis

Related RGIS project papers:

  • “Nondeterministic queries in a Relational Grid Information Service”, In proceedings of SC03.
  • “Scoped and Approximate queries in a Relational Grid Information Service”, In proceedings of Grid2003.
ad