the science of complex networks and the internet lies damned lies and statistics
Download
Skip this Video
Download Presentation
The Science of Complex Networks and the Internet: Lies, Damned Lies, and Statistics

Loading in 2 Seconds...

play fullscreen
1 / 78

The Science of Complex Networks and the Internet: Lies, Damned Lies, and Statistics - PowerPoint PPT Presentation


  • 119 Views
  • Uploaded on

The Science of Complex Networks and the Internet: Lies, Damned Lies, and Statistics. Walter Willinger AT&T Labs-Research [email protected] Outline. The Science of Complex Networks (“Network Science”) What “Network Science” has to say about the Internet

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' The Science of Complex Networks and the Internet: Lies, Damned Lies, and Statistics' - chava-mendoza


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
the science of complex networks and the internet lies damned lies and statistics

The Science of Complex Networks and the Internet:Lies, Damned Lies, and Statistics

Walter Willinger

AT&T Labs-Research

[email protected]

outline
Outline
  • The Science of Complex Networks (“Network Science”)
  • What “Network Science” has to say about the Internet
  • What “engineering” has to say about the Internet
  • Engineered vs. random network models
  • Implications
acknowledgments
Acknowledgments
  • John Doyle (Caltech)
  • David Alderson (Naval Postgraduate School)
  • Steven Low (Caltech)
  • Yin Zhang (Univ. of Texas at Austin)
  • Matthew Roughan (U. Adelaide, Australia)
  • Anja Feldmann (TU Berlin)
  • Lixia Zhang (UCLA)
  • Reza Rejaie (Univ. of Oregon)
  • Mauro Maggioni (Duke Univ.)
  • Bala Krishnamurthy, Alex Gerber, Shubho Sen, Dan Pai (AT&T)
  • … and many of their students and postdocs
slide4

NETWORK SCIENCE

January, 2006

  • “First, networks lie at the core of the economic, political, and social fabric of the 21st century.”
  • “Second, the current state of knowledge about the structure, dynamics, and behaviors of both large infrastructure networks and vital social networks at all scales is primitive.”
  • “Third, the United States is not on track to consolidate the information that already exists about the science of large, complex networks, much less to develop the knowledge that will be needed to design the networks envisaged…”

http://www.nap.edu/catalog/11516.html

network science in theory
“Network Science” in Theory …
  • What?

“The study of network representations of physical, biological, and social phenomena leading to predictive models of these phenomena.” (National Research Council Report, 2006)

  • Why?

“To develop a body of rigorous results that will improve the predictability of the engineering design of complex networks and also speed up basic research in a variety of applications areas.” (National Research Council Report, 2006)

  • Who?
    • Physicists (statistical physics), mathematicians (graph theory), computer scientists (algorithm design), etc.
basic questions ask by network scientists
Basic Questions ask by Network Scientists

Question 1

To what extent does there exist a “network structure” that is

responsible for large-scale properties in complex systems?

  • Performance
  • Robustness
  • Adaptability / Evolvability
  • “Complexity”
basic questions ask by network scientists cont
Basic Questions ask by Network Scientists (cont.)

Question 2

Are there “universal laws” governing the structure (and

resulting behavior) of complex networks? To what extent is

self-organization responsible for the emergence of system

features not explained from a traditional (i.e., reductionist)

viewpoint?

basic questions ask by network scientists cont1
Basic Questions ask by Network Scientists (cont.)

Question 3

How can one assess the vulnerabilities or fragilities

inherent in these complex networks in order to avoid

“rare yet catastrophic” disasters? More practically,

how should one design, organize, build, and manage

complex networks?

observation
Observation
  • The questions motivating recent work in “Network Science” are “the right questions”
    • network structure and function
    • technological, social, and biological
  • The issue is whether or not “Network Science” in its current form has been successful in providing scientifically solid answers to these (and and other) questions.
  • Our litmus test for examining this issue
    • Applications of the current “Network Science” approach to real systems of interest (e.g., Internet)
as scientists why should we care
As scientists, why should we care?
  • “Network Science” as a new scientific discipline …
slide11
Publications in Network Science Literature by Discipline(As recorded by the Web of Science1 on October 1, 2007; courtesy D. Alderson)
slide12

Most Cited Publications in Network Science Literature (As recorded by the Web of Science1 on October 1, 2007; courtesy D. Alderson)

13279

as scientists why should we care1
As scientists, why should we care?
  • “Network Science” as a new scientific discipline …
  • “Network Science” for the masses …
as scientists why should we care2
As scientists, why should we care?
  • “Network Science” as a new scientific discipline …
  • “Network Science” for the masses …
  • “Network Science” for the (Internet) experts …
as scientists why should we care3
As scientists, why should we care?
  • “Network Science” as a new scientific discipline …
  • “Network Science” for the masses …
  • “Network Science” for the Internet experts …
  • “Network Science” for undergraduate/graduate students in Computer Science/Electrical Engineering
slide18

The “New Science of Networks”

  • New course offerings
    • http://www.cc.gatech.edu/classes/AY2010/cs8803ns_fall/
    • http://www.netscience.usma.edu/about.php
    • http://nicomedia.math.upatras.gr/courses/mnets/index_en.html
    • http://www-personal.umich.edu/~mejn/courses/2004/cscs535/index.html
    • http://www.phys.psu.edu/~ralbert/phys597_09-fall
as scientists why should we care4
As scientists, why should we care?
  • “Network Science” as a new scientific discipline …
  • “Network Science” for the masses …
  • “Network Science” for the Internet experts …
  • “Network Science” for undergraduate/graduate students in Computer Science/Electrical Engineering
  • … and most importantly, because we want to know how serious a science “Network Science” is ….
the main points of this talk
The Main Points of this Talk …

I will show that in the case of the Internet …

The application of “Network Science” in its current form has led to conclusions that are not controversial but simply wrong.

I will deconstruct the existing arguments and generalize the potential pitfalls common to “Network Science.”

I will also be constructive and illustrate an alternative approach to “Network Science” based on engineering considerations.

what does network science say about the internet
What does “Network Science” say about the Internet
  • Illustration with a case study
    • Problem: Internet router-level topology
    • Approach: Measurement-based
    • Result: Predictive models with far-reaching implications
  • Textbook example for the power of “Network Science”
    • Appears solid and rigorous
    • Appealing approach with surprising findings
    • Directly applicable to other domains
what does network science say about the internet1
What does “Network Science” say about the Internet
  • Measurement technique
    • traceroutetool
    • traceroute discovers compliant (i.e., IP) routers along path between selected network host computers
running traceroute basic experiment
Running traceroute: Basic Experiment
  • Basic “experiment”
    • Select a source and destination
    • Run traceroute tool
  • Example
    • Run traceroute from my machine in Florham Park, NJ, USA to www.duke.edu
running traceroute www duke edu from nj
Running “traceroute www.duke.edu” from NJ
  • 1 fp-core.research.att.com (135.207.16.1) 2 ms 1 ms 1 ms
  • 2 ngx19.research.att.com (135.207.1.19) 1 ms 0 ms 0 ms
  • 3 12.106.32.1 1 ms 1 ms 1 ms
  • 4 12.119.12.73 2 ms 2 ms 2 ms
  • 5 tbr1.n54ny.ip.att.net (12.123.219.129) 4 ms 5 ms 3 ms
  • 6 ggr7.n54ny.ip.att.net (12.122.88.21) 3 ms 3 ms 3 ms
  • 7 192.205.35.98 4 ms 4 ms 8 ms
  • 8 jfk-core-02.inet.qwest.net (205.171.30.5) 3 ms 3 ms 4 ms
  • 9 dca-core-01.inet.qwest.net (67.14.6.201) 11 ms 11 ms 11 ms
  • 10 dca-edge-04.inet.qwest.net (205.171.9.98) 11 ms 15 ms 11 ms
  • 11 gw-dc-mcnc.ncren.net (63.148.128.122) 18 ms 18 ms 18 ms
  • 12 rlgh7600-gw-to-rlgh1-gw.ncren.net (128.109.70.38) 18 ms 18 ms 18 ms
  • 13 roti-gw-to-rlgh7600-gw.ncren.net (128.109.70.18) 20 ms 20 ms 20 ms
  • 14 art1sp-tel1sp.netcom.duke.edu (152.3.219.118) 23 ms 20 ms 20 ms
  • 15 webhost-lb-01.oit.duke.edu (152.3.189.3) 21 ms 38 ms 20 ms
what does network science say about the internet2
What does “Network Science” say about the Internet
  • Measurement technique
    • traceroutetool
    • traceroute discovers compliant (i.e., IP) routers along path between selected network host computers
  • Available data: from large-scale traceroute experiments
    • Pansiot and Grad (router-level, around 1995, France)
    • Cheswick and Burch (mapping project 1997--, Bell-Labs)
    • Mercator (router-level, around 1999, USC/ISI)
    • Skitter (ongoing mapping project, CAIDA/UCSD)
    • Rocketfuel (state-of-the-art router-level maps of individual ISPs, UW Seattle)
    • Dimes (ongoing EU project)
slide30

http://www.cs.washington.edu/research/networking/rocketfuel/bbhttp://www.cs.washington.edu/research/networking/rocketfuel/bb

what does network science say about the internet cont
What does “Network Science” say about the Internet (cont.)
  • Inference
    • Given: traceroute-based map (graph) of the router-level Internet (Internet service provider)
    • Wanted: Metric/statistics that characterizes the inferred connectivity maps
    • Main metric: Node degree distribution
what does network science say about the internet cont1
What does “Network Science” say about the Internet (cont.)
  • Inference
    • Given: traceroute-based map (graph) of the router-level Internet (Internet service provider)
    • Wanted: Metric/statistics that characterizes the inferred connectivity maps
    • Main metric: Node degree distribution
  • Surprising finding
    • Inferred node degree distributions follow a power law
    • A few nodes have a huge degree, while the majority of nodes have a small degree
power laws and internet topology
Power Laws and Internet Topology

A few nodes have lots of connections

Source: Faloutsos et al (1999)

Most nodes have few connections

what does network science say about the internet cont2
What does “Network Science” say about the Internet (cont.)
  • Inference
    • Given: traceroute-based map (graph) of the router-level Internet (Internet service provider)
    • Wanted: Metric/statistics that characterizes the inferred connectivity maps
    • Main metric: Node degree distribution
  • Surprising finding
    • Inferred node degree distributions follow a power law
    • A few nodes have a huge degree, while the majority of nodes have a small degree
  • Motivation for developing new network/graph models
    • Dominant graph models: Erdos-Renyi random graphs
    • But: Node degrees of Erdos-Renyi random graph models follow a Poisson distribution
what does network science say about the internet cont3
What does “Network Science” say about the Internet (cont.)
  • New class of network models
    • Preferential attachment (PA) growth model
      • Incremental growth: New nodes/links are added one at a time
      • Preferential attachment: a new node is more likely to connect to an already highly connected node (p(k)  degree of node k)
    • Captures popular notion of “the rich get richer”
    • There exist many variants of this basic PA model
    • Generally referred to as “scale-free” network models
  • Key features of PA-type network models
    • Randomness enters via attachment mechanism
    • Exhibit power law node degree distributions
what does network science say about the internet cont4
What does “Network Science” say about the Internet (cont.)
  • Model validation
    • The models “fit the data” because they reproduce the observed node degree distributions
    • The models are simple and parsimonious
  • PA-type models have resulted in highly publicized claims about the Internet and its properties
    • High-degree nodes form a hub-like core
    • Fragile/vulnerable to targeted node removal
    • Achilles’ heel
    • Zero epidemic threshold
beyond the internet
Beyond the Internet …
  • Social networks
  • Information networks
  • Biological networks
  • Technological networks
    • U.S. electrical power grid (data source: FEMA)
beyond the internet1
Beyond the Internet …
  • Social networks
  • Information networks
  • Biological networks
  • Technological networks
    • U.S. electrical power grid (data source: FEMA)
    • Western U.S. power grid: 4921 nodes, 6594 links
    • J.-W. Wang and L.-L. Rong, “Cascade-based attack vulnerability on the US power grid,”Safety Science 47, 2009
    • NYT article, April 18, 2010: “Academic paper in China sets off alarms in U.S.”
  • Interdependent networks (e.g., Internet and power grid)
    • S.V. Buldyrev, R. Parshani, G. Paul, H.E. Stanley and S. Havlin, “Catastrophic cascade of failures in interdependent networks”, Nature 464 (April 2010)
on the impact of network science
On the Impact of “Network Science” …
  • On the scientific community as a whole
    • General excitement (huge number of papers)
    • The Internet story has been repeated in the context of biological networks, social networks, etc.
    • Renewed hope that large-scale complex networks across the domains (e.g., engineering, biology, social sciences) exhibit common features (universal properties).
on the impact of network science1
On the Impact of “Network Science” …
  • On the scientific community as a whole
    • General excitement (huge number of papers)
    • The Internet story has been repeated in the context of biological networks, social networks, etc.
    • Renewed hope that large-scale complex networks across the domains (e.g., engineering, biology, social sciences) exhibit common features (universal properties).
  • On domain experts (e.g., Internet researchers, biologists)
    • General disbelief
    • We “know” the claims are not true …
    • Back to basics ….
slide48
Basic Question

Do the available Internet-related connectivity

measurements and their analysis support the sort

of claims that can be found in the existing complex

networks literature?

  • Key Issues
  • What about data hygiene?
  • What about statistical rigor?
  • What about model validation?
on measuring internet connectivity
On Measuring Internet Connectivity
  • No central agency/repository
  • Economic incentive for ISPs to obscure network structure
  • Direct inspection is typically not possible
  • Based on measurement experiments, hacks
  • Mismatch between what we want to measure and can measure
  • Specific examples covered in this talk
    • Physical connectivity (routers, switched, links)
measurements traceroute tool
Measurements: traceroute tool
  • traceroute www.duke.edu
    • traceroute to www.duke.edu (152.3.189.3), 30 hops max, 60 byte packets
    • 1 fp-core.research.att.com (135.207.16.1) 2 ms 1 ms 1 ms
    • 2 ngx19.research.att.com (135.207.1.19) 1 ms 0 ms 0 ms
    • 3 12.106.32.1 1 ms 1 ms 1 ms
    • 4 12.119.12.73 2 ms 2 ms 2 ms
    • 5 tbr1.n54ny.ip.att.net (12.123.219.129) 4 ms 5 ms 3 ms
    • 6 ggr7.n54ny.ip.att.net (12.122.88.21) 3 ms 3 ms 3 ms
    • 7 192.205.35.98 4 ms 4 ms 8 ms
    • 8 jfk-core-02.inet.qwest.net (205.171.30.5) 3 ms 3 ms 4 ms
    • 9 dca-core-01.inet.qwest.net (67.14.6.201) 11 ms 11 ms 11 ms
    • 10 dca-edge-04.inet.qwest.net (205.171.9.98) 11 ms 15 ms 11 ms
    • 11 gw-dc-mcnc.ncren.net (63.148.128.122) 18 ms 18 ms 18 ms
    • 12 rlgh7600-gw-to-rlgh1-gw.ncren.net (128.109.70.38) 18 ms 18 ms 18 ms
    • 13 roti-gw-to-rlgh7600-gw.ncren.net (128.109.70.18) 20 ms 20 ms 20 ms
    • 14 art1sp-tel1sp.netcom.duke.edu (152.3.219.118) 23 ms 20 ms 20 ms
    • 15 webhost-lb-01.oit.duke.edu (152.3.189.3) 21 ms 38 ms 20 ms
traceroute measurements revisited 1
Traceroute measurements revisited (1)
  • traceroute is strictly about IP-level connectivity
    • Originally developed by Van Jacobson (1988)
    • Designed to trace out the route to a host
  • Using traceroute to map the router-level topology
    • Engineering hack
    • Example of what we can measure, not what we want to measure!
  • Basic problem #1: IP alias resolution problem
    • How to map interface IP addresses to IP routers
    • Largely ignored or badly dealt with in the past
    • New efforts in 2008 for better heuristics …
traceroute measurements revisited 2
Traceroute measurements revisited (2)
  • traceroute is strictly about IP-level connectivity
  • Basic problem #2: Layer-2 technologies (e.g., MPLS, ATM)
    • MPLS is an example of a circuit technology that hides the network’s physical infrastructure from IP
    • Sending traceroutes through an opaque Layer-2 cloud results in the “discovery” of high-degree nodes, which are simply an artifact of an imperfect measurement technique.
    • This problem has been largely ignored in all large-scale traceroute experiments to date.
slide57

(a)

(b)

traceroute measurements revisited 3
Traceroute measurements revisited (3)
  • The irony of traceroute measurements
    • The high-degree nodes in the middle of the network that traceroute reveals are not for real …
    • If there are high-degree nodes in the network, they can only exist at the edge of the network where they will never be revealed by generic traceroute-based experiments …
  • Additional sources of errors
    • Bias in (mathematical abstraction of) traceroute
    • Has been a major focus within CS/Networking literature
    • Non-issue in the presence of above-mentioned problems
traceroute measurements revisited 4
Traceroute measurements revisited (4)
  • Bottom line
    • (Current) traceroute measurements are of little use for inferring router-level connectivity
    • It is unlikely that future traceroute measurements will be more useful for the purpose of router-level inference
  • Lessons learned
    • Key question: Can you trust the available data?
    • Critical role of Data Hygiene in the Petabyte Age
    • Corollary: Petabytes of garbage = garbage
    • Data hygiene is often viewed as “dirty/unglamorous” work
taking model validation more serious
Taking Model validation more serious …
  • Criticism of conventional model validation
    • For one and the same observed phenomenon, there are usually many different explanations/models
    • The ability to reproduce a few graph statistics does not constitute “serious” model validation
    • Model validation should be more than “data fitting”
  • What constitutes “more serious” model validation?
    • There is more to networks than connectivity …
    • When “nodes” and “links” have specific meaning …
    • What do real networks look like?
cisco 12000 series routers
Cisco 12000 Series Routers
  • Modular in design, creating flexibility in configuration.
  • Router capacity is constrained by the number and speed of line cards inserted in each slot.

Source: www.cisco.com

router technology constraint

3

10

high BW low degree

high degree low BW

2

10

1

10

Bandwidth (Gbps)

15 x 1-port 10 GE

15 x 3-port 1 GE

0

10

15 x 4-port OC12

15 x 8-port FE

Technology constraint

-1

10

Degree

0

1

2

10

10

10

Router Technology Constraint

Cisco 12416 GSR, circa 2002

Total Bandwidth

slide65

Intermountain

GigaPoP

U. Memphis

Indiana GigaPoP

WiscREN

OARNET

Great Plains

Front Range

GigaPoP

U. Louisville

NYSERNet

Arizona St.

Qwest Labs

Iowa St.

Star-

Light

U. Arizona

CHECS-NET

Oregon

GigaPoP

WPI

Pacific

Wave

Pacific

Northwest

GigaPoP

SINet

SURFNet

ESnet

MANLAN

U. Hawaii

NREN

NISN

GEANT

USGS

UniNet

MAGPI

DREN

CENIC

Northern

Crossroads

0.1-0.5 Gbps

0.5-1.0 Gbps

1.0-5.0 Gbps

5.0-10.0 Gbps

TransPAC/APAN

Tulane U.

LaNet

PSC

SOX

UMD NGIX

North Texas

GigaPoP

Drexel U.

DARPA

BossNet

Texas

GigaPoP

Mid-Atlantic

Crossroads

Texas Tech

SFGP/

AMPATH

Miss State

GigaPoP

UT Austin

NCNI/MCNC

U. Florida

UT-SW

Med Ctr.

U. So. Florida

Jackson St.

Florida A&M

U. So. Miss.

Northern Lights

Merit

OneNet

Kansas

City

Indian-

apolis

Denver

Chicago

Seattle

New York

Wash

D.C.

Sunnyvale

Los Angeles

Atlanta

Houston

Abilene Backbone

Physical Connectivity

(as of August 2004)

slide66

OC-3 (155 Mb/s)

OC-12 (622 Mb/s)

GE (1 Gb/s)

OC-48 (2.5 Gb/s)

10GE (10 Gb/s)

Cisco 750X

Cisco 12008

Cisco 12410

Abilene

Los Angeles

Abilene

Sunnyvale

CENIC Backbone (as of January 2004)

The Corporation for Education Network Initiatives in California (CENIC) acts as ISP for the state\'s colleges and universities

http://www.cenic.org

COR

dc1

dc1

dc2

OAK

dc2

SAC

hpr

dc1

hpr

FRG

dc2

dc1

hpr

dc1

dc3

SVL

FRE

dc1

  • Like Abilene, its backbone is a sparsely-connected mesh, with relatively low connectivity and minimal redundancy.
  • no high-degree hubs?
  • no Achilles’ heel?

SOL

dc1

BAK

dc1

SLO

hpr

dc1

hpr

LAX

dc2

dc3

dc1

TUS

dc1

SDG

hpr

dc3

dc1

slide68
Back to the Basic Question:

Do the available Internet-related connectivity measurements

and their analysis support the sort of claims that can be

found in the existing complex networks literature?

Short Answer: No!

  • Longer Answer:
  • Real-world router-level topologies look nothing like PA-type networks
  • The results derived from PA-type models of the Internet are not “controversial” – they are simply wrong!
  • “The tragedy of science – the slaying of a beautiful hypothesis by an ugly fact.” (T. Huxley)
what went wrong
What Went Wrong?
  • No critical assessment of available data
  • Ignore all networking-related “details”
    • Randomness enters via generic attachment mechanism
    • Overarching desire to reproduce power law node degree distributions
  • Low model validation standards
    • Reproducing observed node degree distribution
how to avoid such fallacies
How to avoid such Fallacies?
  • Know your data!
  • Take model validation more serious!
  • Apply an engineering perspective to engineered systems!
internet modeling an engineering perspective
Internet Modeling: An Engineering Perspective
  • Surely, the way an ISP designs its physical infrastructure is not the result of a series of coin tosses …
    • ISPs design their router-level topology for a purpose, namely to carry an expected traffic demand
    • Randomness enters in terms of uncertainty in traffic demands
    • ISPs are constrained in what they can afford to build, operate, and maintain (technology, economics).
  • Decisions of ISPs are driven by objectives (performance) and reflect tradeoffs between what is feasible and what is desirable (heuristic optimization)
    • Constrained optimization as modeling language
    • Power law node degrees are a non-issue!
heuristically optimized topologies hot

Step 1: Constrain to be feasible

Step 2: pick traffic demand model

1000000

100000

Bj

10000

Abstracted Technologically Feasible Region

1000

Bandwidth (Mbps)

Step 3: Compute max flow

xij

100

10

Bi

degree

1

10

100

1000

Heuristically Optimized Topologies (HOT)

Given realistic technology constraints on routers, how well is the network able to carry traffic?

hot design principles

Hosts

HOT Design Principles

Mesh-like core of fast, low degree routers

Cores

High degree nodes are at the edges.

Edges

slide74

102

node rank

101

100

101

node degree

Preferential Attachment

HOT model

slide75

HOT- vs. PA-type Network Models

Features

HOT-type/ Internet

PA-type models

Core nodes

Fast, low degree

Slow, high degree

High degree nodes

Edge

Core

Degree distribution

Highly Variable

Power law

Generation

Designed

Random

Performance

High throughput

Low throughput

Attack Tolerance

Robust

Fragile

Fragility

Hijack network

Attack hubs

implications of this engineering perspective
Implications of this Engineering Perspective
  • Important lessons learned
    • Know your data! – they typically reflect what we can measure rather than what we would like to measure
    • Avoid the allure of PA-type network models! – there exist more relevant, interesting, and rewarding network models that await discovery
    • Details do matter! – layers, protocols, feedback control, etc.
  • Network resilience – more than “knocking out” nodes/links
    • NYC 9/11/2001, Baltimore tunnel fire (July 2001)
    • Eastern US/Canada blackout (August 2003)
    • Taiwan earthquake (December 2006)
    • Hijack BGP (“blackholing”, YouTube and Pakistan ISP, 2008)
and always keep in mind
And always keep in mind …

“When exactitude is elusive, it is better to be approximately right than certifiably wrong.”

(B.B. Mandelbrot)

some related references
SOME RELATED REFERENCES
  • L. Li, D. Alderson, W. Willinger, and J. Doyle, A first-principles approach to understanding the Internet’s router-level topology,Proc. ACM SIGCOMM 2004.
  • J.C. Doyle, D. Alderson, L. Li, S. Low, M. Roughan, S. Shalunov, R. Tanaka, and W. Willinger. The "robust yet fragile" nature of the Internet.PNAS 102(41), 2005.
  • D. Alderson, L. Li, W. Willinger, J.C. Doyle. Understanding Internet Topology: Principles, Models, and Validation.ACM/IEEE Trans. on Networking 13(6), 2005.
  • R. Oliveira, D. Pei, W. Willinger, B. Zhang, L. Zhang. In Search of the elusive Ground Truth: The Internet\'s AS-level Connectivity Structure.Proc. ACM SIGMETRICS 2008.
  • B. Krishnamurthy and W. Willinger.What are our standards for validation of measurement-based networking research? Proc. ACM HotMetrics Workshop 2008.
  • W. Willinger, D. Alderson, and J.C. Doyle. Mathematics and the Internet: A Source of Enormous Confusion and Great Potential. Notices of the AMS, Vol. 56, No. 2, 2009. Reprinted in: The Best Writing on Mathematics, Princeton University Press, 2010.
  • M. Roughan, W. Willinger, O. Maennel, D. Perouli, and R. Bush. 10 Lessons from 10 years of measuring and modeling the Internet’s Autonomous Systems. IEEE JSAC Special Issue on “Measurement of Internet topologies,” 2011.
ad