Towards a transparent and proactively managed internet
This presentation is the property of its rightful owner.
Sponsored Links
1 / 26

Towards a Transparent and Proactively-Managed Internet PowerPoint PPT Presentation


  • 84 Views
  • Uploaded on
  • Presentation posted in: General

Towards a Transparent and Proactively-Managed Internet. Ehab Al-Shaer School of Computer Science DePaul University. Yan Chen EECS Department Northwestern University. Motivations. The Internet has evolved to become a un-cooperative ossificated network of networks

Download Presentation

Towards a Transparent and Proactively-Managed Internet

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Towards a transparent and proactively managed internet

Towards a Transparent and Proactively-Managed Internet

Ehab Al-Shaer

School of Computer Science

DePaul University

Yan Chen

EECS Department

Northwestern University


Motivations

Motivations

  • The Internet has evolved to become a un-cooperative ossificated network of networks

    • Network has to be treated as a blackbox

      • Performance of even neighboring networks are opaque

      • Inter-domain routing based on policies but not performance

      • Have to resort to overlay networks which are suboptimal

    • Diagnosis and fault location extremely hard

  • Network config management reactive and expensive

    • Reactive configurations: tune after deployment

    • Vulnerable: manually handled and subject to conflicts

    • Imperative & fragmented: need to access several specific devices in order to implement a service goal


Proposed solution i transparent internet

Proposed Solution I: Transparent Internet

  • Every network shares its measurement and management information with other networks when necessary (glass box)

    • Link-level performance: delay, loss rate, available bandwidth, etc.

    • Management info

      • Configuration: QoS setting, traffic policing

      • Middle box settings: firewalls, etc.

  • The information sharing

    • As part of the inter-domain protocols: Transparent Gateway Protocols (TGP)

    • Other applications: leverage DHT


Analogy to the airline alliance

Analogy to the Airline Alliance

  • When airlines compose multi-lag flights, they need more than just route info.

    • Type of aircraft, # of vacancies, probability of punctuation, etc.

  • Such open model is mutual beneficial

    • Provide the best flight composition for clients

    • Similarly, open network model can provide best communications for applications


Proposed solution ii proactive configuration management

Proposed Solution II: Proactive Configuration Management

  • Proactive verification: configuration verified and translated to different vendor specific devices

  • Proactive validation: Test the configuration changes on the real archived network traffic without interrupting the main operation network

  • Autonomic configuration: configurations are auto-tuned dynamically to achieve the “objectives

Dynamic Validation: auto-tuning

Deploying

Optimizing

Evaluating

defining

Verifying

Validation


Objectives

Objectives

Provides a completely transparent view of the Internet to networks and applications

  • Diagnosis & trouble shooting becomes extremely easy

    • No more Internet tomography needed

  • Flexible inter-domain routing

    • Not just based on policy or # of AS/hops

    • Flexible metrics based on bandwidth, latency, etc.

  • Global traffic engineering

    • Each AS performs its own local traffic engineering

    • Provide AS path-level routing guide

  • Unified framework that applications query (push/pull) info as needed

    • Streaming media, content distribution

    • Anomaly/security applications


Flexible inter domain routing

Flexible Inter-domain Routing

  • Multiple routing paths with TGP

    • Incorporate measurement info into AS paths

    • Bandwidth-intensive and latency-intensive applications can take different AS paths.

  • Challenge: inter-domain routing based on bandwidth without making reservation

  • Solution: Discretize the bandwidth for better stability

    • Though stability is a classical problem, not unique to TGP


Global traffic engineering

Global Traffic Engineering

  • For the current Internet, only local optimum is achieved in each AS

    • Allowing the network to handle all traffic patterns possible, within the networks ingress-egress capacity constraints (e.g. two phase routing)

  • With global information, we can potentially achieve global optimum (or Nash equilibrium)

    • Each AS is a selfish individual

    • A center (or each AS) infers the Nash equilibrium

    • Each AS can try the Nash equilibrium, or attempt to benefit itself based on the inferred Nash equilibrium


Example of benefit of global te

2G

2G

1G

2G

Example of Benefit of Global TE

1G traffic to AS 1

AS 4

AS 2

1G

AS 5

AS 1

1G traffic to AS 1

AS 3


Example of benefit of global te1

0.5G

0.5G

0.5G

1G

1.5G

2G

1G

2G

2G

Example of Benefit of Global TE

  • Without Global TE

1G traffic to AS 1

AS 4

AS 2

1G

AS 5

AS 1

1G traffic to AS 1

AS 3


Example of benefit of global te2

1G

1G

1G

1G

2G

1G

2G

2G

Example of Benefit of Global TE

  • With Global TE

1G traffic to AS 1

AS 4

AS 2

1G

AS 5

AS 1

1G traffic to AS 1

AS 3


Unified transparency framework for various functionality

Unified Transparency Framework for Various Functionality

  • Sharing of anomaly/security-related measurement

    • Various characteristics of traffic: heavy hitter, heavy changes, histogram, etc.

    • Self-diagnosis to survivability

  • Adaptations

    • Routing adaptations at router level or application level


Practical issues and solutions

Practical Issues and Solutions

  • Incentives for information sharing

    • Mandatory for next-generation Internet ?

    • Alliance model for incremental growth

  • Security/cheating: Trust but verify

    • Trust most of the info shared but periodically verify

      • Much easier than the current Internet tomography unless many ASes collude

    • Verification part of the protocol

      • Some fields in the packet headers designed for that purpose


Backup materials

Backup Materials


Measurement info to share

Measurement Info to Share

  • Basic metrics

    • Delay, loss rate, capacity, available bandwidth

    • Demand (or traffic volume) and application types

  • Intra-AS Measurement Info

    • Link-level info

      • Queried only when necessary

    • Aggregated Info

      • OD flow level info

      • Path segment b/t entry and exit points in each AS

  • Inter-AS Measurement Info

    • General AS relationship

    • AS-level topology

    • Inter-AS link metrics


Transparent internet architecture

Transparent Internet Architecture

Combined w/ routing info and

export to neighboring ASes

through TGP protocol

Provide global retrievable

Management Information Base (MIB)

with DHT

Network link-level monitoring


Methodology

iterate

Algorithm design

Realistic simulation

Methodology

  • Network topology

  • Web workload

  • Network end-to-end latency measurement

Analytical evaluation

PlanetLab tests


Tgp mib dissemination architecture

replica

cache

always update

adaptive

coherence

client

DHT mesh

TGP MIB Dissemination Architecture

  • Leverage Distributed Hash Table - Tapestry for

    • Distributed, scalable location with guaranteed success

    • Search with locality

data

source

data plane

Dynamic Replication/Update

and Replica Management

Replica Location

Web

server

SCAN server

Overlay Network Monitoring

network plane


Towards a transparent and proactively managed internet

Adaptive Overlay Streaming Media

Stanford

UC San Diego

UC Berkeley

X

HP Labs

  • Implemented with Winamp client and SHOUTcast server

  • Congestion introduced with a Packet Shaper

  • Skip-free playback: server buffering and rewinding

  • Total adaptation time < 4 seconds


Summary

Summary

  • A tomography-based overlay network monitoring system

    • Selectively monitor a basis set of O(n logn) paths to infer the loss rates of O(n2) paths

    • Works in real-time, adaptive to topology changes, has good load balancing and tolerates topology errors

  • Both simulation and real Internet experiments promising

  • Built adaptive overlay streaming media system on top of TOM

    • Bypass congestion/failures for smooth playback within seconds


Tie back to scan

Tie Back to SCAN

Provision: Dynamic Replication

+ Update Multicast Tree Building

Replica Management:

(Incremental) Content Clustering

Network DoS Resilient

Replica Location: Tapestry

Network End-to-End Distance Monitoring

Internet Iso-bar: latencyTOM: loss rate


Contribution of my thesis

Contribution of My Thesis

  • Replica location

    • Proposed the first simulation-based network DoS resilience benchmark and quantify three types of directory services

  • Dynamically place close to optimal # of replicas

    • Self-organize replicas into a scalable app-level multicast tree for disseminating updates

  • Cluster objects to significantly reduce the management overhead with little performance sacrifice

    • Online incremental clustering and replication to adapt to users’ access pattern changes

  • Scalable overlay network monitoring


Existing cdns fail to address these challenges

Existing CDNs Fail to Address these Challenges

No coherence for dynamic content

X

Unscalable network monitoring - O(M ×N)

M: # of client groups, N: # of server farms

Non-cooperative replication inefficient


Problem formulation

Problem Formulation

  • Subject to certain total replication cost (e.g., # of URL replicas)

  • Find a scalable, adaptive replication strategy to reduce avg access cost


Scan scalable content access network

SCAN: Scalable Content Access Network

CDN Applications (e.g. streaming media)

Provision: Cooperative

Clustering-based Replication

Coherence: Update Multicast

Tree Construction

Network Distance/ Congestion/ Failure

Estimation

User Behavior/

Workload Monitoring

Network Performance

Monitoring

red: my work, black: out of scope


Comparison of content delivery systems cont d

Comparison of Content Delivery Systems (cont’d)


  • Login