potential applications of shared bottleneck detection sbd michael welzl n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Potential Applications of Shared Bottleneck Detection (SBD) Michael Welzl PowerPoint Presentation
Download Presentation
Potential Applications of Shared Bottleneck Detection (SBD) Michael Welzl

Loading in 2 Seconds...

play fullscreen
1 / 23

Potential Applications of Shared Bottleneck Detection (SBD) Michael Welzl - PowerPoint PPT Presentation


  • 110 Views
  • Uploaded on

Potential Applications of Shared Bottleneck Detection (SBD) Michael Welzl. NNUW-1 Simula , Oslo 18. 09. 2013. Context. SBD has been around for a while Papers dating back more than a decade Approach: correlate per-path measurements (e.g. OWD, ..)

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Potential Applications of Shared Bottleneck Detection (SBD) Michael Welzl' - sean-fuller


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
potential applications of shared bottleneck detection sbd michael welzl
Potential Applications ofShared Bottleneck Detection (SBD)

Michael Welzl

NNUW-1Simula, Oslo

18. 09. 2013

context
Context
  • SBD has been around for a while
    • Papers dating back more than a decade
    • Approach: correlate per-path measurements (e.g. OWD, ..)
    • Never been a “hot topic”; said to be too unreliable, e.g. in IETF
    • Note: applications have different reliability requirements
  • Major problem: real-life validation
    • Most papers are poor in this aspect
      • E.g. real-life test without really validating
    • Hard to do in most testbeds
      • We tried and failed in PlanetLab: no shared bottlenecks
  • NorNet could make a real difference here
multi path transfer
Multi-path transfer
  • Better congestion control for MPTCP, CMT-SCTP
    • cf.: SofianeHassayoun, JanardhanIyengar and David Ros, “Dynamic Window Coupling for Multipath Congestion Control”. In Proceedings of IEEE ICNP 2011, Vancouver, October 2011.
  • Basic issue:On N paths, act like 1 flow or like N?
    • N might be unfair if the flows share the same bottleneck
    • 1 might be too conservative if they don’t
multi path overlay ec gin lft scenario
Multi-path overlay:EC-GIN LFT scenario

Multipath file transfer not beneficial

due to shared bottleneck

Multipath file transfer

(AB + ACB) beneficial

file transfer delay prediction
File transferdelayprediction
  • Considertelling A and B thatittakes 5 minutesto send a 10 GB fileto C
    • But thenthey send atthe same time...
  • This isneededforscheduling in distributedcomputing
distributed computing abstract concrete wf mapping
Distributed computing: abstract-concrete WF mapping

Tasks = {T1, T2, T3, T4}Resources = {R1, R2, R3, R4}Data transfers = {D1, D2, D3, D4}

Unnoticedby

traditionalschedulingalgorithms!

incorporating the net
Incorporating the net
  • Theory (analysis, “abstract” simulations)
    • Network-aware variant of Heterogeneous Earliest Finish Time (HEFT) scheduling algorithm developed in Ph.D. thesisMurtazaYousaf, "Accurate Shared Bottleneck Detection and File Transfer Delay Prediction for Grid Scheduling", December 2008, University of Innsbruck, Austria.
  • Next step: more realistic simulations, using ns-2
    • Simulator available, developed in master thesis
    • Never been used! Get it from: http://heim.ifi.uio.no/michawe/research/projects/ec-gin-uibk/
  • Next step: real-life tests using SBD
    • Nothing done yet!
per flow qos without signaling to routers

2. yes

4. ok

1. may I join?

3. I quit

Per-flow QoS without signaling to routers

Traditional method: signalingtoedgerouters (e.g. with COPS) atthispoint!

Synchronizationofdistributed (P2P based)database; all flowsknownto all brokers

Synchronizationofdistributed (P2P based)database; all flowsknownto all brokers

Synchronizationofdistributed (P2P based)database; link capacitiesknownto all brokers

continuous measurements;update to BB upon path change

realistic
Realistic?
  • Yes, when you’re in control of alltraffic, know shared bottlenecks, and know their capacity
    • Several Ph.D. theses, some in “EC-GIN” EU projectOne of them:KashifMunir, "Admission and Congestion Control for Deadline Constrained Data Transfers", May 2009, University of Innsbruck, Austria.
    • BDTS system at INRIA Lyon, successfully applied in French Grid’5000 environment
    • Incorporated in INRIA spin-off Lyatiss
slide11

INRIA’s

JBDTS GUI

network monitoring
Network monitoring
  • “With CloudWeaver, users are able to gain full insight into the underlying topology, capacity, usage and vulnerability of a cloud deployment; anticipate congestion and fix bottlenecks, maximize resource utilization according to flow patterns and minimize infrastructure costs.”
    • Wouldn’t it be nice to do something along these lines for networks where you cannot control everything?
how we use the internet today 2 stories
How we use the Internet today: 2 stories
  • I clean our flat while listening to Spotify via my wife’s laptop
    • in parallel, downloading files via my own
    • Suddenly I begin to think:“please, dear downloads, don’t make the music stop!”
  • I am in a hotel room, using Skype with video to see my daughter
    • Quality barely good enough
    • I avoid clicking on anything
    • Note: that’s different when I talk to my mother...
a major problem
A major problem
  • We may have become used to this, but that doesn’t mean it’s good?!
    • Would like to specify: do not interrupt Spotify / Skype(or know: do downloads disturb Spotify / Skype or not?)
  • These were just two examples
    • Downloads can also have different priorities
    • When I download two files, I try to guess whether the downloads slow each other down
ingredients of this controlled fairness soup
Ingredients of this controlled-fairness soup
  • Shared bottleneck detection
    • for the user: know about mutual influence of transfers
    • for upload and download: control fairness only among flows that share a bottleneck
  • Coupled congestion control for separate flows
    • Solutions exist (CM, TCB interdependence (RFC 2140))
  • Alternative to coupled cc.: tunable-aggression-TCP
    • Solutions exist
  • E2E-signaling of fairness requirements
    • Doesn’t exist?!
  • ... and a GUI that shows transfers by application;existing tools can do that
coupling cc of separate flows
Coupling cc. of separate flows
  • Does something very wrong if flows do NOT share the same bottleneck!
  • But if they do, a ton of potential very positive side-effects
    • Less delay
    • Less packet loss
    • Less signalling(N flows don’t need N*feedback about the same path)
    • More controllable behavior(sender-side scheduling vs. “fighting it out” on the bottleneck)
    • Better performance for short or application-limited flows(TCP does use itorloseit; with shared congestion control, ifflow 1 doesn’t use it, maybe flow 2 does.Skip slow start: again less queuing delay from slow start overshoot)
ietf rmcat proposal flow state exchange fse
IETF RMCAT proposal:“Flow State Exchange” (FSE)
  • The result of searching for minimum-necessary-standardization: only define what goes in / out, how data are maintained
    • Could reside in a single app(e.g. browser)and/or in the OS

Traditional CM

FSE-based CM

Another possibleimplementationof flow coordination

CM

CM

Stream 1

Stream 1

Stream 1

FSE

FSE

Stream 2

Stream 2

Stream 2

coupled cc thinking ahead
Coupled cc: thinking ahead
  • If we can relax update frequency, can we coordinaterates across hosts?
  • May need to detect shared bottlenecks on both sides

H2

H1

H3

conclusion
Conclusion
  • Some SBD mechanisms exist
    • real-life validation is urgently needed!
  • A long list of potential applications
    • Better cc for multi-path transfers
      • Or decisions to apply multi-path, e.g. in an overlay
    • Predicting file transfer delay
      • Important input for scheduling in distributed processing
    • Deeper knowledge about a net; good for monitoring or even QoS (-ish) mechanisms
      • At the “least QoS” end of the spectrum, just tell users which flows influence each other, let them apply priorities
    • Coupled congestion control
      • Many possibilities there, e.g. cross-host-control