The design and implementation of declarative networks l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 31

The Design and Implementation of Declarative Networks PowerPoint PPT Presentation


  • 156 Views
  • Updated On :
  • Presentation posted in: General

The Design and Implementation of Declarative Networks. Boon Thau Loo University of Pennsylvania, University of California-Berkeley *. *This dissertation was completed as a graduate student at the University of California- Berkeley. Declarative Networking. A declarative framework for networks:

Download Presentation

The Design and Implementation of Declarative Networks

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The design and implementation of declarative networks l.jpg

The Design and Implementation of Declarative Networks

Boon Thau Loo

University of Pennsylvania, University of California-Berkeley*

*This dissertation was completed as a graduate student at the University of California- Berkeley


Declarative networking l.jpg

Declarative Networking

  • A declarative framework for networks:

    • Network protocols are declaratively specified using a database query language

    • Distributed query engine executes specifications to implement network protocols

  • Success of database research:

    • 70’s – today: Database research has revolutionized data management

    • Today: Similar opportunity to revolutionize the Internet architecture


Motivation l.jpg

Motivation

  • Internet faces many challenges today:

    • Unwanted, harmful traffic

    • Complexity/fragility in Internet routing

    • Proliferation of new applications and services

  • Efforts at improving the Internet:

    • Evolutionary: App-level “Overlay” networks

    • Revolutionary: Clean-slate designs

      • NSF GENI initiative, FIND program

Opportunity: Software tools that can significantly accelerate network innovation


A declarative network l.jpg

Recursive Query Execution

Network protocol

Distributed database

Network State

Distributed Dataflow

Network messages

A Declarative Network

messages

Dataflow

Dataflow

messages

Dataflow

messages

Dataflow

Dataflow

Distributed recursive query

Dataflow

Declarative Networks

Traditional Networks


The case for declarative networking l.jpg

The Case for Declarative Networking

  • Ease of programming:

    • Compact and high-level representation of protocols

    • Orders of magnitude reduction in code size

      • Declarative Chord DHT is 48 lines instead of 10,000.

    • Easy customization

  • Safety:

    • Queries are “sandboxed” within query processor

    • Potential for static analysis of safety

  • What about efficiency?

    • No fundamental overhead when executing standard routing protocols

    • Application of well-studied query optimizations

    • Note: Same question was asked of relational databases in the 70’s.


Main contributions of dissertation l.jpg

Main Contributions of Dissertation

  • Declarative Routing [SIGCOMM ’05]:

    • Extensible Routers: balance of flexibility, efficiency and safety

  • Declarative Overlays [SOSP ’05]:

    • Rapid prototyping of new overlay networks

  • Database Fundamentals [SIGMOD ‘06]:

    • Network specific query language and semantics

    • Distributed recursive query execution strategies

    • Query optimizations, classical and new


A breadth of use cases l.jpg

A Breadth of Use Cases

  • Example implementations to date:

    • Textbook routing protocols

    • Chord DHT

    • Narada mesh for end-system multicast

    • Distributed Gnutella/Web crawlers

    • Pastry DHT

    • Replication protocols

    • Lamport/Chandy snapshots

    • Paxos distributed consensus

    • Overlays for host mobility

    • Sensor network protocols

  • P2 declarative networking system

    • http://p2.cs.berkeley.edu


Outline l.jpg

Outline

  • Background

  • The Connection: Routing as a Query

    • Execution Model

    • Path-Vector Protocol Example

      • Query specification  protocol implementation

  • Query Processing

  • Beyond routing: Declarative Overlays

  • Ongoing work @ Penn


Traditional router l.jpg

Neighbor Table updates

Forwarding Table updates

Packets

Packets

Traditional Router

Routing

Protocol

Control Plane

Forwarding Plane

Traditional Router


Declarative router l.jpg

Declarative Queries

Neighbor Table updates

Forwarding Table updates

Packets

Packets

Declarative Router

Routing

Protocol

Query Engine

Control Plane

Forwarding Plane

Declarative Router

Traditional Router


All pairs reachability l.jpg

All-Pairs Reachability

R1: reachable(S,D)link(S,D)

R2: reachable(S,D)link(S,Z),reachable(Z,D)

“For all nodes S,D, If there is a link from S to D, then S can reach D”.

link(a,b) – “there is a link from node a to node b”

reachable(a,b) – “node a can reach node b”

Input: link(source, destination)

Output: reachable(source, destination)


All pairs reachability12 l.jpg

All-Pairs Reachability

R1: reachable(S,D)link(S,D)

R2: reachable(S,D)link(S,Z),reachable(Z,D)

“For all nodes S,D and Z, If there is a link from S to Z, AND Z can reach D, then S can reach D”.

Input: link(source, destination)

Output: reachable(source, destination)


Towards network datalog l.jpg

Towards Network Datalog

  • Specify tuple placement

    • Value-based partitioning of tables

  • Tuples to be combined are co-located

    • Rule rewrite ensures body is always single-site

  • All communication is among neighbors

    • No multihop routing during basic rule execution

    • Link-restricted rules: Enforced via simple syntactic restrictions


Network datalog l.jpg

Location Specifier [email protected]

All-Pairs Reachability

Network Datalog

R1: reachable(@S,D)link(@S,D)

R2: reachable(@S,D)link(@S,Z), reachable(@Z,D)

Query: reachable(@a,N)

Query: reachable(@M,N)

link

link

link

link

Input table:

b

c

d

a

reachable

reachable

reachable

reachable

Output table:

Query: reachable(@a,N)


Path vector in network datalog l.jpg

Path Vector in Network Datalog

R1: path(@S,D,P) link(@S,D), P=(S,D).

path(@Z,D,P2),

P=SP2.

path(@S,D,P)

link(@Z,S),

R2:

Query: path(@S,D,P)

Add S to front of P2

Input: link(@source, destination)

Query output: path(@source, destination, pathVector)


Slide16 l.jpg

path

path

path

Query Execution

R1: path(@S,D,P) link(@S,D), P=(S,D).

R2: path(@S,D,P)  link(@Z,S), path(@Z,D,P2), P=SP2.

Query: path(@a,d,P)

link

link

link

link

Neighbor table:

a

b

c

d

Forwarding table:


Slide17 l.jpg

Matching variableZ= “Join”

path

path

path

Query Execution

R1: path(@S,D,P)  link(@S,D), P=(S,D).

R2: path(@S,D,P) link(@Z,S), path(@Z,D,P2), P=SP2.

Query: path(@a,d,P)

link

link

link

link

Neighbor table:

Communication patterns are identical to those in the actual path vector protocol

a

b

c

d

path(@a,d,[a,b,c,d])

path(@b,d,[b,c,d])

Forwarding table:


Other routing examples l.jpg

Other Routing Examples

  • Best-Path Routing

  • Distance Vector

  • Dynamic Source Routing (Wireless)

  • Policy Decisions

  • QoS-based Routing

  • Link-state

  • Multicast Overlays (Single-Source & CBT)


Outline19 l.jpg

Outline

  • Background

  • The Connection: Routing as a Query

  • Query Processing

  • Beyond routing: Declarative Overlays

  • Sampling of ongoing work


Recursive query evaluation l.jpg

10

9

3-hop

8

7

6

5

2-hop

4

3

1-hop

2

1

Recursive Query Evaluation

  • Semi-naïve evaluation:

    • Iterations (rounds) of synchronous computation

    • Results from iteration ith used in (i+1)th

9

7

5

2

4

10

1

8

0

3

6

Link Table

Path Table

Network

Problem: Unpredictable delays and failures


Pipelined semi na ve psn l.jpg

Pipelined Semi-naïve (PSN)

  • Fully-asynchronous evaluation:

    • Computed tuples in any iteration pipelined to next iteration

    • Natural for network protocols

9

10

7

9

5

6

2

4

10

1

3

Relaxation of semi-naïve

8

0

8

5

3

2

7

6

4

1

Link Table

Path Table

Network


Pipelined evaluation l.jpg

  • p(x,z) :- p1(x,y), p2(y,z), …, pn(y,z), q(z,w)

recursive w.r.t. p

Pipelined Evaluation

  • Challenges:

    • Does PSN produce the correct answer?

    • Is PSN bandwidth efficient?

      • I.e. does it make the minimum number of inferences?

  • Proofs for

    • Basic technique: local timestamps


Execution plan l.jpg

Execution Plan

Strands

Network Out

Network In

Messages

Messages

Single Node

  • Nodes in execution plan (“operators”):

    • Network operators (send/recv, cc, retry, rate limitation)

    • Relational operators (selects, projects, joins, aggregates)

    • Flow operators (mux, demux, queues)


Localization rewrite l.jpg

Matching variable Z = “Join”

Matching variable Z = “Join”

Localization Rewrite

  • Rules may have body predicates at different locations:

R2: path(@S,D,P)  link(@S,Z), path(@Z,D,P2), P=SP2.

Rewritten rules:

R2a: linkD(S,@D) link(@S,D)

R2b: path(@S,D,P) linkD(S,@Z), path(@Z,D,P2), P=SP2.


Localized rule compilation l.jpg

Join

path.Z = linkD.Z

Project

path(S,D,P)

path

Send to path.S

linkD

Join

linkD.Z = path.Z

Project

path(S,D,P)

linkD

Send to path.S

path

Localized Rule Compilation

R2b: path(@S,D,P)  linkD(S,@Z), path(@Z,D,P2), P=SP2.

Execution Plan

Network In

Network Out


Optimizations l.jpg

PV/DV (Wired)  DSR (Wireless)

Hybridized protocols: Zone Routing Protocol

Optimizations

  • Traditional: evaluate in the NW context

    • Aggregate Selections

    • Magic Sets rewrite

    • Predicate Reordering

  • New: motivated by NW context

    • Multi-query optimizations:

      • Query Results caching

      • Opportunistic message sharing

    • Cost-based optimizations

      • Neighborhood density function

      • Hybrid rewrites


Beyond routing declarative overlays l.jpg

Beyond Routing: Declarative Overlays

  • Language extensions to support events and soft-state predicates

  • Chord Routing, including:

    • Multiple successors

    • Stabilization

    • Optimized finger maintenance

    • Failure detection

  • 48 rules

  • 11 table definitions

  • MIT-Chord: x100 more code

  • Another example:

    • Narada mesh in 22 rules

10 pt font


Outline28 l.jpg

Outline

  • Background

  • The Connection: Routing as a Query

  • Query Processing

  • Beyond routing: Declarative Overlays

  • Ongoing work @ Penn


Ongoing work @ penn l.jpg

Ongoing Work @ Penn

  • Declarative secure networking

    • Difficult to design/implement/reason about secure networks

    • Network Datalog + logic-based security languages [NetDB ’07]

    • Authenticated path vector protocol, DNSSEC, secure DHTs,…

    • Moving forward:

      • Exploit fine-grained control over networks and security policies

  • Data-centric querying and routing in heterogeneous networks

    • Internet: Wired infrastructure with wireless clouds at the edges

    • Flexible network support for mobility [ACM MobiArch ’07]

      • Declarative queries for addressing and naming mobile hosts

      • Session-aware customizable QoS routing

    • Moving forward:

      • Declarative wireless ad-hoc networks

      • Cost-based query optimizations to adapt protocols


Summary l.jpg

Summary

  • Declarative networking:

    • Declarative Routing:

      • Extensible routing infrastructure

    • Declarative Overlays

      • Rapid prototyping overlay networks

    • Database fundamentals

      • Query language

      • New distributed query execution strategies and optimizations

      • Semantics in dynamic networks

    • P2 declarative networking system (http://p2.cs.berkeley.edu)


Many thanks l.jpg

Many Thanks…

  • Advisors: Joseph M. Hellerstein, Ion Stoica

  • Collaborators:

    • UC Berkeley: Tyson Condie, Ryan Huebsch

    • Intel Research: David Gay, Petros Maniatis, Timothy Roscoe

    • Yahoo! Research: Minos Garofalakis, Raghu Ramakrishnan

    • Rice University: Atul Singh

    • Many others…


  • Login