the design and implementation of declarative networks
Download
Skip this Video
Download Presentation
The Design and Implementation of Declarative Networks

Loading in 2 Seconds...

play fullscreen
1 / 31

The Design and Implementation of Declarative Networks - PowerPoint PPT Presentation


  • 208 Views
  • Uploaded on

The Design and Implementation of Declarative Networks. Boon Thau Loo University of Pennsylvania, University of California-Berkeley *. *This dissertation was completed as a graduate student at the University of California- Berkeley. Declarative Networking. A declarative framework for networks:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'The Design and Implementation of Declarative Networks' - ely


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
the design and implementation of declarative networks

The Design and Implementation of Declarative Networks

Boon Thau Loo

University of Pennsylvania, University of California-Berkeley*

*This dissertation was completed as a graduate student at the University of California- Berkeley

declarative networking
Declarative Networking
  • A declarative framework for networks:
    • Network protocols are declaratively specified using a database query language
    • Distributed query engine executes specifications to implement network protocols
  • Success of database research:
    • 70’s – today: Database research has revolutionized data management
    • Today: Similar opportunity to revolutionize the Internet architecture
motivation
Motivation
  • Internet faces many challenges today:
    • Unwanted, harmful traffic
    • Complexity/fragility in Internet routing
    • Proliferation of new applications and services
  • Efforts at improving the Internet:
    • Evolutionary: App-level “Overlay” networks
    • Revolutionary: Clean-slate designs
      • NSF GENI initiative, FIND program

Opportunity: Software tools that can significantly accelerate network innovation

a declarative network

Recursive Query Execution

Network protocol

Distributed database

Network State

Distributed Dataflow

Network messages

A Declarative Network

messages

Dataflow

Dataflow

messages

Dataflow

messages

Dataflow

Dataflow

Distributed recursive query

Dataflow

Declarative Networks

Traditional Networks

the case for declarative networking
The Case for Declarative Networking
  • Ease of programming:
    • Compact and high-level representation of protocols
    • Orders of magnitude reduction in code size
      • Declarative Chord DHT is 48 lines instead of 10,000.
    • Easy customization
  • Safety:
    • Queries are “sandboxed” within query processor
    • Potential for static analysis of safety
  • What about efficiency?
    • No fundamental overhead when executing standard routing protocols
    • Application of well-studied query optimizations
    • Note: Same question was asked of relational databases in the 70’s.
main contributions of dissertation
Main Contributions of Dissertation
  • Declarative Routing [SIGCOMM ’05]:
    • Extensible Routers: balance of flexibility, efficiency and safety
  • Declarative Overlays [SOSP ’05]:
    • Rapid prototyping of new overlay networks
  • Database Fundamentals [SIGMOD ‘06]:
    • Network specific query language and semantics
    • Distributed recursive query execution strategies
    • Query optimizations, classical and new
a breadth of use cases
A Breadth of Use Cases
  • Example implementations to date:
    • Textbook routing protocols
    • Chord DHT
    • Narada mesh for end-system multicast
    • Distributed Gnutella/Web crawlers
    • Pastry DHT
    • Replication protocols
    • Lamport/Chandy snapshots
    • Paxos distributed consensus
    • Overlays for host mobility
    • Sensor network protocols
  • P2 declarative networking system
    • http://p2.cs.berkeley.edu
outline
Outline
  • Background
  • The Connection: Routing as a Query
    • Execution Model
    • Path-Vector Protocol Example
      • Query specification  protocol implementation
  • Query Processing
  • Beyond routing: Declarative Overlays
  • Ongoing work @ Penn
traditional router

Neighbor Table updates

Forwarding Table updates

Packets

Packets

Traditional Router

Routing

Protocol

Control Plane

Forwarding Plane

Traditional Router

declarative router

Declarative Queries

Neighbor Table updates

Forwarding Table updates

Packets

Packets

Declarative Router

Routing

Protocol

Query Engine

Control Plane

Forwarding Plane

Declarative Router

Traditional Router

all pairs reachability
All-Pairs Reachability

R1: reachable(S,D)link(S,D)

R2: reachable(S,D)link(S,Z),reachable(Z,D)

“For all nodes S,D, If there is a link from S to D, then S can reach D”.

link(a,b) – “there is a link from node a to node b”

reachable(a,b) – “node a can reach node b”

Input: link(source, destination)

Output: reachable(source, destination)

all pairs reachability12
All-Pairs Reachability

R1: reachable(S,D)link(S,D)

R2: reachable(S,D)link(S,Z),reachable(Z,D)

“For all nodes S,D and Z, If there is a link from S to Z, AND Z can reach D, then S can reach D”.

Input: link(source, destination)

Output: reachable(source, destination)

towards network datalog
Towards Network Datalog
  • Specify tuple placement
    • Value-based partitioning of tables
  • Tuples to be combined are co-located
    • Rule rewrite ensures body is always single-site
  • All communication is among neighbors
    • No multihop routing during basic rule execution
    • Link-restricted rules: Enforced via simple syntactic restrictions
network datalog

Location Specifier “@S”

All-Pairs Reachability

Network Datalog

R1: reachable(@S,D)link(@S,D)

R2: reachable(@S,D)link(@S,Z), reachable(@Z,D)

Query: reachable(@a,N)

Query: reachable(@M,N)

link

link

link

link

Input table:

b

c

d

a

reachable

reachable

reachable

reachable

Output table:

Query: reachable(@a,N)

path vector in network datalog
Path Vector in Network Datalog

R1: path(@S,D,P) link(@S,D), P=(S,D).

path(@Z,D,P2),

P=SP2.

path(@S,D,P)

link(@Z,S),

R2:

Query: path(@S,D,P)

Add S to front of P2

Input: link(@source, destination)

Query output: path(@source, destination, pathVector)

slide16

path

path

path

Query Execution

R1: path(@S,D,P) link(@S,D), P=(S,D).

R2: path(@S,D,P)  link(@Z,S), path(@Z,D,P2), P=SP2.

Query: path(@a,d,P)

link

link

link

link

Neighbor table:

a

b

c

d

Forwarding table:

slide17

Matching variableZ= “Join”

path

path

path

Query Execution

R1: path(@S,D,P)  link(@S,D), P=(S,D).

R2: path(@S,D,P) link(@Z,S), path(@Z,D,P2), P=SP2.

Query: path(@a,d,P)

link

link

link

link

Neighbor table:

Communication patterns are identical to those in the actual path vector protocol

a

b

c

d

path(@a,d,[a,b,c,d])

path(@b,d,[b,c,d])

Forwarding table:

other routing examples
Other Routing Examples
  • Best-Path Routing
  • Distance Vector
  • Dynamic Source Routing (Wireless)
  • Policy Decisions
  • QoS-based Routing
  • Link-state
  • Multicast Overlays (Single-Source & CBT)
outline19
Outline
  • Background
  • The Connection: Routing as a Query
  • Query Processing
  • Beyond routing: Declarative Overlays
  • Sampling of ongoing work
recursive query evaluation

10

9

3-hop

8

7

6

5

2-hop

4

3

1-hop

2

1

Recursive Query Evaluation
  • Semi-naïve evaluation:
    • Iterations (rounds) of synchronous computation
    • Results from iteration ith used in (i+1)th

9

7

5

2

4

10

1

8

0

3

6

Link Table

Path Table

Network

Problem: Unpredictable delays and failures

pipelined semi na ve psn
Pipelined Semi-naïve (PSN)
  • Fully-asynchronous evaluation:
    • Computed tuples in any iteration pipelined to next iteration
    • Natural for network protocols

9

10

7

9

5

6

2

4

10

1

3

Relaxation of semi-naïve

8

0

8

5

3

2

7

6

4

1

Link Table

Path Table

Network

pipelined evaluation

p(x,z) :- p1(x,y), p2(y,z), …, pn(y,z), q(z,w)

recursive w.r.t. p

Pipelined Evaluation
  • Challenges:
    • Does PSN produce the correct answer?
    • Is PSN bandwidth efficient?
      • I.e. does it make the minimum number of inferences?
  • Proofs for
    • Basic technique: local timestamps
execution plan
Execution Plan

Strands

Network Out

Network In

Messages

Messages

Single Node

  • Nodes in execution plan (“operators”):
    • Network operators (send/recv, cc, retry, rate limitation)
    • Relational operators (selects, projects, joins, aggregates)
    • Flow operators (mux, demux, queues)
localization rewrite

Matching variable Z = “Join”

Matching variable Z = “Join”

Localization Rewrite
  • Rules may have body predicates at different locations:

R2: path(@S,D,P)  link(@S,Z), path(@Z,D,P2), P=SP2.

Rewritten rules:

R2a: linkD(S,@D) link(@S,D)

R2b: path(@S,D,P) linkD(S,@Z), path(@Z,D,P2), P=SP2.

localized rule compilation

Join

path.Z = linkD.Z

Project

path(S,D,P)

path

Send to path.S

linkD

Join

linkD.Z = path.Z

Project

path(S,D,P)

linkD

Send to path.S

path

Localized Rule Compilation

R2b: path(@S,D,P)  linkD(S,@Z), path(@Z,D,P2), P=SP2.

Execution Plan

Network In

Network Out

optimizations

PV/DV (Wired)  DSR (Wireless)

Hybridized protocols: Zone Routing Protocol

Optimizations
  • Traditional: evaluate in the NW context
    • Aggregate Selections
    • Magic Sets rewrite
    • Predicate Reordering
  • New: motivated by NW context
    • Multi-query optimizations:
      • Query Results caching
      • Opportunistic message sharing
    • Cost-based optimizations
      • Neighborhood density function
      • Hybrid rewrites
beyond routing declarative overlays
Beyond Routing: Declarative Overlays
  • Language extensions to support events and soft-state predicates
  • Chord Routing, including:
    • Multiple successors
    • Stabilization
    • Optimized finger maintenance
    • Failure detection
  • 48 rules
  • 11 table definitions
  • MIT-Chord: x100 more code
  • Another example:
    • Narada mesh in 22 rules

10 pt font

outline28
Outline
  • Background
  • The Connection: Routing as a Query
  • Query Processing
  • Beyond routing: Declarative Overlays
  • Ongoing work @ Penn
ongoing work @ penn
Ongoing Work @ Penn
  • Declarative secure networking
    • Difficult to design/implement/reason about secure networks
    • Network Datalog + logic-based security languages [NetDB ’07]
    • Authenticated path vector protocol, DNSSEC, secure DHTs,…
    • Moving forward:
      • Exploit fine-grained control over networks and security policies
  • Data-centric querying and routing in heterogeneous networks
    • Internet: Wired infrastructure with wireless clouds at the edges
    • Flexible network support for mobility [ACM MobiArch ’07]
      • Declarative queries for addressing and naming mobile hosts
      • Session-aware customizable QoS routing
    • Moving forward:
      • Declarative wireless ad-hoc networks
      • Cost-based query optimizations to adapt protocols
summary
Summary
  • Declarative networking:
    • Declarative Routing:
      • Extensible routing infrastructure
    • Declarative Overlays
      • Rapid prototyping overlay networks
    • Database fundamentals
      • Query language
      • New distributed query execution strategies and optimizations
      • Semantics in dynamic networks
    • P2 declarative networking system (http://p2.cs.berkeley.edu)
many thanks
Many Thanks…
  • Advisors: Joseph M. Hellerstein, Ion Stoica
  • Collaborators:
    • UC Berkeley: Tyson Condie, Ryan Huebsch
    • Intel Research: David Gay, Petros Maniatis, Timothy Roscoe
    • Yahoo! Research: Minos Garofalakis, Raghu Ramakrishnan
    • Rice University: Atul Singh
    • Many others…
ad