P. 2. P2: Implementing Declarative Overlays. Timothy Roscoe Boon Thau Loo, Tyson Condie, Petros Maniatis, Ion Stoica, David Gay, Joseph M. Hellerstein, Intel Research Berkeley and U. C. Berkeley. Overlay. Internet. Overlays: a broad view.
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
P
2
P2: Implementing Declarative Overlays
Timothy Roscoe
Boon Thau Loo, Tyson Condie,Petros Maniatis, Ion Stoica, David Gay, Joseph M. Hellerstein,
Intel Research Berkeleyand U. C. Berkeley
Overlay
Internet
“Overlay”: the routing and message forwarding component of any non-trivial distributed system
Overlay
Internet
Distributed systems is all about overlays
In theory
Figure out right properties
Get the algorithms and protocols
Implement them
Tune them
Test them
Debug them
Repeat
But in practice
No global view
Wrong choice of algorithms
Incorrect implementation
Pathological timeouts
Partial failures
Impaired introspection
Homicidal boredom
Next to no debug support
It’s hard enough as it isDo I also need to reinvent the wheel every time?
if k in (n, n.successor)
return n.successor
else
return n.successor. lookup(k)
if k in (n, n.successor)
return n.successor
else
return n.successor. lookup(k)
send response( Req, K, SAddr ) to Req
where lookup( NAddr, Req, K ) @ NAddr
andnode ( NAddr, N ),
and succ ( NAddr, Succ, SAddr ),
andK in ( N, Succ ],
if k in (n, n.successor)
return n.successor
else
return n.successor. lookup(k)
send response( Req, K, SAddr ) to Req
where lookup( NAddr, Req, K ) @ NAddr
andnode ( NAddr, N ),
and succ ( NAddr, Succ, SAddr ),
andK in ( N, Succ ],
send lookup( Req, K, SAddr ) to SAddr
wherelookup( NAddr, Req, K ) @ Naddr
andnode ( NAddr, N ),
and succ ( NAddr, Succ, SAddr ),
andK not in ( N, Succ ],
if k in (n, n.successor)
return n.successor
else
return n.successor. lookup(k)
send response( Req, K, SAddr ) to Reqwhere lookup( NAddr, Req, K ) @ NAddr & node ( NAddr, N )& succ ( NAddr, Succ, SAddr ) & K in ( N, Succ ]
send lookup( Req, K, SAddr ) to SAddrwhere lookup( NAddr, Req, K ) @ Naddr & node ( NAddr, N )& succ ( NAddr, Succ, SAddr ) & K not in ( N, Succ ]
send response( Req, K, SAddr ) to Reqwhere lookup( NAddr, Req, K ) @ NAddr & node ( NAddr, N )& succ ( NAddr, Succ, SAddr ) & K in ( N, Succ ]
send lookup( Req, K, SAddr ) to SAddrwhere lookup( NAddr, Req, K ) @ Naddr & node ( NAddr, N )& succ ( NAddr, Succ, SAddr ) & K not in ( N, Succ ]
send response( Req, K, SAddr ) to Reqwhere lookup( NAddr, Req, K ) @ NAddr & node ( NAddr, N )& succ ( NAddr, Succ, SAddr ) & K in ( N, Succ ]
send lookup( Req, K, SAddr ) to SAddrwhere lookup( NAddr, Req, K ) @ Naddr & node ( NAddr, N )& succ ( NAddr, Succ, SAddr ) & K not in ( N, Succ ]
send response( Req, K, SAddr ) to Reqwhere lookup( NAddr, Req, K ) @ NAddr & node ( NAddr, N )& succ ( NAddr, Succ, SAddr ) & K in ( N, Succ ]
send lookup( Req, K, SAddr ) to SAddrwhere lookup( NAddr, Req, K ) @ Naddr & node ( NAddr, N )& succ ( NAddr, Succ, SAddr ) & K not in ( N, Succ ]
send response( Req, K, SAddr ) to Reqwhere lookup( NAddr, Req, K ) @ NAddr & node ( NAddr, N )& succ ( NAddr, Succ, SAddr ) & K in ( N, Succ ]
send lookup( Req, K, SAddr ) to SAddrwhere lookup( NAddr, Req, K ) @ Naddr & node ( NAddr, N )& succ ( NAddr, Succ, SAddr ) & K not in ( N, Succ ]
send response( Req, K, SAddr ) to Reqwhere lookup( NAddr, Req, K ) @ NAddr & node ( NAddr, N )& succ ( NAddr, Succ, SAddr ) & K in ( N, Succ ]
send lookup( Req, K, SAddr ) to SAddrwhere lookup( NAddr, Req, K ) @ Naddr & node ( NAddr, N )& succ ( NAddr, Succ, SAddr ) & K not in ( N, Succ ]
send response( Req, K, SAddr ) to Reqwhere lookup( NAddr, Req, K ) @ NAddr & node ( NAddr, N )& succ ( NAddr, Succ, SAddr ) & K in ( N, Succ ]
send lookup( Req, K, SAddr ) to SAddrwhere lookup( NAddr, Req, K ) @ Naddr & node ( NAddr, N )& succ ( NAddr, Succ, SAddr ) & K not in ( N, Succ ]
send response( Req, K, SAddr ) to Reqwhere lookup( NAddr, Req, K ) @ NAddr & node ( NAddr, N )& succ ( NAddr, Succ, SAddr ) & K in ( N, Succ ]
send lookup( Req, K, SAddr ) to SAddrwhere lookup( NAddr, Req, K ) @ Naddr & node ( NAddr, N )& succ ( NAddr, Succ, SAddr ) & K not in ( N, Succ ]
lookup
Strand 1
response
node
succ
lookup
Strand 2
lookup
Strand 1
U
lookup
R
D
x
P
Strand 2
lookup
Sched
...
C
R
x
C
...
C
T
x
C
Q
u
e
...
u
e
Q
u
e
u
e
D
e
m
u
U
x
T
D
node
succ
...
x
P
1. Distributed system specified in a query language
Network
Overlay
Description
2. Compiled into optimized graph of dataflow elements
3. Graph executed directly to maintain routing tables and network overlay state
Packets in
Packets out
10 pt font
Compare with Bamboo non-adaptive timeout figures…
But …
Once you have a distributed query processor, lots of things fall off the back of the truck…
http://P2.berkeley.intel-research.net