Adopting Ideas from Interplanetary Networking for Sensor Network Applications

Adopting Ideas from Interplanetary Networking for Sensor Network Applications Andrew Parker, UCLA Scott Burleigh, JPL Richard Guy, UCLA Deborah Estrin, UCLA CENS Seminar

Interplanetary Network Vision • Picture of sensor networks in space * V. Cerf, InterPlanetary Internet, 2004 CENS Seminar

Why We Care CENS Seminar

Outline • Mexico Seismic Array Deployment: Set Up • Interplanetary Networking: Set Up & Relevance • Delay Tolerant Networking (DTN): Architecture • DTN vs. Email • Mexico Deployment: DTN Approach • Modular DTN Architecture for Sensor Networks CENS Seminar

Mexico Deployment • People: Paul Davis, Deborah Estrin, Richard Guy, Martin Lukac, John Wallace, Monica Kohler, Ramesh Govindan, Igor Stubailo, Allen Husker, Katie Mika, John Propst, Sam Irvine, Jeremy Elson, et al. • Seismic array of 50 nodes • 5 km apart • Spanning 250 km • Through jungle, mountain, and urban environments • Targeting end of Q1 2005 CENS Seminar

Data Story CENS Data Communications Controller • Every bit is important. Can’t lose data. • 24 bits / sample, 100 samples / sec, 3 channels = 900 bytes/sec, 3.24 MB/hour, 1.2-1.8 MB/s compressed, per Node • 1 GB Flash will hold 20 days of compressed data • Conventional methods of retrieval • Satellite uplink (expensive) • Data mule (graduate student) • Is there a better way? Q330 24 bit Digitizer Guralp Seismometer Stargate CENS Seminar

A Better Way: Networking the Array • Goal: Multihop data from the edge into Mexico City • Mexico deployment is different from typical sensor network applications • Every bit of data is important • Radio connectivity is directional • Due to distance, power, and regulations, we’re using directional antennas • Routing is over a linear topology • Radio communication is not the greatest power consumer CENS Seminar

Braided String • Introduces some path redundancy • 802.11b • 200 mw radio (SMC PCMCIA card) • Yagi antenna • 20 dbi • 30 degree spread • Tested with 2-way splitter • 5 - 10 km in LA area • 4 Mb/s - 150 kb/s • Parabolic antenna • 24 dbi • 5 - 10 degree spread • 4 way splitter • Tested to ~ 25 km • Closer to 1 Mb/s CENS Seminar

Routing is Hard to Do Right • Possible approach: Static Routes - topology is linear and nodes are immobile • Problem: Still as brittle as a single string -- can’t back track around breaks • Possible approach: Use AODV, DSR, Roofnet, etc. • Problem: Flapping links will make establishing end-to-end routes nearly impossible • What to do, what to do…? CENS Seminar

Interplanetary Networking: Motes in Space • Huge transmission delays • Several seconds to the Moon • 5 - 20 minutes to Mars • 1 hour to Jupiter • 7 hours to Pluto • Lossy links • Long disconnects • Sometimes predictable CENS Seminar

Mars Communication http://www.astrosurf.com/lombry/qsl-mars-communication3.htm CENS Seminar

Mars Connectivity CENS Seminar

Oops! CENS Seminar

Properties of Interplanetary Networking • Links are lossy and high delay • Very high round trip times, • Even for single hop • Links are often disrupted • Node mobility • Node powered down • Asymmetric links • Asymmetric node capabilities • End to end paths unreliable • Interactive / chatty protocols break • Need to communicate across varying network technologies, including non-IP networks CENS Seminar

Delay Tolerant Network Architecture • Messaging service (Like Email) • Insulates applications from network behavior • Asynchronous, deferred transmission • Non-interactive end-to-end You’veGot DTN! CENS Seminar

Delay Tolerant Networking • Bundles are Basic Unit of Transport • Written as files onto persistent store • Resilient across server / node restarts • Custody Transfer of Bundles • Represents a QOS agreement: custodian tries real hard to transfer custody and not delete until this has happened • Conceptually moves the “end” of an end to end transaction • Custody is asserted, rather than given. • Overlay routing of Bundles among Custodians • Custodians may be separated by intermediate nodes. CENS Seminar

Delay Tolerant, Not Run-Over-by-a-Bus Tolerant • Bad things can still happen with Custody Transfer • Brings up the question of trust. When should a node trust another more than it itself? • Risk vs. Resource trade-off VS CENS Seminar

Bundle Agents and Clients • Client nodes register with Bundle Agents to send and receive Bundles on their behalf • Email analogy: Pop or IMAP client connecting to a server • Based on the destination name, delivery is made to the corresponding Bundle Agent. • At this point it’s considered to be done. Again, similar to Email. • Naming scheme allows for late binding and separation of names from nodes • Determination of which node(s) receive the message may be deferred • Connects different domains via gateways • Late binding names allow different domains to communicate (through gateways). • Connects loosely coupled “Internets” • Official DTN specification uses Tuples: • (Interdomain Label, Opaque Intradomain Label) CENS Seminar

DTN vs. Email DTN? Email? What’s the difference? Professor Culler CENS Seminar

DTN vs. Email Biggist Difference: DTN Overlay DTN is able to make progress towards the destination, even when no contemporaneous route exists Cool Professor Culler CENS Seminar

Mexico Deployment: Routing • DTN has the “Tuple” address: (Interdomain, Intradomain) • Doesn’t really apply to Mexico • Single domain • Hybrid Routing Approach • You know where you want to get to, but not how • Overlay static on top of mesh routing • Using Roofnet • link-quality aware • minmizes number of transmissions • Use mesh routing to tell you the next hop towards the furthest reachable downstream candidate CENS Seminar

Hybrid Examples • Normal case • Simple break • Partition • Reconnect CENS Seminar

Mexico Deployment: Handling Data • DTN suggests exchanging Bundles (Files) • Divide data into one-hour segments • Augment with meta-data • To, From, Data, Size, etc. • 1.2 - 1.8 MB compressed • Store as files on disk (even on intermediate hops) • Easy to recover from server and node restarts • Human manageable CENS Seminar

Node Software Architecture Seismic Data (Q330) Upstream Nodes CENS NODE Duiker Inbox Drafts Bundle Data File Bundle Data File Bundle Data File Bundle Receiver Routing Data Bundle Forwarder Bundle Sender Outbox Bundle Downstream Node Bundle Bundle CENS Seminar

Managing Space • Nodes are equipped with 2 - 4 GB of storage: 40 - 80 days of data • A Bundle is deleted when ACK is received from sink for locally generated data • ACK is just another bundle • Application-level ACK • When space gets low • Node refuses to accept transient data (“route around me”). Better than accept and drop. • Delete transient data when space gets low • No intermediate custody transfer • This policy delays the deletion of original data the longest CENS Seminar

Transmission Priority • Transmission priority affects performance • Want to avoid starvation • Priority can be based on: • Age • Transient vs. Local • Number of hops traversed • Number of times sent • Some other measure of how hard it was to get this far CENS Seminar

What about Off the Shelf DTN? • There is an official Bundle specification • There are 1 1/2 reference implementations of the Bundle specification • Why not use it? • Because… • Only reason why Mars worked, and only way Mexico will work, is because scenario specific information was used. You MUST do this. • It’s impossible to do a good job in all scenarios • The more generic you try to be, the more it looks like flooding. • The more options and switches you support, the more brittle and complex it becomes • See Sendmail configuration file • How do we remain flexible and customizable, yet stay SIMPLE? CENS Seminar

A Modular DTN Software Architecture • Like Click • Modular software IP router • Easy to plug modules together • Easy to create new modules • Results in an optimized router // Declare three elements… src :: FromDevice(eth0); ctr :: Counter; sink :: Discard; // .. Connect them together src -> ctr; ctr -> sink; CENS Seminar

Click’s Elegance • Click modules consume and produce the same data structure • IP packets • Same is nice and simple. Easy to compose modules if they input/output the same thing • IP is nice too (as opposed to something obscure or too generic) • Strong foundation in specifications • Many compliant implementations as examples • People are already familiar with certain features and behaviors of IP routers CENS Seminar

Click and DTN - High Level Click DTN • IP packets • IP specifications • Lots of existing IP routers • Lots of specific IP features/behavior to choose from • Bundles • Bundle specifications • Experimental and evolving • Very few Bundle servers • Lots of Bundle features and behavior to choose from • In this area, Bundles have a richer set of potential behavior than IP CENS Seminar

Introducing this Approach to Sensor Networks User Kernel Emulation Array Client Server Emulation Mode HostMote Protocol /dev/servicename /dev/fusd MN MN MN MN Node 001 Node 002 Node 003 Node N … kfusd.o Robust multi-process, microkernel architecture Simulation Framework with real RF channels Visualization Tools Transceiver Transceiver Transceiver Transceiver Mote Mote Mote Mote • EmStar • Component based framework for sensor network applications (on Linux) • Lots of services and applications already existing • Simulation / Emulation / Deployed Modes • Heterogeneous applications (Stargates / Motes) • Debugging, monitoring, and visualization • Growing user-base • UCLA, USC, MIT, Umass Amherst, Ohio State, etc. • Intel Research, Xerox PARC, Microsoft Research CENS Seminar

Relationships Between EmStar Modules Unclear Example EmRun Configuration File include link/link.run &link_udp(udp0); &link_linkstats(udp0,ls0,show="leds:core"); &link_neighbors(ls0,show="hide"); process mdiff { type = once; noclean; cmd = "devel/microdiff/mdiff --uses ls0"; waitfor = ls0; } process mdiff_test { waitfor = mdiff; type = once; noclean; cmd = "devel/microdiff/mdiff_test"; } process mdiff_filter { waitfor = mdiff; type = once; noclean; cmd = "devel/microdiff/mdiff_filter"; } How to bring Click’s readability and ease-of-use to EmStar? CENS Seminar

Bolt: Architecture Description Language for EmStar • Makes relationships between modules of an EmStar application explicit • Began as a class project with Eddie Kohler and Todd Millstein last spring • Implemented in CIL and Ocaml CENS Seminar

Bolt Config.: White Board to Emacs Buffer udp0 = udp() ; ls0 = linkstatsd(); nd = neighbord(); mdiff = Mdiff(); mdiff_simple_app [packet_dev] <-> [app_packet_opts] mdiff; mdiff_simple_filter [packet_dev] <-> [filter_packet_opts] mdiff; mdiff [lu_opts] <----------------------> [lp_opts] ls0 [lu_opts] <-> [opts] udp0; mdiff [sc_opts] <- [s_opts] nd [ls_opts] <-- [s_opts] ls0; nd [lu_opts] <-> [lp_opts] ls0; Instantiating components Connecting component devices Data Flow CENS Seminar

BUT WAIT, THERE’S MORE!… CENS Seminar

Click Modules vs. EmStar Modules • Click modules are C++ classes • EmStar modules are separate processes • Click modules communicate via function calls • EmStar modules communicate by passing bits or text via device files • Since Click modules are compiled, they benefit from basic compiler checks and optimizations. Click itself does higher level checks and optimizations • Wouldn’t it be nice if EmStar modules enjoyed similar benefits? CENS Seminar

Bolt: Compiler-like Analysis and Optimization for EmStar • It Slices! It Dices! • It does static analysis and optimizations not otherwise possible across EmStar modules • No user changes or annotation required of EmStar code • Though it could help • Bolt statically infers type-safety violations between two functions in different processes (EmStar modules) • Despite type-obscuring function calls and heavy use of function pointers • Bolt uses subgraph isomorphism to identify andswap out combinations of modules with more efficient combinations that perform the same function • Analysis not completely sound nor complete, but hard given the circumstances CENS Seminar

Implementing DTN Using Bolt • DTN looks like any other set of EmStar modules • What exactly are the EmStar modules that make up the DTN suite? • Click has benefited from the existence of numerous IP router implementations • DTN must build up experience from real deployments (Mexico and others) • Identify and extract reusable DTN related components, for example: • Managing a large number of concurrent file transfers between a pair of nodes • Despite large delay and disconnections • Recover across restarts • LTP (S. Burleigh) & File Mover (A. Parker) • Goal is not necessarily to be able to build a complete DTN solution for your particular situation • Rather, to reuse the ones that make sense, and build from there CENS Seminar

References • DTN: http://www.dtnrg.org • IPN: http://www.ipnsig.org • Click: http://www.pdos.lcs.mit.edu/click • EmStar: http://cvs.cens.ucla.edu/emstar • Bolt: http://lecs.cs.ucla.edu/~adparker/Bolt CENS Seminar

Adopting Ideas from Interplanetary Networking for Sensor Network Applications