Matthias Keller , Jan Beutel , Lothar Thiele

1. How Was Your Journey? Uncovering Routing Dynamics in Deployed Sensor Networks with Multi-hop Network Tomography Matthias Keller, Jan Beutel, Lothar Thiele SenSys 2012, Toronto, ON, Canada

2. Multi-hop data collection • Tree-based routing protocol • Low-power operation

3. Packets from C C B End-to-end delay S A Why did the end-to-end packet delay increase? A: Node C was disconnected B: Node C suffers from a weak link D: Weak link at upstream node C: Backpressure

4. Problem: Missing Global Network State Detailed analysis requires network state from all nodes at all times F F A C A C S B S B D E D E Tomography Only partial information frompacket sources Global network state

5. Multi-Hop Network Tomography F A C Exclusion of reordering Packet correlation S S B D E 1 1 2 2 1 • Incoming packets • Information frompacket source • Order of arrival at the sink • Per-packet network path • Per-hop arrival order • Per-hop arrival times

6. System Model A B C S 1 1 2 2 1 Sensor nodes • Single FIFO queue • Generate packets • Forward packets of other nodes Source address Sequence number First hop receiver Sojourn time + Payload + Set once at source Updated in network

7. Information Reconstruction S A B C 1 1 2 2 1 For every packet: • Start at packet source • Go to next hop • At this hop: Locate two anchor packets • Last packet generated before our arrival • First packet generated after our arrival • Extract next hop from anchor packets 1 2 Base for anchor packet selection 3 4

8. Packet Correlation Example S B C A 1 1 2 2 1 For packet : 1 Anchor packets Next hop Timing + next hop B 1 1 2 A Timing + next hop Select anchor packets 2 1 S • Per-packet network path • Per-hop arrival order • Per-hop arrival times

9. The Problem of Path Changes Path change! A B C 2 1 2 1 S 2 1 2 1 2 1 1 1 1 D 1 1 Three orderings are possible at the sink: E correct ordering 2 1 1 1 incorrect ordering 1 2 1 1 incorrect ordering 2 1 1 1

10. Exclusion of Reordering Packet classification • Guaranteed to not have beenreordered • Ready for packet correlation “Reliable” packets Packet stream Not “reliable” packets • May have been reordered • Removed from tomography

11. Definition: Reliable Packet A packet kis reliable, if it fulfills two properties: From our observations at the sink, we can guarantee that • packet kcan only have travelled along exactly one path, and that • the order relation between packet kand any other reliable packet is consistent along all packet queues in the network including the sink.

12. Determining a Reliable Set For every packet: • Start at packet source • Go to next hop • Determine possible anchor packets (worst-case analysis) • Single next hop receiver? • Arrived at the sink in order? • Extract next hop receiver 1 2 3 Stop on error “Reliable” if next hop is sink 4

13. Verification of Single Possible Path • Idea: At each hop, verify that packet can only have left to one single next hop • Problem: Per-hop timing is yet unknown A B E Per-hop worst-case analysis: Next hop Single possible next receiver E E E E 1) Next hop E E A A Multiple possible next receivers 2) Next hop E E E ? Uncertainty due to lost information 3) Window of possible arrival

14. Verification of Consistent Ordering Generally hard problem, easier after single path verification E E E • Packets that can only have left to a single next hop are always surrounded by at least two potential anchor packets that point to the same next hop Next hop 1 Anchor packets 3 2 1 Forwarded traffic 1 1 • Theorem (proof in paper): • If all potential anchor packets arrived at the sink in order, also forwarded packets were not reordered at this hop 2 1 1 2 1 3 Guaranteed to not have been reordered at E

15. Validation & Evaluation • Testbed experiments for comparison with ground truth • 90 TMote Sky nodes running CTP Noe/LPL (TWIST) • 25 TinyNode nodes running Dozer (FlockLab) • Real-world deployments (PermaSense) • >140 million packets • Simulation in Castalia (100-hop line) • Explore scaling properties and limitations a b c # of packets with reconstructed path information Reconstructed packets = # of packets received

16. Testbed Tomography Results Dozer 25 TinyNode nodes CTP Noe 90 Tmote Sky nodes Reconstructed information is correct in all cases Inter-packet interval (IPI)

17. PermaSense Deployments • 10-40 TinyNode nodes running Dozer • Data yield >98% • Matterhorn, 2008, >78 million received packets • Jungfraujoch, 2009, >48 million received packets • Dirruhorn, 2010, >20 million received packets • Aiguille du Midi, 2012

18. Deployment Tomography Results Artifacts of unintentionally removed and now missing packets in data repository Matterhorn: 99.5% Jungfraujoch: 91.5% Dirruhorn: 93.7%

19. Sensitivity of Performance to Data Yield CTP Noe (testbed) Dozer (testbed) CTP, 100 hop line (simulation) Dozer, large delays (testbed)

20. Accuracy of Per-Hop Arrival Times 90th Percentile of Arrival Time Uncertainty [Distance between earliest and latest arrival at a node] Accuracy of obtained arrival times is not a function of the packet delay, but upper bound by the IPI

21. Stepping Stone for Passive Monitoring • Non-intrusive information reconstruction at the sink is possible • Testbed experiments with CTP and Dozer proof the correctness of reconstructed data • Results from very large data sets confirm applicability of the approach to real-world problems • Accuracy of tomography is sensitive to the amount of data (yield), but not to its age (packet delay) www.permasense.ch