Historic Integrity in Peer-to-Peer Systems

1. Historic Integrity inPeer-to-Peer Systems Petros Maniatis, Mary Baker Computer Science Department Stanford University

2. Talk Outline • Motivation • The Problem • Secure Timelines • Timeline Entanglement • Evaluation • The Future Stanford Networking Seminar

3. Motivation Time and history are really important!

4. Time and History in Distributed Systems • Time • A clock—logical or physical—assigning “time” to system events • Time stamp • The “time” when a system event occurs • History • All system events that ever happened • The time stamps of those events Stanford Networking Seminar

5. Who Cares About Time? • Resource management / mutual exclusion • Two requesters vie for the same resource • Earlier requester wins • Prior art issues and time stamping • Two parties invent the same thing • He who publishes it first wins (a patent, glory, a PhD, etc.) Stanford Networking Seminar

6. Who Cares About History? • Backups • Checkpoints • Archival storage • Version control systems (CVS) • Versioned file systems (VMS, Plan 9, Elephant) • Accounting Stanford Networking Seminar

7. Who Cares About History? (2) • Reputation systems • Was the reputation of this account accrued or magically built overnight? • Accountable systems • E.g., p2p storage • At time t, node A accepted document D • At time t1>>t, node A claims it never saw document D • Can I catch the fraud? Stanford Networking Seminar

8. The Problem What’s there and what isn’t

9. Approaches • Logical clocks [Lamport 1978] • Partial temporal ordering of all events globally • Everyone is well-behaved • Network synchronization • Keeps all well-behaved nodes synchronized • Assumes well-known bounds on misfortune • Secure time stamping • Trusted third party maintains ordering of events • It gives authoritative answers on what happened when Stanford Networking Seminar

10. Why Is That Not Enough? • P2P systems are vulnerable to malice • They are chaotic • They are fragile • They are very dynamic • They are large • Past approaches are • Vulnerable to malicious attacks, and/or • Unscalable, and/or • Not survivable Stanford Networking Seminar

11. Our Approach • We solve a narrower, more “selfish” problem • Every peer • Maintains a local view of global time and history • Safeguards the integrity of portions of global history it cares about • Only trusts itself or provable information • Requirements • Efficient, scalable, survivable, aggressively decentralized • Timeweave: a tamper-evident historic record for a distributed system Stanford Networking Seminar

12. Secure Timelines The fingerprints of history

13. Logical Clocks • A “counter” of the state changes of a system • The clock advances by one tick • Every time the system state changes • Every time a message is sent • Every time a message is received • I can change my record of old states and nobody knows • I can move my clock backwards and nobody knows Stanford Networking Seminar

14. Secure Timelines • A secure timeline is • An “authenticated” logical clock, and • A historic record of the state transitions of a system • Every time step has • A logical time (a counter of time steps) • An annotation (a fingerprint of the current system state) • An authenticator Stanford Networking Seminar

15. Secure Timelines (2) • Authenticator is a one-way hash of: • Previous logical time • Previous authenticator • Previous state Stanford Networking Seminar

16. One-Way Functions andThe Arrow of Time • A function f is one-way if • Given f(a) it is intractable to find a (preimage) • Given <a,f(a)> it is intractable to find b such that f(b)=f(a) (second preimage) • It is intractable to find <a, b> such that f(a)=f(b) (collision) • SHA1, for example, is considered one-way • If f(a)=b, then I can safely assume that a was known before b • Otherwise, someone must have found the appropriate a for b, either intentionally or by luck Stanford Networking Seminar

17. Secure Timelines Are Tamper-Evident • If authenticator 5 is known • I can request a proof that the previous state was state 4 • Proof is the application of the one-way function to state 4, yielding authenticator 5 • Similar proofs for previous states or authenticators • There is no way that the system can prove any other state as its state at time 4 or before Stanford Networking Seminar

18. How Do Timelines Work? • Holders of timeline authenticators can ask: • Was State3 the state of the system at time 3? • Did State2 precede State4? • Answers are provable: • A proof is a trace of the one-way functions applied • If it leads to the current authenticator, it’s valid Stanford Networking Seminar

19. How Do Timelines Work? (2) Stanford Networking Seminar

20. What Is a Timeline Good for? • Did event E happen at time t? • Find the state at time t • Check if event E is included in the state • Did event E happen before event E’? • Find the state S including event E • Find the state S’ including event E’ • If S precedes S’, E happened before E’ • Only meaningful within a single system • How can I extract temporal orderings of events across different systems? Stanford Networking Seminar

21. Timeline Entanglement A bridge over trust boundaries

22. Distributed Timelines • A secure timeline spans the states of a single system • How do we transition to a distributed system of mutually distrustful peers? • Major security requirement: • Correctness: If a precedes b, no peer can convince me that b precedes a • Timeline Entanglement • Include authenticators from one timeline into the state of all others Stanford Networking Seminar

23. Timeline Entanglement • Every node stores samples of the timeline of every other node in the group • “All-to-all” entanglement • Inclusion of my sample in a foreign timeline builds a precedence • From the past of my timeline • To the future of the foreign timeline • Done regularly (not only when messages are sent) • Say every 10 minutes Stanford Networking Seminar

24. Timeline Entanglement (2) • A sends B a “timeline thread” • Signed <A,i,authenticatori> • B includes the thread in its system state • In a thread archive • B sends A an “entanglement receipt” • Contains a proof of thread storage • Shows that A’s thread is included in the computation of B’s next authenticator Stanford Networking Seminar

25. Entanglement Checks • Continuity • B checks that A’s thread follows the same timeline as last time they talked • Acknowledgement • Receipt shows that B archived A’s thread in its current state • Reverse continuity • A checks B’s receipt against past threads from B Stanford Networking Seminar

26. Entanglement Properties • Commitment • After A tells B its authenticator, it can’t change its timeline’s past • After B acknowledges A’s thread, it can’t change its thread archive • Cross-domain ordering • Events before <A,3> precede events after <B,2> provably Stanford Networking Seminar

27. Local View of Global History • Through entanglement, A samples B’s timeline Stanford Networking Seminar

28. Local View of Global History • Through entanglement, A samples B’s timeline • This gives A a “coarse” view of B’s history Stanford Networking Seminar

29. Local View of Global History • Through entanglement, A samples B’s timeline • This gives A a “coarse” view of B’s history • Coarseness results in loss of temporal resolution • X, Y and Z all appear to A temporally indistinguishable Stanford Networking Seminar

30. Local View of Global History • Through entanglement, A samples B’s timeline • This gives A a “coarse” view of B’s history • Coarseness results in loss of temporal resolution • X, Y and Z all appear to A temporally indistinguishable • Entanglement frequency determines granularity Stanford Networking Seminar

31. Entangled Node Architecture • Secure timeline • Timeline thread archive • Contains all threads received from peers • Included in the node’s accountable state • Necessary to prove I have archived a thread • Entanglement receipt archive • Holds receipts received for outgoing threads • Not included in accountable state • I don’t need to prove I’ve archived the receipts for my threads Stanford Networking Seminar

32. How Does Timeline Entanglement Work? Stanford Networking Seminar

33. Evaluation Benefits, performance and scalability

34. Entanglement Benefits • Secure Temporal Mapping • I can map a remote logical time onto my own trust domain • Historic Integrity • I am a witness to all remote timelines • If I care about a remote event and it changes, I’ll catch the fraud • Survivability • I can prove the temporal ordering of local events with remote events, even after the remote timeline dies Stanford Networking Seminar

35. Secure Temporal Mapping • A whishes to know when B’s time was 2 • It looks in its outgoing threads for a thread arriving at B at or before 2 • It looks in its incoming threads for a thread sent from B after 2 • These map B,2 to A’s interval (1,7) • X maps to interval (1,7) and, therefore, precedes Y Stanford Networking Seminar

36. Why Is Temporal Mapping Useful? • Any event at remote timelines can be mapped to the local timeline • Any ordering among events is reduced to a locally verifiable ordering • I don’t need to trust the remote timelines to decide what came first • Limitation: mapping loses some temporal resolution Stanford Networking Seminar

37. Performance • How large a problem can we solve before the solution becomes unmanageable? • Free variables • Size of entangled set (group size) • Entanglements per time step per node • Metrics • Maintenance overhead • CPU & Disk I/O used by thread production, verification, location and storage • Communication overhead • Sending data rate Stanford Networking Seminar

38. Performance for Increasing Entanglement Load • 10-minute, 1-hour and 6-hour entanglement resolutions Stanford Networking Seminar

39. Performance for Increasing Group Size • 0.5, 1 and 2 entanglements per time step Stanford Networking Seminar

40. Future Directions Limitations, extensions, applications

41. Complete and Connected Entanglement Graphs • Entanglement graph is complete • Every peer entangles with every other peer • Complete entanglement has limited scalability • For high resolution (10 minutes), a group size of 600 nodes takes 100ms per step • To scale to larger groups, move to a less dense graph Stanford Networking Seminar

42. Transitive Entanglement • Nodes only entangle with their immediate neighbors • B only entangles with A,C, D, G and J • Arbitrary mappings are performed through successive time hops • To map from B’s timeline to I’s timeline, first map to G and then to I • Routing algorithms use the “aggregate temporal loss” metric • To choose between B-G-I and B-J-I, check which induces the least temporal resolution loss in the mapping Stanford Networking Seminar

43. Globally-Authenticated Historic File Systems • Technique • Retain history of file system changes (e.g., Plan 9) • Maintain a secure timeline of the file system state • Entangle timelines across a very large file server population • Benefit • Automatic global-scale file time stamping • Very difficult to delete unobtrusively documents that were originally included in the system Stanford Networking Seminar

44. Conclusion • Time and history are important concepts for current and future p2p systems • We extend the notion of logical clocks to build a tamper evident history of a system’s states • We combine logical clocks of many systems through timeline entanglement • An efficient mechanism to maintain a consistent, tamper-evident history of a large system Stanford Networking Seminar

45. The End Technical report and other info at http://identiscape.stanford.edu