Efficient Simulation of Large-Scale P2P Networks: Compact Data Structures

1. Efficient Simulation of Large-Scale P2P Networks:Compact Data Structures PDP 2007, Naples, Italy, February 7-9, 2007 Andreas Binzenhöfer, University of Würzburg binzenhoefer@informatik.uni-wuerzburg.de

2. Authors • University of Würzburg • Andreas Binzenhöfer • Tobias Hoßfeld • Technical University of Munich • Gerald Kunzmann • Hamburg University of Technology • Kolja Eger • Technical Report • Daniel Schlosser (Würzburg) • Robert Nagel (Munich) • Jens Oberender (Passau) • Ivan Dedinski (Passau)

3. Categories of Simulation Efficiency

4. Outline • Queue Efficiency • Calendar Queue in P2P Simulations • Event Efficiency • Periodic Events • Dynamic Events • Kademlia Bucket Refreshes • State Efficiency • Process Handlers • Conclusion

5. Queue Efficiency Reducing the time needed to handle events

6. Problem Definition Hold Time = Enqueue + Dequeue Operation list heap calendar queue

7. * * Calendar Queue • Works like a regular desktop calendar • Each year has Nd (=365) day • Each day consist of Td (=24) hours • Enqueue operation: Calculate index as • Dequeue operation: Move from day to day until event found Scheduled for: current year next year year after next year Day 1 Day 2 Day 3 Day Nd

8. Enqueue Operation no longer in O(1) Too many days: Dequeue Operation no longer in O(1) Calendar Queue: Problems Very long days:

9. 4096 days of length 100ms each

10. 32768 days of length 1ms each

11. event-file Two Queues for User and Simulation Events • event file can be given to different simulators • new user models without having to change the simulator • log files and real traces can be translated into simulator input • simulation time reduced due to the pre-calculated events joins leaves searches messages timeouts maintenance routines

12. Event EfficiencyReducing the number of events

13. Kademlia Bucket Refreshes • Roughly log2(n) buckets for each peer (where n = overlay size) • Refresh bucket as soon as it has not been used for one hour • Example: peer X 4380 bucket 1 3960 5820 bucket 2 bucket 3 Time of next refresh

14. Simple Solution (additional memory required): bucket i Obsolete refresh events are being skipped Kademlia Bucket Refreshes Simple Solution (no additional memory required): • Algorithm: • search event • delete event • insert new event Time consuming bucket i Refresh event moved every time peer uses bucket • Algorithm: • search event • delete event • insert new event • store time of next • refresh at peer Many redundant events X X X

15. Algorithm: • search event • delete event • insert new event • store time of next • refresh at peer Kademlia Bucket Refreshes Improved Solution (Simulation time): log2(n)·n refresh events: 1.7 million for 100000 peers bucket i Store actual time of next refresh for each bucket Improved Solution (Memory): bucket 1 current minimum bucket 2 bucket 3 Use only one single event for all buckets of a peer

16. State EfficiencyReduce memory needed to represent states

17. Process Handler • Avoid redundant information of parallel processes • Example – Parallel search: • Each participating peer stores similar information • This information is stored multiple times in RAM Day 1 Day 2 Day 3 Day Nd process handler Ra = 3 xyz = 48

18. Process Handler • Ra : Number of remaining pointers to the process handler • Last pointer frees memory • Advantage: Avoid redundant information Day 1 Day 2 Day 3 Day Nd process handler Ra = 3 xyz = 48

19. Example: Search Handler • Search Handler stores • Search initiator • Searched ID • Timeout values • Number of (positive) answers • … process handler Ra=3 sender=48 query=667 timeout=4567 answers=2

20. Conclusion • Typical structure of P2P events • allows for queue hold time of O(1) • does not require dynamic adaptation of Calendar queue • Careful design of events can dramatically reduce • simulation time • required memory • example: Kademlia bucket refreshes • Process handlers • exploit parallel/distributed nature of P2P algorithms • reduce redundant information

21. Q&A