1 / 50

The DDS Benchmarking Environment

The DDS Benchmarking Environment. James Edmondson jedmondson@gmail.com. Vanderbilt University Nashville, TN. Topics. What is the DBE? What technologies does the DBE use? How does the DBE setup the environment? What problems had to be dealt with? What features will be added?

howell
Download Presentation

The DDS Benchmarking Environment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The DDS Benchmarking Environment James Edmondson jedmondson@gmail.com Vanderbilt University Nashville, TN

  2. Topics • What is the DBE? • What technologies does the DBE use? • How does the DBE setup the environment? • What problems had to be dealt with? • What features will be added? • Demonstration • Conclusion and Wrap Up

  3. What is the DBE? • DDS Benchmarking Environment • Sets up the testing environment and framework • Configures and launches implementation specific services • Launches Publishers and Subscribers • Coordinates interaction, logging info, and stats collection • Allows for easily scaling the number of participants • Sets up the statistics for collection by Bagatel Publishers Throughput Networking Subscribers

  4. What is the DBE? • May be used to test other middleware or technologies • Focus is on documented, reusable resources Versatility Dependability Publishers Throughput Networking Subscribers

  5. What does the DBE do?

  6. What does the DBE do?

  7. What does the DBE do?

  8. What does the DBE do?

  9. What does the DBE do?

  10. What does the DBE do? Start the Subscribers

  11. What does the DBE do?

  12. What does the DBE do?

  13. What does the DBE do?

  14. What does the DBE do?

  15. What does the DBE do? Start the Publisher

  16. What does the DBE do?

  17. What does the DBE do?

  18. What technologies does the DBE use? • Scripts are written in Perl and use PerlAce module • Remote execution is accomplished via SSH • Logs, stats and test info are stored on a NFS partition • All tests are currently ran on Fedora Core Linux machines • Configured to test NDDS, TAO DDS, and Splice • Synchronization between tests is maintained by using lockfiles DBE Linux SSH NFS Perl lockfiles

  19. How does the DBE setup the environment? • Most path information is updated using the ~/.ssh/environment file • Some custom environment variables are set by DBE scripts on individual clients • Individual Test IDs are assigned to each test • Directories are created on the NFS partition for stats and logs DBE

  20. How does the DBE setup the environment? • Most path information is updated using the ~/.ssh/environment file • Some custom environment variables are set by DBE scripts on individual clients • Individual Test IDs are assigned to each test • Directories are created on the NFS partition for stats and logs DBE

  21. How does the DBE setup the environment? • Most path information is updated using the ~/.ssh/environment file • Some custom environment variables are set by DBE scripts on individual clients • Individual Test IDs are assigned to each test • Directories are created on the NFS partition for stats and logs DBE Change $ENV

  22. How does the DBE setup the environment? • Most path information is updated using the ~/.ssh/environment file • Some custom environment variables are set by DBE scripts on individual clients • Individual Test IDs are assigned to each test • Directories are created on the NFS partition for stats and logs DBE Change $ENV

  23. How does the DBE setup the environment? • Most path information is updated using the ~/.ssh/environment file • Some custom environment variables are set by DBE scripts on individual clients • Individual Test IDs are assigned to each test • Directories are created on the NFS partition for stats and logs DBE Change $ENV Generate ID

  24. How does the DBE setup the environment? • Most path information is updated using the ~/.ssh/environment file • Some custom environment variables are set by DBE scripts on individual clients • Individual Test IDs are assigned to each test • Directories are created on the NFS partition for stats and logs DBE Change $ENV Generate ID

  25. How does the DBE setup the environment? • Most path information is updated using the ~/.ssh/environment file • Some custom environment variables are set by DBE scripts on individual clients • Individual Test IDs are assigned to each test • Directories are created on the NFS partition for stats and logs DBE Change $ENV Generate ID Create Directories

  26. How does the DBE setup the environment? • Publishers and subscribers are notified of test settings • Result file locations • QoS settings on a per subscriber basis • Network settings • Test specific settings (number of messages to send, etc.) Pass Test Info DBE Repos Publishers Subscribers

  27. What problems had to be dealt with? • Lack of a terminal with remote command execution with SSH • TCP and UDP buffer sizes were too small • Synchronization amongst publishers and subscribers Subscriber Next Test Subscriber Wait Publisher Publisher Publisher Buffer Buffer

  28. What problems had to be dealt with? • Flock() doesn’t work as expected with stateless NFS flock(“myfile”) touch(“myfile.lock”) • Shared libraries are referenced locally • Block buffering of stdout block buffering stdout file stderr file immediate

  29. What problems had to be dealt with? • More problems on request ;)

  30. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE

  31. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Create Publisher

  32. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Create Publisher

  33. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Create Publisher Publisher

  34. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Publisher

  35. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Create Subscriber Publisher

  36. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Create Subscriber Publisher

  37. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Create Subscriber Publisher Subscriber

  38. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Publisher Subscriber

  39. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Create Subscriber Publisher Subscriber

  40. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Create Subscriber Publisher Subscriber

  41. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Create Subscriber Publisher Subscriber Subscriber

  42. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Publisher Subscriber Subscriber

  43. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Kill Subscriber 1 Publisher Subscriber Subscriber

  44. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Kill Subscriber 1 Publisher Subscriber Subscriber

  45. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Kill Subscriber 1 Publisher Subscriber

  46. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Kill Subscriber 1 Publisher Subscriber

  47. What features will be added? • New tests will require new features • One such feature is per node scripting of subscribers and publishers • Feature will allow dynamic starting and stopping of participants DBE Publisher Subscriber

  48. What other additions are planned? • The ability to launch logging services • Memory and CPU usage will need to be recorded • The ability to test other middleware and applications

  49. Demonstration Primer messages = 100 Stats messages = 1000 Test type = throughput Implementation = NDDS

  50. Conclusion and Wrap Up • The DBE is an excellent suite of tools for automating DDS testing on a networked environment • New test types will be added to gain further insight into the DDS implementations • Many new features will also be added to better reveal problems and benefits of the various DDS products • We would like to thank our AFRL and Vanderbilt Sponsors for their continued support

More Related