OnTimeMeasure I ntegration with Gush

1. OnTimeMeasureIntegration with Gush Prasad Calyam, Ph.D. (PI) pcalyam@osc.edu Paul Schopis, (Co-PI) Tony Zhu (Software Programmer) Alex Berryman (REU Student) Prototype Design Last Updated: January 21st 2011

2. OnTimeMeasure Overview • Goal: • Provide GENI community with a shared measurement service for provisioning on-going and on-demand measurement requests • Expected Outcomes: • OnTimeMeasure Software to perform centralized and distributed measurement orchestration and provisioning of active measurements • Centralized orchestration for continuous monitoring, persistent measurements storage and processed network measurement feeds • Distributed orchestration for on-demand (real-time) measurement requests without need for persistent measurements storage • Measurement service that uses OnTimeMeasure software in GENI experiments to enable: • Network paths monitoring • Network weather forecasting • Network performance anomaly detection • Network-bottleneck fault-location diagnosis GENI Project Wiki – http://groups.geni.net/geni/wiki/OnTimeMeasure Researcher Web-portal – http://ontime.oar.net

3. Gush Overview • Gush runs on a user desktop and executes commands on Client nodes in a GENI Slice • The application description is written in an XML format with following specs: • Web addresses of software packages • List of target clients • Commands to be executed • Two configuration files are required in Gush (filled in using a helper script): • Directory.xml: defines where to locate resources during execution ‘port map slice’ • Gush queries PlanetLab database about user slice and auto populates the Directory.xml file • Gush.prefs: define working dir and client prefix Source: Gush project website - http://gush.cs.williams.edu

4. Advantages of OnTimeMeasure-Gush Integration • Improve Experiment Measurements in Classroom: • Students can instrument active and passive measurements within GENI; query/analyze measurement time series by using OnTimeMeasure-Gush • Get on-demand plots of measurement time series results of their experiments with simple commands in Gush • Don’t need to teach students GNUPlot • Installation Ease of OnTimeMeasure: • Configure once and re-use in new slices • Login and install on a number of nodes from user desktop • Gush takes care of ProtoGENI/PlanetLab authentication • Management Automation, Scalability and Security of OnTimeMeasure : • Identical configuration process on Node Beacons can be done in parallel across large number of nodes • GENI experimenter can securely script the request/management/query of OnTimeMeasure measurements through Gush to address a network science and engineering research issue

5. OnTimeMeasure Software Modules • Customizable software [*] developed at OSC/OARnet • Two main modules installed within a GENI experiment slice • Node Beacon • Installs tools that measure network health metrics such as: route changes, delay, jitter, loss, bandwidth • TCP/UDP Iperf, Traceroute, Ping, Pathload, OWAMP, etc. • Runs measurements based on a schedule and outputs results • Root Beacon • Installs Apache, MySQL and other packages • Creates database tables and configuration files • Generates measurement schedules for node beacons • Collects data and provides dashboard visualization, statistical analysis (i.e., anomaly detection and weather forecasting) with alarm generation NOTE: Typically, two or more Node Beacons and one Root Beacon need to be installed on separate dedicated servers in a slice; it is however possible to install Root Beacon on one of the Node Beacon servers • Download Node/Root Beacon Software – http://ontime.oar.net/download/OnTimeMeasure_latest.php [*] P. Calyam, C.-G.Lee, E. Ekici, M. Haffner, N. Howes, “Orchestrating Network-wide Active Measurements for Supporting Distributed Computing Applications”, IEEE Transactions on Computers Journal(TC), 2006.

6. OnTimeMeasure Software Modules (2) • Two modules that are external to the GENI experiment slice • OnTime Beacon • User Features: Web-interface client for users to login, manage slice information and interactively control Node/Root Beacons within their slices • Admin Features: User management (approve new users, maintain user information), Meta-data information repository NOTE: Typically, users do NOT need to install OnTime Beacon software and should use the instance hosted at - http://ontime.oar.net; GENI aggregate providers who wish to have a dedicated OnTime Beacon for their users can install their own instance of OnTime Beacon on a dedicated server • OnTime Control • Command-line interface client for users to login, manage slice information and control Node/Root Beacons within their slices in an automated manner NOTE: Typically, users need to install OnTime Control software in stand-alone mode on their desktops; if using experiment workflow tools such as GUSH, OnTime Control software will need to be installed on a Node/Root Beacon • Download OnTime Beacon Software (Beta!) – http://ontime.oar.net/download/OnTimeBeacon_latest.php • Download OnTime Control Software (Beta!) – http://ontime.oar.net/download/OnTimeControl_latest.php

7. OnTimeMeasure Architecture

8. OnTimeMeasure Centralized Orchestration • Centralized scheduling for continuous monitoring, persistent measurements storage and processed network measurement feeds • Useful for “network weathermaps” and long-standing experiments with advanced measurement analysis capabilities

9. OnTimeMeasure User Workflow (ProtoGENI or PlanetLab Aggregates) • User creates an experiment slice using ProtoGENI or PlanetLab control framework tools • For ProtoGENI slice creation, see - http://groups.geni.net/geni/wiki/OnTime-Install • For PlanetLab slice creation, see - http://groups.geni.net/geni/wiki/OTM-PlanetLabInstall • User registers at the “Researcher Web-Portal” (http://ontime.oar.net), provides Slice Rspec information and requests installation of measurement instance • Slice RSpec should include reservation of any required measurement resources • Each experiment slice needing a measurement service gets its own OnTimeMeasure software instance • Node/Root Beacons need to be installed as slivers based on the instructions provided in Step-1 for the specific aggregate • Valid login to the “Researcher Web-Portal” allows user to interact with the measurement service in his/her experiment slice. Specifically, the user can: • Submit measurement requests • Control the measurement service • Query measurement data

10. Deployment Steps for OnTimeMeasure-Gush Otm-deploy.xml directory.xml Root Beacon • Slice must be created using an RSpec file or using graphical interfaces • XML file of the application description must be created with at least two parts: • Commands to be executed only on the Root Beacon • Commands to be executed on all of the Node Beacons • Description of the slice resources must be written in directory.xml • Gush.prefs must be configured correctly as per Gush setup instructions Node Beacon gush.prefs Gush Controller Root Beacon Node Beacon Node Beacon GENI Slice

11. OnTime Control (Stand-alone Mode) Node Beacon Node Beacon • User installs OnTime Control on his/her desktop • Requires Python install • Manually configure slice topology in .yaml files • User executes python scripts that query a SQL database on the Root Beacon • Requires root db user information • Measurement result time-series files/plots are automatically downloaded into current directory on user desktop Slice Topolgy .yaml files Node Beacon DB GENI Slice Root Beacon User Desktop Python Scripts OnTime Control

12. OnTime Control (GUSH Integration mode) Node Beacon Node Beacon • OnTime Control package resides on Root/Node Beacon; Gush is installed on user desktop • Write a set of XML experiment files in the Gush format that interacts with the OnTime Control python scripts • Gush topology description (directory.xml) is passed to the OnTime Control • Output of the OnTime Control is a URL of a text/plot file containing the results of the measurement • Is it possible to auto-download URL information to current directory of user desktop within Gush? Node Beacon GENI Slice Python Scripts OnTime Control DB Root Beacon Slice Topology otm-control.xml directory.xml User Desktop GUSH Controller

13. Integration Timeline • Prototype • Software bundle and Wiki page (Milestone: End of January 2011) • OnTimeMeasure-Gush Demo • GENI experiment use case (Milestone: March GEC10) • GENI User Case Studies • Class for CDN at Williams College? (Milestone: ?)

14. Reference List • OnTime Control Wiki • http://groups.geni.net/geni/wiki/OTM-Control • OnTimeMeasure-PlanetLab Wiki • http://groups.geni.net/geni/wiki/OTM-PlanetLabInstall • OnTimeMeasure-Gush Wiki • To be created