1 / 5

GEC19 Developer Topics: Tools for Scaling GENI Experiments

In this session, we explore innovative strategies for scaling GENI experiments, particularly when testing with multiple Hadoop clusters. Instead of allocating excessive resources upfront, we propose the creation of a miniaturized version of your experiment, akin to an atom or molecule, to ensure essential functionality and streamline configuration. The goal is to facilitate easy scaling from small to large configurations while maintaining control and monitoring capabilities. Join the discussion on available tools, potential workflows, and collaborative opportunities to enhance scalability in experimental setups.

geneva
Download Presentation

GEC19 Developer Topics: Tools for Scaling GENI Experiments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GEC19 Developer Topics:Tools for Scaling GENI Experiments Marshall Brinn, GPO March 19, 2014

  2. Motivating Example Suppose I want to run an experiment involving 20 Hadoop clusters of 4 nodes each. • I could allocate 80 resources up front and work on configuring them. • But experience shows that this can be a frustrating waste of time and resources. • It also can be counter-productive, as we are simultaneously: • Trying to debug problems with essential functionality • Trying to fix problems with large-scale configuration

  3. Example [2] • Better, I should design a miniaturized version of my experiment • Like an atom or molecule: the very smallest version that still is recognizably the same thing in some sense. • Once that is running, I’d like to be able to “turn a crank” on my mini-configuration and produce a configuration that will run at a larger scale  …

  4. Many dimensions of scaling The experimenter bears the ultimate responsibility for making an experiment configurable or orchestrated. But what can we do to help scale those capabilities? • Being able to “turn the crank” involves scaling many different pieces • Resource topology: Nodes (of different types) and Links (representing different relationships) • Experiment configuration: The ability to configure a large configuration based on the template of the smaller (e.g. setting up who provides what services for which clients) • Experiment orchestration: The ability to control, instrument and monitor all the elements of the experiment in “the same way” as it was done in the small experiment

  5. What can we do? These slides are intended to start a conversation within the developer community that I’d like to continue at the Coding Sprint this PM. • What do we have now among all current GENI tools and libraries that would help with this? • What are we planning to do in (or could we add to) the upcoming set of tool development efforts? • Might a copy-and-paste capability on GUI’s enable this kind of topology/config scaling? • What kind of workflow might we be able to offer “on the cheap” to experimenters to achieve this kind of configuration scaling • That is, it doesn’t all need to be one tool but could be a workflow drawn from several tools

More Related