ExoGENI : tools and capabilities for developers

1. ExoGENI: tools and capabilities for developers Ilya Baldin, Paul Ruth, AnirbanMandal, Jonathan Mills, Claris Castillo, Chris Heermann RENCI, UNC-CH Jeff Chase, Victor Orlikowski Duke University

2. What’s new for GEC20? • Image snapshot script • Easy way to capture the ExoGENI image • Tested and ‘GPO approved’ • Replaces ‘playpen’ • Speaks-for in controllers • New libraries for building tools

3. Plans for GEC21 • Release ndlLib • Port Flukes to use ndlLib • Develop other applications with it • Image capture • Integrate with NEuca • Make it work for bare-metal nodes • Obsolete playpen – it was a temporary awkward solution.

4. More on what’s next • Part of upcoming ORCA 5.0 • More storage slivering for • Cisco racks NetApp • New IBM racks and Dell Racks with server-based storage • With I/O isolation • Add support to attach to bare-metal nodes • Virtual wire support for OVS • To improve network performance isolation for VMs • Hybrid network support for switches • Ethernet VLANs for those who care about performance • OpenFlow for those who actually care about it • Fully dynamic slices • Add/remove links/nodes at runtime • Inter-slice stitching

5. Overview • Details on image snapshot utility • Image portability with InstaGENI • Turning tools into libraries • ExoGENI Storage slivering • ExoGENI Stitching

6. ExoGENI Image Background • Images defined by XML metadata file • Image (AMI) - mandatory • Kernel (AKI) - optional • Ramdisk (ARI) - optional • Hosted on HTTP server • Permanent server at your institution • Soon will work with DropBox and Google Drive • Slice requests require • URL to metadata file – goes into the ‘name’ in RSpec • Hash of metadata file – goes into the ‘vesrion’ in RSpec

7. ExoGENI Image Background: Metadata <images> <image> <type>ZFILESYSTEM</type> <signature>9efee8fa9c6705455c87a42477bd4ebd04d39c81</signature> <url>http://geni-images.renci.org/images/standard/centos/centos6.3-v1.0.10.tgz</url> </image> <image> <type>KERNEL</type> <signature>5e76d034e27cb861275fc3d1bd9d97c3905ef5a2</signature> <url>http://geni-images.renci.org/images/standard/centos/kernels/vmlinuz-2.6.32-431.5.1.el6.x86_64</url> </image> <image>` <type>RAMDISK</type> <signature>c999527cc0d8739a3d3f38d48f03fa6c73151cca</signature> <url>http://geni-images.renci.org/images/standard/centos/kernels/initramfs-2.6.32-431.5.1.el6.x86_64.img</url> </image> </images>

8. ExoGENI Image Snapshot • Snapshot of running ExoGENI VM • Creates: image, kernel, ramdisk, metadata • High level steps • Create/modify a VM – update OS, install needed packages • Run the script, make sure you have enough disk space • Use ExoGENI storage slivering to help here (next topic) • Copy the new image files to an http server • Insert metadata URL and hash into a request More details on the ExoBlog: http://www.exogeni.net/2014/06/creating-a-custom-image-from-an-existing-virtual-machine/

9. Image portability with IG • Porting from EG to IG is a complicated and error-prone process • Porting from IG to EG has not yet been achieved • Ultimately image snapshot utility should work for IG nodes to capture image that works on EG

10. Log into your VM • Get even more coffee • Submit RSpec • Your RSpec “disk_image” is the “Metadata URL” • Create a new Rspec • Boot a VM • Click: <Your Image Name> • Click: Experimentation -> List ImageIDs • Toggle “Global” to “Yes” • Get Coffee / Wait for email from Emulab declaring victory • Click “Submit” • Set types to only “pcvm” • Set “EC2 User@Node”:  i.e. root@152.54.14.44 • Set “Filename”:  should be preset with project dir and “Description Name”.  Example: “/proj/GENISciShakedown/images/Centos6.3.ndz” • Set “OS Version”: Version of the image??? I’m not sure what this is for. • Set “Operating System”:   Linux • Set “Description”: choose a description • Set "Descriptor Name”:  choose a name • Click: “Import an Amazon EC2 Instance Image” • Click: Experimentation -> List ImageIDs • Log into www.emulab.net • Perform ec2 import • cat key to >> .ssh/authorized_keys” • Copy emulab keys into ExoGENI instance • Make user the “EC2 user@node" is in the shudders file (i.e.  pruth@node ) • yum install ruby zip unzip sudo e2fsprogs rsync • setup for ec2 import  • edit /usr/local/etc/emulab/rc/rc.mounts and change this line: • Create a file called /etc/emulab/genvmtype and stick in one line that says “xen” • Allow sudo to run without a tty.  Add the following to the sudoers file:    "Defaults    !requiretty”  • Add  /users • Fix resov.conf and hostname • Install client side tools • Install pubsub • yum install perldhclientopensshmysql bison byacc flex boost gitmysql-server mysql-develmysql-embedded-develmysql-proxy-devellibtoolperl-Time-HiRes yum install python-develhttpd man httpd-devel dos2unix swig swig-devel boost-develpcap-devellibpcap-develmysql-devel db4-devel db4 libdb-develcompat-dbtcsh • install Emulabtools • Modify the VM as you want. • cleanup anything the post boot script did • cleanup hostname • cleanup sudoers • remove users if desired • remove public keys (from at least root) • chkconfigneucaoff • /etc/init.d/neucastop • Clean up VM • Boot a Centos VM on ExoGENI.

11. Turning tools into libraries • ExoGENI has a number of tools, that are now being converted into libraries for developing more tools • Existing and newly-developed libraries • Written in Java • orca.ndl– NDL parsing and generation • Used by ORCA, Flukes, NDL converter • orca.ndl-conversion.core – Conversions between NDL and other formats (GENI RSpec, PG Rspec, graph representations) • ndlLib – easy application-level slice management abstractions– in progress • Integration with pubsub for status notifications • Slice visualization, graph layouts • Available from snapshot Maven repository • http://ci-dev.renci.org/nexus/content/repositories/geni-orca-snapshot/

12. Registries • Built on CouchDB • REST APIs • Registries for • Images • Stitchports • Actors • …

13. Storage Slivering • Overview of architecture and implementation

14. Storage Architecture

15. Slices with storage

16. Lun Creation Time Estimates

17. ExoGENI stitching • Unlike GENI stitching, ExoGENI stitching logic is • Integrated into ORCA `controller’ that speaks GENI AM API • Makes it easy to place requests for complex inter-domain topologies • Integrated with ORCA resource brokering/ticketing system • Provides reliable single-shot provisioning that (except when OSCARS on AL2S or ESnet decides to fall over) • We considered pulling it out of the controller and making a toolkit • Still in consideration, no immediate plans though

18. Reasons to use ExoGENI native stitching • ExoGENI stitching • Any XO rack to any other XO rack transparently to the user • AL2S, ION, ESnet, BEN • Works with various corner cases • Internal label translations by transit providers • E.g. NICTA • Inability of AL2S to connect racks that share the same AL2S port • E.g. FIU to UFL • Multipoint VLAN support – create large Layer 2 domains • Storage stitching support – create large slivers of iSCSI storage • Stitch ports – connect slices to the outside world

20. Ways to use it • GENI Tools (omni, Flack, GENI Portal): • Works with other rack types • Limited topology embedding support • ExoGENI native tools: • Expose full range of capabilities • Multi-point • Storage slivering and stitching • Slice modifications (elastic slices) • Use regular GENI credentials • Work only with ExoGENI racks • More information: • http://tinyurl.com/l5qsbwy • http://www.exogeni.net

21. Thank you! • http://www.exogeni.net • http://wiki.exogeni.net