GEC17 Using ExoGENI - PowerPoint PPT Presentation

gec17 using exogeni n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
GEC17 Using ExoGENI PowerPoint Presentation
Download Presentation
GEC17 Using ExoGENI

play fullscreen
1 / 26
GEC17 Using ExoGENI
192 Views
Download Presentation
betha
Download Presentation

GEC17 Using ExoGENI

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. GEC17 Using ExoGENI IlyaBaldinibaldin@renci.org

  2. ExoGENI Overview

  3. Testbed • 14 GPO-funded racks built by IBM • Partnership between RENCI, Duke and IBM • IBM x3650 M4 servers (X-series 2U) • 1x146GB 10K SAS hard drive +1x500GB secondary drive • 48G RAM 1333Mhz • Dual-socket 8-core CPU • Dual 1Gbps adapter (management network) • 10G dual-port Chelseo adapter (dataplane) • BNT 8264 10G/40G OpenFlow switch • DS3512 6TB sliverable storage • iSCSI interface for head node image storage as well as experimenter slivering • Cisco(UCS-B) and Dell configuration also exist • Each rack is a small networked cloud • OpenStack-based with NEuca extensions • xCAT for baremetal node provisioning • http://wiki.exogeni.net

  4. ExoGENI • ExoGENI is a collection of off-the shelf institutional clouds • With a GENI federation on top • xCAT – IBM product • OpenStack- RedHat product • Operators decide how much capacity to delegate to GENI and how much to retain for yourself • Familiar industry-standard interfaces (EC2) • GENI Interface • Mostly does what GENI experimenters expect

  5. ExoGENI at a glance

  6. Rack Software Stack

  7. Deployment structure

  8. An ExoGENI cloud “rack site”

  9. ExoGENI unique features • ExoGENI racks are separate aggregates but also act as a single aggregate • Transparent stitching of resources from multiple racks • ExoGENI is designed to bridge distributed experimentation, computational sciences and Big Data • Already running HPC workflows linked to OSG and national supercomputers • Newly introduced support for storage slivering • Strong performance isolation is one of key goals

  10. ExoGENI experimenter tools • GENI tools: Flack, GENI Portal, omni • Give access to common GENI capabilities • Also mostly compatible with • ExoGENI native stitching • ExoGENI automated resource binding • ExoGENI-specific tools: Flukes • Accepts GENI credentials • Access to ExoGENI-specific features • Elastic Cluster slices • Storage provisioning • Stitching to campus infrastructure

  11. ExoGENI activities Presentation title goes here

  12. Experimentation • Compute nodes • Up to 100 VMs in each full rack • A few (2) bare-metal nodes • BYOI (Bring Your Own Image) • True Layer 2 slice topologies can be created • Within individual racks • Between racks • With automatic and user-specified resource binding and slice topology embedding • Stitching across I2, ESnet, NLR, regional providers. Dynamic wherever possible • OpenFlow experimentation • Within racks • Between racks • Include OpenFlow overlays in NLR (and I2) • On-ramp to campus OpenFlow network (if available) • Experimenters are allowed and encouraged to use their own virtual appliance images • Since Dec 2012 • 2500+ slices

  13. Topology embedding and stitching Workflows, services, etc. Multi-homed cloud hosts with network control Computed embedding Virtual colo campus net to circuit fabric Virtual network exchange

  14. Slice half-way around the world • ExoGENI rack in Sydney, Australia • Multiple VLAN tags on a pinned path from Sydney to LA • Internet2/OSCARS ORCA-provisioned dynamic circuit • LA, Chicago • NSI statically pinned segment with multiple VLAN tags • Chicago, NY, Amsterdam • Planning to add dynamic NSI interface • ExoGENI rack in Amsterdam • ~14000 miles • 120ms delay

  15. ExoGENI slice isolation • Strong isolation is the goal • Compute instances are KVM based and get a dedicated number of cores • VLANs are the basis of connectivity • VLANs can be best effort or bandwidth-provisioned (within and between racks)

  16. Scientific Workflows • Workflow Management Systems • Pegasus, Custom scripts, etc. • Lack of tools to integrate with dynamic infrastructures • Orchestrate the infrastructure in response to application • Integrate data movement with workflows for optimized performance • Manage application in response to infrastructure • Scenarios • Computational with varying demands • Data-driven with large static data-set(s) • Data-driven with large amount of input/output data

  17. Scientific Workflows

  18. Dynamic Workflow Steps (Computational)

  19. Workflow slices • 462,969 condor jobs since using the on-ramp to engage-submit3 (OSG)

  20. Dynamic Workflow Steps (On-Ramp)

  21. Dynamic Workflow Steps (Data-driven)

  22. Hardware-in-the loop slices • Hardware-in-the-Loop Facility Using RTDS & PMUs (FREEDM Center, NCSU)

  23. New GENI-WAMS Testbed Latency & Processing Delays Packet Loss Network Jitters Cyber-security Man-in-middle attacks AranyaChakrabortty, Aaron Hurz (NCSU) YufengXin (RENCI/UNC)

  24. Thank you! • http://www.exogeni.net • ExoBlog http://www.exogeni.net/category/exoblog/ • http://wiki.exogeni.net