University of Massachusetts Amherst November 2 nd , 2011

1. GENI DiCloud University of Massachusetts Amherst November 2nd, 2011

2. Project Summary • DiCloud enables GENI experiments to incorporate cloud computing resources • Focuses on Amazon web services • EC2 (VMs), EBS (block storage), S3 (object storage) • How are cloud resources different from other GENI resources? • They cost money! • Requires active Amazon account and credit card • Require authentication with Amazon

3. Project Summary • DiCloud tracks per GENI-user Amazon costs • Amazon charges can be complicated • Pay per VM time used • Pay for data transfer in and out • Pay for IOs to block devices • No current facilities for tracking per user costs • Enables controlled access to cloud resources • Stand-alone: setup your own DiCloud instance to track your own costs • GENI CF-mode: place “underneath” GENI CF, where AM pays the bills, but gives GENI users access

4. Elastic Compute Cloud (EC2) • Rent VMs from Amazon “on-demand” • http://wwww.amazon.com/ec2

5. Elastic Compute Cloud (EC2) • Rent VMs from Amazon “on-demand” • Pay per hour based on VM performance • Pay for data transfer in/out

6. Elastic Block Store (EBS) • Rent block storage from Amazon • Pay per GB-month (currently $0.10) • Also pay per 1 million IO requests (currently $0.10)

7. Simple Storage Service (S3) • Rent object storage from Amazon • Pay per GB-month • Pay for data transfer in/out

8. Web Portal

9. Web Portal

10. Web Portal

11. Tutorial Step 5: Web Portal

12. Tutorial Step 5: Web Portal • Now logout and log back in as user “geni” • Then click on “Request EC2 resources” • Use ami-013d6c44 as the AMI id

13. Tutorial Step 5: Web Portal • Now go back to your user page • Click on status next to the VM you just created

14. Tutorial Step 5: Web Portal • Now lets create an S3 bucket • Click on “Request new S3 buckets”

15. Tutorial Step 5: Web Portal • Once back at the user page, put something in your bucket • Select the s3_test file in ~/Tutorials/DiCloud/s3_test

16. Tutorial Step 5: Web Portal • Now get that same thing back from your bucket

17. Tutorial Step 5: Web Portal • Now lets create an EBS volume and attach it to our VM • Click request new EBS volumes • Make it 1GB, in us-west-1, in us-west-1<availability num>

18. Tutorial Step 5: Web Portal • Attach the volume to our VM • Click on attach, enter instance id, and sdg for device

19. Tutorial Step 5: Web Portal • If we ssh into our VM we can see that the volume is now attached

20. Demo Data Flow • Dynamic end-to-end Nowcasting • Mapping Nowcast Workflows onto GENI archived netcdf data aggregated multi-radar data Nowcast images for display “raw” live data Radar Nodes Archival Storage Upstream LDM feed Nowcast Processing Post to Web

21. Generate “raw” live data ViSE/CASA radar nodes http://stb.ece.uprm.edu/current.jsp Ingest mulit-radar data feeds Merge and grid multi-radar data Generate 1min, 5min, and 10min Nowcasts Send results over NLR to Umass Repeat Use proxy to track usage-based spending on Amazon and enforce quotas and limits http://geni.cs.umass.edu/vise/dicloud.php ViSE views steerable radars as shared, virtualized resources http://geni.cs.umass.edu/vise “raw” live data Nowcast images for display DiCloud Archival Service (S3) LDM Data Feed (EC2) Multi-radar NetCDF Data Nowcast Processing

22. Demo Overview • Dynamic end-to-end Nowcasting on GENI • Slice of sensing, networking, computing, and storage archived netcdf data aggregated multi-radar data Nowcast images for display “raw” live data Radar Nodes Archival Storage Upstream LDM feed Nowcast Processing Post to Web Generate Nowcasts Archive radar data Amazon S3 Archived data available to downstream nodes Generate “raw” data ViSE/CASA radars