1 / 0

Creating an AMI at Amazon’s EC2

Creating an AMI at Amazon’s EC2. Joe Steele jrsteele@unomaha.edu. Cloud Computing. What if I don’t have my own cluster? Cloud computing refers to a cluster that invites users to send jobs. ( SaaS –Software as a Service)

yaakov
Download Presentation

Creating an AMI at Amazon’s EC2

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Creating an AMIat Amazon’s EC2

    Joe Steele jrsteele@unomaha.edu
  2. Cloud Computing What if I don’t have my own cluster? Cloud computing refers to a cluster that invites users to send jobs. (SaaS –Software as a Service) Computation, software, data access, and storage services that do not require user knowledge of the location or configuration of the system. Term comes from the cloud drawing used in the past to represent the telephone network, later represents the internet.
  3. Cloud Computing
  4. Amazon E2 aws.amazon.com Computing cluster – create an account and provide a credit card. Let Amazon take care of the hardware.
  5. Cloud BioLinux JCVI (J. Craig Venter Institute) created cloud version of NERC BioLinux VM. An Ubuntu machine with over 100 NEBC software packages. Image stored at EC2, is available to be copied at no charge, by EC2 users.
  6. http://aws.amazon.com
  7. Create a new account
  8. Enter your information
  9. Sign up for an EC2 account
  10. Click on “Sign up for Amazon EC2”
  11. EC2 Account Signing up for EC2 automatically signs you up for Amazon Simple Storage Service, and Amazon Virtual Private Cloud. Requires credit card information. No charges until you start using the services. Amazon will email with Access Identifiers, and instructions for your first log in.
  12. Click on “AWS Management Console”
  13. Click the EC2 Tab
  14. Launch an Instance
  15. I recommend biolinux
  16. Click “Select”
  17. Pricing Amazon has a variety of VM sizes available – pricing is at: http://aws.amazon.com/ec2/pricing/ You are charged for CPU usage, for data storage, and for data transferred to or from Amazon. Charges continue until a VM is “Terminated”. You can set up a small test VM for free – select “Micro” for the size.
  18. Kernel defaults are fine
  19. Key files For biolinux, you don’t need to add “User Data”. For VipDac (on the Back-Up slides) the Amazon key IS needed. When you create a Key Pair (next slide), save the *.pem file to a safe place.
  20. Create a Key Pair
  21. PEM file Save the *.pem file to a safe place. Linux: the *.pem file has to be protected: >chmod 400 <key_file>.pem Windows: Download PuTTYgen, and use it to change the *.pem file to a *.ppk file.
  22. Create security group
  23. Launch
  24. Machine info
  25. “Terminate” to end charges
  26. First connection to Biolinux A window opens, telling you how to connect to your new VM, eg,: “ssh -i key_pair_name.pem root@ec2-76-202-01-919.compute-1.amazonaws.com” However, for biolinux, or any unbuntu VM, do: ssh –i key_pair_name.pem ubuntu@ec2-76-202-01-919.compute-1.amazonaws.com from a Linux machine, make sure the key_pair_name.pem is set to read only for the user only.
  27. First Connection From Windows: Use PuTTYgen to change key_pair_name.pem to key_pair_name.ppk. Use PuTTY to connect, and put key_pair_name.ppk under “SSH” -> “Auth”. When connected to biolinux, you will be asked to set a password for ubuntu. If not, you need to start NX client: >./configure_freenx.sh
  28. NX Use NX for the graphical display (built in to biolinux already). Open source, can be found at http://www.nomachine.com/ Must ssh into VM FIRST, using the key pair. You can stick with the ubuntu user, or add new users: >adduser <username> >groups >usermod -G <grp1>,<grp2>,ssh <username>
  29. Start NX
  30. “Configure”
  31. Configure NX Note the use of “GNOME” and “LAN”. The Host should be “ec2-72-…amazonaws.com”. Also, the “Key…” works with the default key.
  32. BioLinux over NX
  33. BioLinux over NX
  34. BioLinux over NX
  35. BioLinux over NX
  36. Data Stored at Amazon There are large datasets stored at Amazon, available for use – free of charge (mostly). You are charged for any data you copy. http://aws.amazon.com/datasets to search through them. Datasets that might be of interest in bioinformatics are listed at: http://biodoc.ist.unomaha.edu/wiki/Inventory_of_amazon_data
  37. http://aws.amazon.com/datasets
  38. Datasets Human DNA sequences: 1000 Genomes Project (7,300 GB) Ensembl Annotated Human Genome - FASTA (115 GB) Ensembl Annotated Human Genome - MySQL (200 GB) GenBank (200 GB) Human Liver Cohort (Sage Bionetworks) (0.6 GB) Illumina - Jay Flatley's Human Genome Data Set. (350 GB) YRI Trio Data - complete genome sequence for three individuals (700 GB) Other (might include some human data): Ensembl - FASTA DB (100 GB) Influenza Virus (including Swine Flu) - from NCBI (1 GB) UniGene - from NCBI (10 GB) PubChem Library - from NCBI (230 GB)
  39. Public Snapshots
  40. Select “Volumes”
  41. Create a Volume
  42. Instance Information
  43. Attach it to your Instance
  44. Mount the Volume From your VM: >sudomkfs –t ext3 /dev/sdf >sudomkdir /mnt/datasets >sudo mount –t ext3 /dev/sdf /mnt/datasets 200GB of genbank data are now in /mnt/datasets
  45. Test Usage for the First Year
  46. CPU charges
  47. Data Transfer charges
  48. Data storage charges
  49. Back Up slides
  50. VipDac Tutorial
  51. VipDac Tutorial
  52. VipDac Tutorial
  53. VipDac Tutorial
  54. VipDac Tutorial
  55. VipDac Tutorial
  56. VipDac Tutorial
  57. VipDac Tutorial
  58. VipDac Tutorial
  59. VipDac Tutorial
  60. VipDac Tutorial
  61. VipDac Tutorial
  62. VipDac Tutorial
  63. VipDac Tutorial
  64. VipDac Tutorial
  65. VipDac Tutorial
  66. VipDac Tutorial
More Related