1 / 25

Management and contextualization of scientific virtual appliances

Germán Moltó Associate Professor at the Universidad Politécnica de Valencia ( Spain ) gmolto@dsic.upv.es. Management and contextualization of scientific virtual appliances. For the Cloud!. OUTLINE of the talk. Outline Introduction and Overview of the GRyCAP

amable
Download Presentation

Management and contextualization of scientific virtual appliances

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Germán Moltó AssociateProfessor at the Universidad Politécnica de Valencia (Spain) gmolto@dsic.upv.es Management and contextualization of scientific virtual appliances Forthe Cloud!

  2. OUTLINE of thetalk • Outline • Introduction and Overview of theGRyCAP • Scientific Cloud Computing • Contextualization: Scientific Virtual Appliances • Virtual Appliances Repositories and Catalogs • ScientificApplications • Conclusions and Future Challenges

  3. Thegrycap in a slide Grid and High Performance Computing Group • Group of the Area of Information Technologies and Computational Science Created on 1986 by Vicente Hernández and Composed by 28 Researchers (http://www.grycap.upv.es). • Adoption of Parallel and Distributed Computing Technologies for Improving the Performance of Scientific Applications. • Evolution to Grid and Cloud Technologies • E-Science: Support for Science Research through the Collaborative Use of Distributed Resources. e-Government EngineeringSimulation Proteomics BiomedicalComputation MedicalImaging Photonics e-Science e-Infrastructure Middleware Grid Technologies Cloud Technologies Parallel Computing Distributed Computing NumericalComputation

  4. Scientificapplications • ScientificApplicationstypicallyrequire: • Largecomputationalpower. • Itsrequirementsmightexceedtheresources of a single machine • Processinglargeamount of data. • Combination of SeveralTechniques • High Performance Computing • Using multiple processors to solve a problem. • Grid Computing • Enablethecollaborativeusage of resourcesfrommultipleorganizationstofacetheefficientexecution of large-dimensionproblems.

  5. Gridcomputing Pros and Cons • Grid Computing has beensuccessfullyemployed in manyscientificareas, althoughsamecaveatsexist.

  6. Cloud computing ForScientific Computing • Cloud Computing advantagesoverGrid Computing: • Itallowstheresourceconsumersto configure theirspecificExecutionEnvironments. • A controlledenviromentiscriticaltoguaranteethesuccessfulexecution of scientificapplications. • Dynamicscaling of infrastructuresforresourceproviders. • Virtual Machines can bedeployedusingworkload-awarestrategies. • Fast and easyaccessto a largeamount of resources. • No needforscientificcomission’sapproval, just use yourCreditCard. • Reduced energy consumption (Green Computing) • Machines are onlyprovisionedwhenthey are requested. • Virtualizationleverages server consolidation.

  7. Thepoint of view of thescientist/Engineer • Scientists and Engineersshouldnotbeconcernedwithimplementationdetails of technology. I don’tcareabouttechnology, I justwant my appstorunthefastestpossible X.509 Proxies CAs Grid VOs SE gLite Globus LFN SURL … Hypervisor Configuration Cloud • Focusonabstractingthedetails of applicationportingtothe Cloud. Deployment Monitoring APIs …

  8. SCIENTIFIC Cloud computing • Scientific Cloud Computing focusesontheexecution of scientificapplicationson a (typically) IaaScloud. • It requires the management and provision of Scientific Virtual Appliances from a Virtual Machine Manager. Google Docs Office Live … MS Azure … Google AppEngine Eucalyptus OpenNebula … Amazon EC2 Source: www.saasblogs.com

  9. Virtual machine managers • VMMsprovidethebasictoolstobuildanIaaS Cloud • Differenttools in thecloud arena for VM management.

  10. Currentlimitation of cloudcomputingtools • Virtual Machine Managers focus on supporting the life cycle of VMs. • Scientific Cloud Computing also requires: • (semi-)Automatedcontextualization of Virtual Machines forscientificapplications Scientific Virtual Appliances (SVA). • ReusingSVAsfromoneexperimenttoanother, alsotoenhanceSVAssharingamongdifferentresearchers. • Wefocuson: • Applicationcontextualization (From a VM to a SVA). • Repositories and catalogs of SVAs.

  11. Virtual appliances • A Virtual Appliance (VA) consists of a Virtual Machine speciallyconfiguredforanApplication. Application App Data Application ComputationalLibraries ApplicationRequirements Middlewares OperatingSystem PersistenceLayer Services Virtual Appliance OperatingSystem Scientific Virtual Appliance

  12. Contextualizingscientific virtual appliances • FromVMstoproductionSVAs … • Contextualizationmeanscreatingtheappropriate SW/HW environmentforthesuccessfulexecution of anapplication. • Virtual Machines needtobecontextualized (IP, DNS, etc.). • SupporttypicallyprovidedbytheVMMs. • Applications need to be contextualized. • Deployed, configured, built, executed. Virtual Machine Scientific Virtual Appliance Contextualization Plain OS ScientificApplicationrunning

  13. Software configurationtools • Many machine configurationtools. • Focusonautomatingthe: • Machine configuration • DNS, Config files, etc. • Installation of commonlyusedpackages: • Web Servers, Application Servers, etc. • Client-Servicetools.

  14. Deployingscientificapplications • Manyscientificapplicationsfollowthesamepatterns …

  15. Automatingapplicationcontextualization (I) ForScientificApplications • We are workingon software for (scientific) applicationcontextualization. • Goal: Software inoculation and configurationintothe VM withminimumuserintervention. • Automation vs SSH-based Manual Installation InstallPackages App AppDescription (XML) CNTXTLZR Configure Contextualization Plan Build Software Dependences Deploy / Run

  16. Automatingapplicationcontextualization (II) • Developed a proof-of-concept toolforscientificapplicationcontextualization. • Python-basedtoensuregoodportability. • Plugin-basedto describe thedeployment of software packages. • XML language • Thetool, application and requirements are stagedintothe VM at boot time viathe VMM capabilities (OpenNebula). • VM isturnedinto a SVA byapplicationcontextualization at boot time.

  17. Toward virtual machine cataloguing • Thereexist VM catalogsoutthere: • VMWareMarketplace • ScienceCloudsMarketplace • BUT… • For human consumption, no APIs, unstructured metadata, etc. • The VM Catalogincludes: • VM Metadata (OS, Software Environment, etc.) • OVF (Open VirtualizationFormat), XML-based. • Links to VM repositories (either local orremote). • MatchmakingalgorithmstoretrievethemostappropriateVMsaccordingtouserrequirements (hard vs soft).

  18. Management of scientific virtual appliances • Theuser/adminprovides a description of the VM in OVF format. • FTP server instances are createdondemandwithdynamic and temporarycredentialsfor VM upload. • Client-SideLibrariestoeasetheinteractionwiththecatalog.

  19. Virtual Machine Repository • The VM Repositoryincludes: • Storage of VMs • Data Access Mechanisms • HTTP and FTP. • GridFTPwouldprovideenhanced X.509-based security. • Virtual Machines considered: • GoldenVMs • Example: JeOS-based, Lowfootprint (Ubuntu JeOS , 380 Mbytes HD) • Pre-ContextualizedVMs • Reusethework done. No needto re-deploy software forever. • Example: A Globus Tookit 4-based VM that can be reused for the deployment of different Grid Services.

  20. Thebigpicture Catalogs, Repositories and Contextualization

  21. Remotecontrollinganapplication • Howto control theApp and accessthe output files insidethe VA? • Werelyonthe Opal 2 Toolkit • Opal 2 Toolkitprovides a WS WrapperforApplications • Operationsforstarting, monitoring and terminatingtheapplication. • Supportfor local, MPI and Globus-basedexecutions. • Output files accessible through Tomcat (computational steering). Generic Opal 2 WSDL App App App Opal 2 Toolkit Application Server (Apache Tomcat) Virtual Appliance Opal 2 Toolkitdeveloped @ NBCR

  22. Web serviceswrappertocomputationalapplications • WS-WrappedApplications can nowbeorchestratedbythe Cloud Enactor (acting as a Task Manager). • Applications can nowbecontrolled (started and monitored) insidetheScientific Virtual Appliance. • Manyinstances of theapplication can beconcurrentlymanaged. Virtual Appliance Cloud Enactor (Task Manager) WS Wrapper (OPAL) Control, Monitor, Access files App API Client-Side OPAL API Hypervisor

  23. Scientificapplications • Simulation of CardiacElectricalActivity • ActionPotentialPropagationonCardiacTissues. • Simulation of Guided Light in Photonic Crystal Fibers • Optimization of SupercontinuumSpectrumusingGeneticAlgorithms. • Optimization of ProteinDesignwith Target Properties • ComputationallyIntensive, SimulatedAnnealing, Monte Carlo.

  24. conclusions • Scientific Cloud Computing requirestoolstoabstracttheinteractionwith Cloud infrastructures. • From Applications to Scientific Virtual Appliances • At theGRyCAPwe are workingon: • ApplicationContextualization • Virtual Appliances Management • The Cloud looks likeanalternativeapproachfortheexecution of scientificapplications. • Definition of SpecificExecutionEnvironments

  25. Challenges in thenearfuture • InteroperabilityamongClouds • Avoidvendorlock-in • Software GatewaysamongInfrastructureProviders • LargeEcosystem of Virtual Machine Managers • They share somefunctionalities and goals • Developers like to code for the winning horse • CommonAPIsfor Cloud Computing • Apache LibCloud, Deltacloud, jclouds, Dasein Cloud API, Fog, etc. • Clouds and Grids must provide Computational Support to Scientific Applications

More Related