1 / 69

Cloud Management Software for Multi-Clouds

Cloud Management Software for Multi-Clouds. Dana Petcu West University of Timisoara and Institute e-Austria Timisoara http://web.info.uvt.ro/~petcu. Motivation. Consumers of Cloud resources need portability of Cloud applications no vendor lock-in.

artan
Download Presentation

Cloud Management Software for Multi-Clouds

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cloud Management Software for Multi-Clouds Dana Petcu West University of Timisoara and Institute e-Austria Timisoara http://web.info.uvt.ro/~petcu

  2. Motivation • Consumers of Cloud resources need • portability of Cloud applications • no vendor lock-in. • Third parties: mediators between the Consumers & Providers • build ad-hoc grouping of Cloud resources to satisfy application requirements to use various resources from Public, Private or Hybrid Clouds • requirements are related to the migration from one Cloud to another, Cloud bursting, or consumption of particular services. • Focus of this talk: • current solutions of overcoming this status

  3. Announced Presentation • The taxonomy of multiple Clouds • Federations, Multi-Clouds, InterClouds and their sub-categories • Overview • the available Cloud management software for Multi-Clouds • Gaps between requirements and offers • Case study • an open-source and deployable PaaS for Multi-Clouds, with live demos

  4. Topics of interest vs. Gartner Report

  5. Agenda – more concrete A step back The future of the Clouds? Projects funded by the European Commission, dealing with Clouds Why Multiple Clouds? Taxonomy of Multiple Clouds Interoperability and portability Model-driven engineering for the Clouds Case study: a run-time environment for Multi-Clouds

  6. A Step Back From Where?

  7. University and Faculty • West University of Timisoara (www.uvt.ro/en) • More than 20 000 students • 11 faculties • Faculty of Mathematics and Computer Science (www.math.uvt.ro) • More than 1000 students (undergraduate, master, PhD) • Two departments: Maths and CS

  8. Department & Research Center • Computer Science Department (web.info.uvt.ro) • Around 700 students (undergraduate, master, PhD) • Studies in Romanian and English • Foreign students coming in Erasmus programme • 35 teachers • Master (English): Artificial Intelligence & Distributed Computing (www.math.uvt.ro/invatamant/cicluri/masterat/informatica/aidc) • Research Center in Computer Science (research.info.uvt.ro) • Parallel & Distributed Computing, AI & Nature Inspired Computing • Runs around 5 national & international R&D projects per year • Manage the biggest supercomputing center of Romania

  9. HPC Center http://hpc.uvt.ro 400 cores Cluster 4000 cores BlueGene/P 3000 cores GPU cluster

  10. Research spin-off, IeAT • Institute e-Austria Timisoara (www.ieat.ro) • 10 years old private research institute in Computer Science • Non-profit association between 3 public institutions (2 universities from Romania and one from Austria) • More than 40 employees • Funded only on projects • R&D project obtained by national/international competitions • Technological transfer type of contracts with industry • PhD and master students working in R&D projects to complete their theses • Support the R&D activities of the universities involved

  11. Parallel & Distributed computing Group • … • 2000-2009 • Grid computing – tools and applications in symbolic computing, Earth Observation • Services – orchestrations, semantics • Parallel computing in image processing, evolutionary computing, formal verification, symbolic computing • 2010-2013 • Cloud computing • Scalability in parallel computing, scheduling

  12. Projects/2013 @ UVT & IeAT • Cloud • EC-FP7 MODAClouds • EC-FP7 mOSAIC • EC-FP7 SPECS • EC-CIP SEED • RO-PNII AMICAS • Grid • EC-FP7 EGI Inspire • Parallel • EC-FP7 HOST • EC-FP7 HP-SEE • Others • EC-FP7 SPaCioS • EC-FP7 SCAPE www.modaclouds.eu www.mosaic-cloud.eu www.specs-project.eu www.seed-project.eu http://amicas.hpc.uvt.ro www.egi.eu http://host.hpc.uvt.ro www.hp-see.eu www.spacios.eu www.scape-project.eu 2012-2015 2010-2013 Sci. lead 2013-2016 2012-2014 2012-2014 2010-2014 2012-2014 Lead 2010-2013 2010-2013 2011-2014

  13. The future of Clouds? Generalities

  14. CC-Definition Source: http://cordis.europa.eu/fp7/ict/ssai/docs/future-cc-2may-finalreport-experts.pdf

  15. Provider perspective Source: http://cordis.europa.eu/fp7/ict/ssai/docs/future-cc-2may-finalreport-experts.pdf Clouds are dynamic (resource) environment that guarantee availability, reliability and related quality aspects through automated, elastic management of the hosted services The automated management aims at optimising the overall resource utilisation whilst maintaining the quality constraints.

  16. User perspective Source: http://cordis.europa.eu/fp7/ict/ssai/docs/future-cc-2may-finalreport-experts.pdf Clouds are environments which provide resources and services to the user in a highly available and quality-assured fashion, thereby keeping the total cost for usage & administration minimal and adjusted to the actual level of consumption. The resources and services should be accessible for theoretically unlimited no.customers from different locations and with different devices with minimal effort and minimal impact on quality. The environment should adhere to security and privacy regulations of the end-user, in so far as they can be met by the internet of services.

  17. Expectations in terms of use cases Source: http://cordis.europa.eu/fp7/ict/ssai/docs/cloud-expert-group/roadmap-dec2012-vfinal.pdf

  18. Main Topics to Address Data Management Communication & Network Resource Description & Usage Resource Management Programmability and Usability Federation, Interoperability, Portability Multiple Tenants Political & Legislatory Security Business & Cost Models

  19. Projects funded by European Commission dealing with the Clouds Catching the gaps between technological advances in different continents?

  20. European Cloud initiatives in 2011 Market Open PaaS Storage Brokering PaaS Web appls Security Testbeds Cloud networking Migration Programming D.Petcu, J.L. Vazquez-Poletti (eds) European Research Activities in Cloud Computing, CSP, UK, Jan 2012, http://www.c-s-p.org/flyers/978-1-4438-3507-7-sample.pdf

  21. Updates on SSAI projects in 2012 Lifecycle Automatic Cloud Broker Data-aaS Big Data Scaling Migration Personal Clouds IoT Energy-aware Mobile Cloud Model-driven multi-Cloud Testing-aaS Open Cloud Open source in Cloud Source: http://ec.europa.eu/digital-agenda/events/cf/ios12/document.cfm?doc_id=23468

  22. WhyMultiple Clouds?

  23. NIST scenarios: Multiple Clouds • Clouds can beused • serially, when moved from one Cloud to another, or • simultaneous, when using services from differentClouds. • Simple scenarios: • [serial] migration from a Private Cloud toa Public Cloud • [simultaneous] HybridCloud, when some services are lying on the Private Cloud,while other services are lying on a Public Cloud

  24. Top 10 Reasons for Multiple Clouds • deal with the peaks in service &resource requestsusing external ones, on demand basis; • optimize costs or improve quality of services; • react to changes of the offertsofthe providers; • follow the constraints, like new locations or laws; • replicatethe applications or services consuming resources or services from differentCloud providers toensure their high availability; • avoid the dependence on only one external provider; • ensure backup-ups to deal with disasters or scheduledinactivity; • act as intermediary; • enhance own Cloud resource and serviceoffers, basedon agreements with other providers; • consume differentservices for their particularitiesnotprovided elsewhere.

  25. Reasons Application developer’s reasons Con-straints User’s reasons Changes Non-depen-dence Costs & QoS Availa-bility Backup Particu-larities Inter-mediate Peaks Enhance offer Provider’s reasons Cloud broker’s reasons

  26. Taxonomy ofMultiple Clouds

  27. Terminology • Multi-Cloud, • CloudFederation, • Inter-Cloud, • Hybrid Cloud, • Cloud-of-Clouds, • SkyComputing, • Aggregated Clouds, • Multi-tier Clouds, • Cross-Cloud, • Cloud Blueprint, • Cloud Merge, • Fog Computing, • Hierarchical Clouds, • Distributed Clouds ...

  28. Delivery models for Multiple Clouds • Federated Clouds • assumes • a formal agreement between the Cloud providers • service providers • are sub-contract capacity from other service providers • offer spare capacity to the federated group of providers. • the consumer of the service • is not aware of the fact that the Cloud provider he or she pays is using the services of another Cloud provider • Multi-Cloud • assumes that • there is no priori agreement between the Cloud providers • a third party (even the consumer) is responsible for the services • contacts the service providers, • negotiates the terms of service consumption, • monitors the fulfillment of the service level agreements, • triggers the migration of codes, data and networking from one provider to another. Source: http://www.buyya.com/papers/InterCloud-Brokering-Taxonomy.pdf

  29. Scenarios for multiple Clouds Federation of Clouds Main issue: Inter- operability Main issue: Portability Multi Cloud 01011 01011 01011 01011 01011 01011 001 001 01011 01011

  30. InterCloud, Cloud Broker & Blueprint • InterCloud: • A Cloud Federation or a Multi-Cloud that includes at least one Cloud Broker and offers dynamic service provisioning • Cloud Broker • an entity that manages the use, performance and delivery of Cloud services and intermediates the relationships between Cloud providers and Cloud consumers • Cloud Blueprint • an enhanced Cloud delivery model, • a reference architecture which breaks down the rigidity of the Cloud stack and transforms it into modular and easily combinable components that offer Integration-as-a-service functionality

  31. Centralized Federations Classification Distributed Clouds Aggregated Federations Peer-to-Peer Federations Horizontal Federations Sky computing Dynamic Federations Cloud Federations Cross- Clouds Vertical Federations Hierarchical Federations Multi-tier Federations Library-based Multi-Clouds Hosted Multi-Clouds Horizontal Multi-Clouds Service-based Multi-Clouds Deployable Multi-Clouds Multiple Clouds Multi- Clouds Hybrid Clouds Bursted Clouds Hierarchical Multi-Clouds Clouds of Clouds Cloud governance SLA –based Cloud brokers Inter- Clouds Cloud Brokers Cloud Market-places Triggered-actionbrokers Cloud Blueprinting

  32. To solve in Cloud Federation Interoperability framework Integration as a service Match-making with available external services Live virtual machine migration Network overlay for connectivity problems Meta-schedulers Monitoring meta-system Intelligent management systems ...

  33. To solve in Multi-Cloud Portability Resource/service selection mechanism and methodology Uniform APIs Search engines Automated deployment Service aggregator Governance ...

  34. Requirements/ Multi-Cloud Development Deployment Execution Portal/service as entry point Service/ resource meta-allocator Generic deployer Search engine Meta-monitor for applications Cloud agnostic extra services Meta-scheduler Semi-automated deployer Match-making service Meta-monitor for services/ resources Tools Interface for user’s requirements Meta- auto-scaler and load-balancer Virtual network mechanisms Selection service Controller of application/service life-cycle Integration service Debugger and tester Credentials management Recommen-dation system QoS control and warning mechanisms Portability support Particularities preservation Seamless join by new Clouds No constraints on Clouds Use standard protocols Principles Abstract service con-trol interfaces Use standard interfaces Support for top Cloud providers Allow dynamic allocation of resources Small overhead

  35. Barriers • Standards not adopted on large scale • Libraries that are loosing the particularities • No comprehensive methodology to compare services • Complexity of the selection multi-criterial problem • Heterogeneity from low (e.g. VM) to high level (e.g. Programming) • Lack of agreement between providers on the interfaces for certain actions • Portability or relocation are moving targets • Lack of trustfullness in Hybrid Clouds as the simple example of Multi-Clouds

  36. Centralized Federations Tools Distributed Clouds Aggregated Federations Peer-to-Peer Federations Horizontal Federations Sky computing NIMBUS Dynamic Federations Cloud Federations Cross- Clouds Xen-Blanket Vertical Federations Hierarchical Federations Multi-tier Federations Library-based Multi-Clouds Hosted Multi-Clouds Horizontal Multi-Clouds Service-based Multi-Clouds Aoleos Deployable Multi-Clouds Multi- Clouds Hybrid Clouds Bursted Clouds Hierarchical Multi-Clouds Clouds of Clouds Cloud governance SLA –based Cloud brokers Inter- Clouds Cloud Brokers Cloud Market-places Triggered-actionbrokers Cloud Blueprinting

  37. Interoperability and portability

  38. Interoperability in Clouds? API spec Q: How to inter- operate? 01 API spec 01 01 01 01 01 01 01 API spec

  39. Interoperability/Clouds- history • Migration – targets VMs • Create, import, share VMs (e.g. use OVF) • Federation – targets networking • Portable VMs moved between clouds and hypervisors without reconfiguring anything • On-demand (burst) – targets APIs • Migration and federation on demand • Interoperability focused on storage and compute (e.g. CDMI, OCCI)

  40. Interoperability definition & dimensions POLICY: Federate, communicate between providers RUNTIME: Migration support DESIGN: Abstract the programmatic differences D. Petcu, Portability and Interoperability between Clouds: Challenges and Case Study, ServiceWave 2011 • Dictionary: • Property referring to the ability of diverse systems to work together • By mottos: • avoid vendor lock-in • develop your application once, deploy anywhere • enable hybrid clouds • one API to rule them all

  41. Current solutions Levels Techs E.g. Strategies, regulations, mode of use Function calls and responses Automation, configuration Standards in deployment & migration Protocols for requests/responses Pre-deployment, work-loads Allocation, admission E.g. Automated translation in code UCI Mediators, frame- works (SLA@SOI) OVF/DMTF, CDMI/SNIA OCCI, Deltacloud jClouds, libcloud, OpenStack Business Domain specific lang. Semantic Semantic repositories Appl & service Abstraction layers Management Standards Techs & infrastr Open protocols Image & data Open APIs Network

  42. Policy Design Run-time Public administration Application/ service abstraction Service automation VM, data & workloads Semantics Management Business Network Contract regulations Business regulations Execution-agnostic appl support Message content & protocols Reconfi-guration at run-time Life-cycle control Import/ share VM, data Uniform access to resources SLA regulations Regulations on use mode Infrastructu-re-agnostic program. Service calls & answers Scaling in and out Deployment uniform procedure Standards for accessing resources Routing optimization mechanisms Opti-mization mechanism Unified interfaces Semantic discovery Workflow manage-ment Migration support Support multiple hypervisors Software-defined networking License flexibility High level models Service and resource discovery Plus-ins for working environment Reservation and setup SLA & performance monitoring Automated provision Standardized allocation &admission Mapping on resources Single sign-on Audit and trust mechanisms

  43. Portability in Clouds? Q: How to port the appl? API spec API spec 01011001 API spec

  44. Portability between Clouds DATA: Import & export functionality SERVICE: On the fly add, reconfig and remove resources FUNCTION: Define appl. functionality in platform-agnostic manner • Ability to use components or systems lying on multiple hardware or software environments • Dimensions:

  45. Portability at XaaS level SaaS • Preserve/enhance functionality when substitute softw • Measures: • open source; proprietary/open formats; • - integration techs; appl server/OS Minim.appl.rewriting while preserve/ enhance control Measures: - proprietary vs.open APIs, progr.languages,data formats - tight vs. loose coupled services - abstract layers for queuing & messaging PaaS IaaS • Appls and data migrate and run at a new provider • Measures: • ability to port VMs and data • underlying configurations across providers

  46. Requirements for portability Economic models, cost-effectiveness, license flexibility, negotiated SLAs, leasing mechanisms Data portability and exchange, scale-out, location-free, workflow management Minimal reimplementation when move, standard APIs, same tools for cloud-based and entreprise-based appls SLA and performance monitoring, QoS aware services, service audit, sets of benchmarks Deploy in multiple clouds with single management tool, navigation between services, automated provisioning, resource discovery and reservation, behavior prediction Single sign-on, digital identities, security Standards, trust mechanisms, authentication Market Application Programming Monitoring Deployment AA & Security

  47. Model-Driven Engineeringand Clouds

  48. MODAClouds focus and challenges MOdel-Driven Approach for design and execution of applications on multiple Clouds • Focus: • on needs of Cloud-based Application Developers and Operators • Challenges: • Avoid vendor lock-in • Support risk analysis and management • Guarantee quality assurance

  49. Project goals MODAClouds helps Build & Run Cloud Apps for Multi-Cloud Environments (IaaS and PaaS) with a framework language and its integrating methods; IDE tools; an automated DSS and a runtime environment. MODAClouds supports and guides through the entire process of developing, monitoring and migrating cloud applications based on QoSinformation to manage apps on multiple clouds in an easy and automated way.

  50. MODAClouds objective • To provide methods, a decision support system, an IDE and a runtime environment to support • High-level design • Early prototyping • Semi-automatic code generation • Automatic (re)deployment • Monitoring and self-adaptation of applications on multi-Clouds with guaranteed QoS

More Related