OpenStack
This presentation is the property of its rightful owner.
Sponsored Links
1 / 23

OpenStack High Availability PowerPoint PPT Presentation


  • 104 Views
  • Uploaded on
  • Presentation posted in: General

OpenStack High Availability. Jakub Pavlik. About me. Jakub Pavlík Cloud Platform Engineer 3 years in Cloud 2 years in OpenStack. High Availability vs. Disaster Recovery.

Download Presentation

OpenStack High Availability

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Openstack high availability

OpenStack High Availability

JakubPavlik


Openstack high availability

About me

  • JakubPavlík

  • Cloud Platform Engineer

  • 3 years in Cloud

  • 2 years in OpenStack


Openstack high availability

High Availability vs. Disaster Recovery

  • High Availability = fault detection & correction procedures to maximize availability of critical services and applications, often in an automated fashion.

  • Disaster Recovery = process of preparing for recovery or continuation of technology infrastructure critical to an organization after a natural or human-induced disaster.

    High Availability ≠ Disaster Recovery!


Openstack high availability

Four types of HA in an OpenStackCloud

Service Resiliency

QoS Cost

Transparency

Data Integrity

…..

Compute Controller

Network Controller

Database

Message Queue

Storage

....

Applications

OpenStack

Control services

VMs

OpenStack Compute

Physical nodes

Physical network

Physical storage

Hypervisor

Host OS

….

Virtual Machine

Virtual Network

Virtual Storage

VM Mobility

Physical infrastructure


Openstack high availability

Physical Infrastructure


Openstack high availability

tcp cloud

VPC

Hardware

Switch 1

Switch 2

Passthru 1

Passthru 2

Passthru 1

Passthru 2

168 cores 3,46GHz ,336 threads

agregation ¼ : 1344 vCPU

2688 GB RAM

28 x 10GE ports

168 cores 2,67GHz ,336 threads

agregation ¼ : 1344 vCPU

1792 GB RAM

28 x 10GE ports

SAN 1

SAN 2

SAN 1

SAN 2

Controller 1

Controller 2

Controller 1

Controller 2


Openstack high availability

OpenStack Control services


Openstack high availability

OpenStack modules – TCP VPC


Openstack high availability

OpenStack High Availability Concepts

  • Stateless services

    • There is no dependency between requests

    • For example APIs: Nova, Keystone, Glance, Cinder, etc.

  • Stateful services

    • An action typically compromises multiple requests

    • For example: MySQL, RabbitMQ, etc.

  • Active/Passive

    • Redundant instances of stateless services are load balanced

    • For Stateful services a replacement resource can be brought online

  • Active/Active

    • Redundant instances of stateless services are load balanced

    • Stateful services are managed in such a way that services are redundant, and that all instances have and identical state.


Openstack high availability

Corosync, Pacemaker and HAProxy

  • Corosync

    • Totem single-ring ordering and membership protocol

    • UDP and InfiniBand based messaging, quorum, and cluster membership to Pacemaker

  • Pacemaker

    • High availability and load balancing stack for the Linux platform.

    • Interacts with applications through Resource Agents (RA)

  • HAProxy

    • Load Balancing and Proxying for HTTP and TCP Applications

    • Works over multiple connections

    • Used to load balance API services


Openstack high availability

MySQL Galera

  • Synchronous multi-master cluster technology for MySQL/InnoDB

  • MySQL patched for wsrep (Write Set REPlication)

  • Active/active multi-master topology

  • Read and write to any cluster node

  • True parallel replication, in row level

  • No slave lag or integrity issues


Openstack high availability

Sample OpenStack HA architecture

  • Stateful

    • Cinder Volume

    • Neutron L3, DHCP agents

    • Ceilometer central agent

    • RabbitMQ

  • Stateless

    • Neutron Server

    • OpenStack APIs

    • Apache web server

    • Nova Scheduler

    • Cinder Scheduler

Neutron agents

(Active)

Neutron agents

(Hot Standby)


Openstack high availability

VMs – Compute nodes


Openstack high availability

VMs HA – two layers

  • Storage

    • Shared storage filesystem – file disks (qcow2, vmdk, vhv)

    • Block storage

  • Network

    • Vanilla Neutron L3 agent (OpenVSwitch, Linux Bridge)

    • Vendor plugins - SDN controller


Openstack high availability

No vSphere Style HA with KVM


Openstack high availability

Non-Shared/Shared Storage filesystem

  • Shared Storage

    • Live migration – just RAM memory

    • Hypervisor Evacuation – The instance will be booted from same disk and data will be preserved

    • CEPH, Gluster, NFS, Samba, GFS

  • Non-Shared Storage

    • Block Live Migration – disk and RAM

    • Hypervisor Evacuation – the instance will be booted from a new disk, but will preserve the configuration, e.g. id, name, uuid

    • Standard filesystem EXT4, etc.


Openstack high availability

Block Storage - Cinder

  • Instance boots from volume

  • iSCSI/FC direct mapping to instance

  • Enable Live Migration

  • Cinder Backends

    • LVM Driver

      • Default linuxiSCSI server

    • Vendor software plugins

      • Gluster, CEPH, VMware VMDK driver

    • Vendor storage plugins

      • EMC VNX, IBM Storwize, Solid Fire, etc.


Openstack high availability

Networking - Vanilla Neutron L3 agent

  • Problems

    • Routing on Linux server (max. bandwith approximately 3-4 Gbits)

    • Limited distribution between more network nodes

    • East-West and North-South communication through network node

  • High Availability

    • Pacemaker&Corosync

    • Keepalived VRRP

    • DVR + VRRP – should be in Juno release


Openstack high availability

Networking – Vendor SDN Controller plugins

  • Examples

    • Juniper OpenContrail, VMware NSX, SDN PLUMgrid

  • Advantages against Neutron L3 agent

    • North-South communication on network devices (iBGP, MLPSoverGRE)

    • East-West communication directly between compute nodes

    • Higher bandwidth (9.7 Gbits per 10Gbits port)

  • High Availability

    • iBGP peering into two routers

    • Native HA implemented inside of network devices


Openstack high availability

Contrail Database

Contrail Database

Contrail Database

Openstack Controller

Openstack Controller

Openstack Controller

Zookeeper

Zookeeper

Zookeeper

OpenStack HA

TCP VPC

VIP

Cassandra

Cassandra

Cassandra

GALERA

MySQL

MySQL

MySQL

RabbitMQ

RabbitMQ

RabbitMQ

Bond Interface

Pacemaker

Corosync

Pacemaker

Corosync

HAProxy

HAProxy

HAProxy

Contrail Config with Analytics & WebUI

Contrail Config with Analytics & WebUI

Contrail Config with Analytics & WebUI

Contrail Control

Contrail Control


Openstack high availability

TCP Virtual Private Cloud


Openstack high availability

HA methods - vendors


Openstack high availability

Thank you for your attention!


  • Login