Configuration Management in the Cloud

1. Configuration Management in the Cloud Puppet and Chef

2. Let’s talk • What is Configuration Management • diff cloud.txtphysical.txt > painful.out • Why is it painful? • Infrastructure as Code • Puppet • Chef • Examples

3. What is Configuration Management My own definition: “The art of keeping everything under control” Wikipedia: “Configuration management (CM) is a systems engineering process for establishing and maintaining consistency of a product’s performance, functional and physical attributes with its requirements, design and operational information throughout its life.”

4. diff cloud.txtphysical.txt > painful.out • Physical: • Resources stay there “forever” • Attributes / properties are static (ips/ hostnames / macaddress) • Some cases is possible to recover the same system • Cloud: • Resources are dynamicand in constant change“Some times they just disappear, WTF is the cloud it should be always there” • Attributes / properties change without notice • Once a system is done, its done

5. Why is it painful? • Config management systems where design for static/physical environments. • Most of them use certs/keys based on hostnames. • With things as “bursting into the cloud” the config management server that supported 100 servers now it has to support 1K, 2K 15K servers. • Most cloud environments cloud instances come and go. • In physical environments you don’t need completely automation from 0 to app • Most CMS’s don’t have rollbacks.

6. Infrastructure as Code in the Cloud • Keep your CM code in repositories (git/svn) • Replicate… replicate… replicate… • The CM system wont do everything by itself • Have your Dev, Test and Prod environments • If something fails… destroy and rebuild • Go Masterless whenever possible

7. Puppet • Pros • Ruby based • Easy to read and learn • You can do pretty much anything • Cons • Custom changes require you to build specific prividers, resources and the DSL is not as good as you would like • Based on certs using hostnames to generate them • Master/Client communication • Does not scale very well

8. Chef • Pros • Ruby based • You literally can code in it • You can apply order to the things he will execute • Provides an encrypted way to pass sensitive data • Provides more utilities (knife and search) • Chef • Master server requires more components • Syntax a little bit more complex • You need to learn ruby to get the good out of it • Master/Client communication

9. Puppet Arch • Semi Masterless • Architecture:

10. Chef Arch • Master/Client • Architecture

11. Puppet Module • Apache • Files • Cert.key • Ca.key • Templates • Vhost.erb • Manifests • Init.pp • Redhat • Install.pp • Config.pp • Postconfig.pp • Service.pp

12. Puppet Code – init.pp • Init.pp Class apache ( $servername = “myserver”, $port = 80, $serveradmin = “admin@email.com” ) { case @::operatingsystem { “redhat”, “centos”: { require apache::redhat::service } “ubuntu”: { require apache::ubuntu::service } default: { require apache::redhat::service } } }

13. Puppet Code – install.pp Class apache::redhat::install ( ) { package { “httpd”: ensure => “latest”; } }

14. Puppet Code – config.pp Class apache::redhat::config ( $servername = $apache::servername, $serveradmin = $apache::serveradmin, $serverport = $apache::serverport ) { require apache::redhat::install file { “/etc/httpd/conf.d/myvhost.conf”: owner => “apache”, group => “apache”, content => template(‘apache/vhost.erb’); } }

15. Puppet Code – service.pp Class apache::redhat::service ( ) { require apache::redhat::config service { “httpd”: ensure => “running”; } }

16. Puppet Masterless • Create bootstrap script that: • Download Repository into the Cloud instance • Create a manifest.pp with the contents of the node definition • Call puppet apply -vd --modulepath=/location/modules/ manifest.pp • Example manifest.ppimport “whatever”class { “apache”: servername => “myserver.com”,serveradmin => “myemail@gmail.com”, port => 8080}

17. Chef Code • Roles • Webserver.json • Cookbooks • Attributes • Default.rb • Files • Cert.key • Ca.key • Templates • Vhost.erb • Libraries • Providers • Resources • Recipes • Default.rb • install.rb • Config.rb • Vhost.rb

18. Chef Roles { "name": ”webserver", "default_attributes": { "service": ”httpd”, “port”: “80”, "packages": { "extras": [ ”httpd" ] } }, "chef_type": "role", "env_run_lists": { }, "run_list": [ "recipe[minitest-handler@1.0.6]", "recipe[basenode@1.0.50]", "recipe[chef-client@1.1.26]", "recipe[release_version@8.0.19]", "recipe[ops@1.0.16]", "recipe[chef-workstation@1.0.5]”, “recipe[apache@1.0.1]” ], "override_attributes": { }, "description": ”webserver", "json_class": "Chef::Role" }

19. Chef Cookbook - Attributes default['dns']['subdomains'] = ['production', 'test', 'development'] default['dns']['basedomain'] = 'demiops.com.' default['dns']['route53']['register'] = true default['dns']['route53']['default_ttl'] = '300' default['resolver']['options'] = ['rotate', 'attempts:5'] default['resolver']['nameservers'] = ['127.0.0.1'] default[‘web’][‘port’] = ‘80’ default[‘web’][‘servername’] = ‘myserver.com’ default[‘web’][‘serveradmin’] = ‘myemail@gmail.com’

20. Chef Cookbook - Recipes Default.rb include_recipe“apache::install" include_recipe”apache::config" include_recipe “apache::vhost" include_recipe”apache::authorized_keys” Authorized_keys.rb cookbook_file "/root/.ssh/authorized_keys" do group "root" owner "root" mode 0600 source "authorized_keys" end

21. Chef in the Cloud • Create a bootstrap script that: • Download the chef repository into the cloud instance • Use minitests to check everything worked • Install chef-client and knife in the instance • Use knife to search chef-client inventory and update dynamically config files • Use ohai

22. Questions ?