Loading in 2 Seconds...
Loading in 2 Seconds...
Meetup Fairfax Lean Agile Engineering Practices CI/CD Continuous Integration Continuous Delivery. Ramakrishnan Srinivasan, AOL ( firstname.lastname@example.org ) Thursday, September 18, 2014 *Everything I say reflects my own opinion only. CI/CD: We will cover. Why CI/CD?
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
MeetupFairfax Lean Agile Engineering PracticesCI/CDContinuous IntegrationContinuous Delivery Ramakrishnan Srinivasan, AOL (email@example.com) Thursday, September 18, 2014 *Everything I say reflects my own opinion only.
CI/CD: We will cover • Why CI/CD? • The development process & its goals • Industry & business demands • Discuss DevOps, CI, CD and available software • The CI/CD setup • Opscode Chef terminology & basics • Chef Project structure & show a few screens • References • Q & A
We will speak to • Operations folks • Quality Assurance analysts • Business managers & analysts • Software engineers
DevOps • Its about increasing efficiencies in the development/QA/deployment process • Its about increasing agility and output of the total system • Emphasizes collaboration of Dev, QA and Ops • QA personnel need to learn some software development in order to automate testing • Operations folks will find their tasks to be much easier • There is an increasing demand for these skills
Typical workflow in s/w shop • Developer works on a story to satisfy specific requirements • Code is checked in to the repository • Developer initiates a build • Upon a successful build, QA team is notified • QA installs the build in their environment • QA tests the delivered functionality • Bugs are reported & resolved • Once cleared, its installed in production
The problems & motivation • Code breaks. Defects are inevitable. • No code change, but an external system has changed - breaking our functionality • We forget to run our unit tests before checking in code • We want to know about issues sooner rather than later • We want to be faster to market, deliver rapidly and often • We want to automate deployment in a configurable way
Business demand: Efficiency • We are constantly striving to improve productivity and efficiency • Developers • Make functions generic • Write unit tests • Code with a view to make debugging easy • Only checkin code when • you are actually done, or • you are sure of not breaking existing functionality
Efficiency .. 2 • Testing: Automate where possible • Goals • Reduce costs • Decrease number of defects • Increase productivity • Spend more time building the product and creating value for the customer • Spend less time on functions/process that can be automated • Do more with less
CI: A solution to improve efficiency • Write unit tests with sufficient coverage for the important parts • Trigger a code build on every checkin or maybe every 5 minutes • Run unit tests as part of the build • Write integration tests for both UI and backend • Run integration tests frequently • Tests must be high quality & cover major functions • Detect issues early, and as they happen
CD: Continuous Deployment • Code and tests must be deployed before it can run • CD is really part of the CI solution • Adap.tv does several deployments to production on a daily basis
CI Practices – Martin Fowler • Maintain a Single Source Repository • Automate the Build • Make Your Build Self-Testing • Everyone Commits To the Mainline Every Day • Every Commit Should Build the Mainline on an Integration Machine • Keep the Build Fast • Test in a Clone of the Production Environment • Make it Easy for Anyone to Get the Latest Executable • Everyone can see what's happening • Automate Deployment • Reference: http://martinfowler.com/articles/continuousIntegration.html
CI/CD: Benefits • Technical • Continuous delivery • Standardized environments • Less complex problems since they are identified quickly • Faster resolution • Business • Faster delivery of features • More stable, controlled, and predictable operating environments • Focus on adding business value rather than fixing issues • Increased process effectiveness • Increased agility
CI/CD: Summary • Automate testing as part of the build • Automate deployment to staging, test or production • Automate rollback • Deploy continuously to production • Incremental new features initially enabled for small set of users, then turned on for a wider audience
Software alternatives for CI/CD • Plenty of alternatives available • Opscode Chef • Puppet • Ansible • Salt • Cobbler • Here are a few: http://alternativeto.net/software/chef/
Software used in my project • need to install Ruby • Berkshelf – used to manage dependencies in Chef • Chef Server on the server • chef-client • Jenkins • SonarQube • git
Servers used in a CI/CD setup • Build server • Deployment server • Chef Server for provisioning & deployment • Chef Client on each participating node • Jenkins for scheduling and coordinating automated build pipelines • SonarQube: continuously analyze and measure code quality • A single server can be used for many of the above • Of course, servers VMs can be started up on demand in the cloud
Chef basics/terminology - I • Environment: settings, properties maintained in each environment like Dev, QA and Production • Bootstrap: first time setup • Server: hub for configuration data • Client/Node: a computer/laptop/server that can run chef-client • Workstation: its configured to use the knife command • knife command: Used to activate Chef functionality
Chef basics/terminology - II • Based on the Ruby language • Attributes: properties, key-value pairs • Resources: commands, e.g. to run a shell script • Templates • Recipes • Cookbooks • Databags: JSON data accessible in a Chef environment that can be encrypted. • Libraries • Overview: http://docs.getchef.com/chef_overview.html
Chef Attributes • key-value pairs • default values can be set • Some values can be overridden in specific environments • e.g. Connecting to Salesforce • Dev, QA and Production instance URLs can be overridden in respective environments • Reference: https://docs.getchef.com/chef_overview_attributes.html
Example project • Java Web application • Apache Tomcat • MySQL or Oracle • Dev, QA and Production environments • CI/CD pipeline
CI/CD Pipeline • build the project • execute unit tests • for each build, create new linux RPM package • deploy to CI server • run smoke tests • automated backend integration tests • automated UI testing jobs (Selenium and CasperJS for automated UI testing) • deploy to production if delivery criteria are met
Chef Resources • Template • Package • Directory • bash, csh • Service • Cron • Deploy • Reference: http://docs.getchef.com/chef/resources.html
Templates • these are static text files • with variables embedded in them • Node attributes can be referenced • to dynamically generate environment-specific files like scripts and config files
Recipes & cookbooks • basic Ruby syntax • the programming part of Chef • has standard programming constructs like loops, conditionals, functions, etc. • a Recipe resides in a Cookbook - collection of recipes that belong together • uses resources (like bash and service) to generate and install software • download new package versions only when available • Just enough Ruby for Chef • Reference: https://docs.getchef.com/essentials_cookbook_recipes.html
Data bags • global variables Chef-wide • stored as JSON data • Can be encrypted • Can be accessed by recipes • Reference: http://docs.getchef.com/essentials_data_bags.html
Libraries • Contain Ruby functions • Provides the ability to execute arbitrary Ruby code • Use to define custom resources, connect to a database, etc. • Use to define template helper modules • Reference: https://docs.getchef.com/essentials_cookbook_libraries.html
Chef Project structure • Project • attributes • environments • Ruby classes • Contain Ruby hashes (KV pairs) • like a JSON dictionary or associative array • Different format than attributes • templates (static text, config files) • data_bags (encryptable JSON data) • libraries (any Ruby code, helper functions)
Chef Development lifecycle • Setup project (one-time) • Create recipes, organized inside cookbooks • Test cookbooks locally using Test Kitchen (good option) or Vagrant • Run cookbooks/recipes • Check the target server • If, e.g. a file is not in place as expected, fix recipe and repeat • Once its working, push to git (or other repository) • Run a recipe: • chef-client -o recipe[my_cookbook::my_recipe]
References • Learn Chef: http://learn.getchef.com/ • Test Kitchen: https://github.com/test-kitchen/test-kitchen • Vagrant: https://www.vagrantup.com/ • ThoughtWorksTreehouse blog • ThoughtWorks: ThoughtWorks: http://www.thoughtworks.com/continuous-integration • How DevOps is killing the Developer • Etsy • Their story • Continuous delivery with product/feature flags: • Continuously deliver with confidence • Etsy deploys over 50 times a day • Adap.tv engineering blog
Thank you!!! • Questions • Discuss • Feedback