1 / 44

Where I Want to be.....

Where I Want to be. +. What I'll Settle For. Requirements. Rock solid stability Automated node addition (discovery) Scales horizontally Service dependency models Easy to write plugins Promotes sane workflows Unified front end view Flexible configuration. Tool Stack.

race
Download Presentation

Where I Want to be.....

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Where I Want to be..... +

  2. What I'll Settle For.....

  3. Requirements • Rock solid stability • Automated node addition (discovery) • Scales horizontally • Service dependency models • Easy to write plugins • Promotes sane workflows • Unified front end view • Flexible configuration

  4. Tool Stack

  5. What????but #monitoringsucks and #ihatenagiosHow could you?

  6. In defense of Nagios • Been around since 1996 • Has Service dependencies • Easy to write plugins • Easy-ish to troubleshoot • ROCK SOLID

  7. Valid attacks on Nagios • No automated discovery • It's complicated to setup • Text files – really? • Front end won't win any beauty contests • Development is slow • Stats collection is a PITA

  8. Solutions • Use Icinga! • Use Puppet to auto configure • Stats – leave it to graphite. It's really good at that • Big boys and girls learn their tools

  9. Icinga • Fork of Nagios • Configurations are compatible • More solid architecture ( core, API, Web, IDODB ) • Nice front end, nice mobile front end • Can use NRPE

  10. High Level View

  11. Configure Icinga Servers using Puppet Standard Types

  12. Things to configure with Standard Types • icinga.cfg (file) => icinga main config file • Apache icinga.conf (file) => http access to each server • cgiauth.cfg (file) => cgi access • cgi.cfg (file) => options, users • templates.cfg (file) got lazy => use for basic classes • idomod.cfg (template) => template for hostname to DB

  13. Configure Icinga using Nagios Types

  14. nagios_hostgroup nagios_service nagios_servicedependency nagios_serviceescalation nagios_serviceextinfo nagios_servicegroup nagios_timeperiod Puppet Nagios Types • nagios_command • nagios_contact • nagios_contactgroup • nagios_host • nagios_hostdependency • nagios_hostescalation • nagios_hostextinfo

  15. Configuring Hosts

  16. Overview

  17. Detailed Overview

  18. Store Configs • Store puppet info in a DB • Retrieve information from DB • Share info across nodes • Use thin_storeconfigs • Set up on puppet master

  19. Exporting Nagios_host Resources • Export = Save to DB • Use facter for dynamic data • PRO TIP: use ENC • PRO TIP: use targets • PRO TIP: hostgroups • PRO TIP: use tags

  20. PRO TIP: Use your ENC

  21. PRO TIP: use targets • Use cfg_dir in icinga.cfg • Create a unique file per host or service • Addition and removal are now super easy • Also default dirs are in a horrible place /etc/nagios

  22. PRO TIP: hostgroups • Add machines to a hostgroup • Add services to a hostgroup • New machines inherit all of the services associated with a hostgroup

  23. PRO TIP: use tags Tags allow you to filter resources so that you only realize those resources that you need

  24. Configuring Services/Commands

  25. Icinga Services • OR 'Stuff I want to monitor' • Associate with a hostgroup • Use a target

  26. Icinga Commands • OR 'What actually gets run' • Use Macros to set paths in resource.cfg

  27. Dependencies

  28. PRO TIP: Dependencies • Unreliable services • Cut down on the number of alerts • Tell me what's really wrong • Route alerts accordingly

  29. Nagios_servicedependencies

  30. NRPE

  31. NRPE • Runs on client • Secured via SSL • Has ACLs • Runs as nobody • Can run commands • Useful for other things...

  32. Configuring NRPE

  33. NRPE Checks

  34. Plugins

  35. exchange.nagios.org

  36. Writing Plugins • Write in any language • Output 1 line to stdout • NRPE/Icinga/Nagios all use exit codes to determine status • Run by hand to check

  37. Workflows

  38. Watching Monitoring

  39. Scheduling Downtime

  40. Filtering

  41. Alerting

  42. #monitoringisawesome • REMOVE unreliable checks • Just MONITOR – don't bolt on - especially stats • TIER your monitoring • Use timeperiods for sanity • Delegate responses • Use dependencies to pin down problems quickly • Work smart

  43. Resources • Icinga http://icinga.org • Puppet http://docs.puppetlabs.com/references/latest/type.html#nagioscommand • NRPE http://nagios.sourceforge.net/docs/3_0/addons.html • IRC ##infra-talk, #icinga, #puppet • Contact christianmague@gmail.com, @maguec, #gaijin (freenode), http://blog.mague.com • Thanks Yvonne Kong, Michael Catlin, Juan Ortega, Anthony Kong, Puppet Labs, Icinga Team

More Related