Quattor ncm components introduction tutorial
This presentation is the property of its rightful owner.
Sponsored Links
1 / 28

quattor NCM components introduction tutorial PowerPoint PPT Presentation


  • 50 Views
  • Uploaded on
  • Presentation posted in: General

quattor NCM components introduction tutorial. German Cancio CERN IT/FIO. Contents. Preparation NCM ‘theory’ Components – what are they, how to run them Packaging NCM exercises Disclaimer:

Download Presentation

quattor NCM components introduction tutorial

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Quattor ncm components introduction tutorial

quattor NCM componentsintroduction tutorial

German Cancio

CERN IT/FIO


Contents

Contents

  • Preparation

  • NCM ‘theory’

    • Components – what are they, how to run them

    • Packaging

  • NCM exercises

    Disclaimer:

    This is not a full quattor tutorial, but an introduction to the NCM framework. In particular, neither CDB/pan, nor SWRep/SPMA, nor AII are covered. A basic understanding of the PAN language is recommended.


Preparation

Preparation

  • Login onto your ‘worker node’ (see whiteboard) as root

  • Please download the following tar.gz file, and uncompress it in your $HOME directory (~root):

    • cd ~

    • wget http://cern.ch/quattor/documentation/tutorials/cern-0404/ncm-tutorial.tgz

    • tar xvfz ncm-tutorial.tgz

  • Run the setup script:

    • ~/ncm-tutorial/bin/tutorial_setup.sh

    • should print [OK]


Ncm environment

Packages

(rpm, pkg)

NCM environment

SWRep Servers

http

cache

SPMA

packages

Mgmt API

nfs

SPMA.cfg

(RPM, PKG)

ACL’s

ftp

SPMA

SPMA

NCM

Components

NCM

Node (re)install?

AII server

Cdispd

PXE

CCM

PXE

handling

Mgmt API

Registration

Notification

ACL’s

Node

Install

DHCP

DHCP

handling

KS/JS

KS/JS

generator

Client Nodes

CCM

CDB


What are components 1 2

What are components? (1/2)

  • “Components” (like SUE “features” or LCFG ‘objects’) are responsible for updating local config files, and notifying services if needed

  • Components do only configure the system

    • Usually, this implies regenerating and/or updating local config files (eg. /etc/sshd_config)

  • Use standard system facilities (SysV scripts) for managing services

    • Components can notify services using SysV scripts when their configuration changes.

  • Components can be run

    • manually

    • via hooks (cron, boot time, etc)

    • via CERN-CC’s notification framework (notd)

    • automatically: register their interest in configuration entries or subtrees, and get invoked in case of changes (disabled at CERN-CC)

  • Possible to define configuration dependencies between components

    • Eg. configure network before sendmail

    • Components won’t run if a pre-dependency is unsatisfied (eg. failing prerequisite component)


What are components 2 2

What are components? (2/2)

  • Components are written as Perl OO class instances

  • Each component has to provide two methods:

  • Configure():

    • invoked when there was a CDB configuration change or on startup

    • Mandatory method

  • Unconfigure():

    • invoked when a component is to be removed

    • Optional method – most of the components won’t need to implement it.


How to run components

How to run components?

  • ncm-ncd (Node Configuration Deployer):

    • framework and front-end for executing components (via cron, cdispd, or manually)

    • dependency ordering of components

  • cdispd (Configuration Dispatch Daemon)

    • Monitors the config profile, and invokes registered components via ncm-ncd if there were changes

    • not described in this tutorial.

  • ncm-query

    • Tool for examining configuration information as cached on the node


Component simplified example

Component (simplified) example

sub Configure {

my ($self,$config) = @_;

# access configuration information

my $arch=$config->getValue('/system/architecture’); # NVA API

$self->Fail (“not supported") unless ($arch eq ‘i386’);

# (re)generate and/or update local config file(s)

open (myconfig,’/etc/myconfig’); …

# notify affected (SysV) services if required

if ($changed) {

system(‘/sbin/service myservice reload’); …

}

}


Components and cdb profile access

Components and CDB profile access

NVA API: configuration access library

  • This library allows to access the node profile’s configuration (hierarchical structure)

  • Most popular methods:

    • $value=$config->getValue(‘/system/kernel/version’);

    • if ($config->elementExists($path)) {…} else {…}

    • $element=$config->getElement($path);while ($element->hasNextElement()) {my $newel=$element->getNextElement(); ...}


Component support libs

Component support libs

  • Reporting functions:

    • $self->log(@array): write @array to component’s log file

    • $self->report(@array): write @array to log and stdout.

    • $self->verbose(@array), $self->debug(level,@array): verbose/debug output

    • $self->warn(@array): writes a [WARN] message, increases # of warnings

    • $self->error(@array): writes an [ERROR] message, increases # of errors

      Failures of reconfigurations are done using ‘error(..)’. Components depending on a failed component are not executed.

  • Advanced support libraries available (revamped from CERN’s SUE):

    • Configuration file manipulation

    • Advanced file operations

    • Process management

    • Exception management libraries

      See /opt/edg/lib/perl/LC for details …


Real component walktrough

Real component walktrough

  • ncm-state (/usr/lib/perl/NCM/Components/state.pm)

    • Updates a local configuration file (/etc/state.desired) with the node’s production state (“production”, “standby”, etc.)


Packaging components files 1 3

Packaging components: files (1/3)

  • Each component is packaged independently and kept in a separate CVS area.

    • Let’s check our example: please cd to

      ~/ncm-tutorial/component-cvs/ncm-state

  • Files to be present in a component CVS module:

    • README – small intro

    • ChangeLog – automatically maintained ChangeLog file.

    • LICENSE – contains license or pointer to it

    • MAINTAINER – one liner with email

    • Makefile – copied from quattor-build-tools, essentially an ‘include’ of the quattor-build-tools


Packaging components files 2 3

Packaging components: files (2/3)

  • config.mk – obligatory and optional definitions

    • COMP – component name

    • DESCR – one liner with component description

    • VERSION – in the format <majorversion>.<minorversion>.<release>

    • RELEASE – RPM release number (always 1 for time being  )

    • AUTHOR – author’s email

    • DATE – dd/mm/yy hh:mm

    • Any other optional definition(s)

  • comp.pm.cin –component source file (a la ‘autotools’)

    • Source format! @TAGS@ get expanded into configuration variable values

    • Configuration variables set in config.mk or predefined inside quattor-build-tools (quattor-Linux.mk, quattor-SunOS.mk)

  • comp.pod.cin – POD doc file. Please follow conventions:

    • NAME

    • SYNOPSIS: Configure() and Unconfigure() documentation

    • RESOURCES: describe all used resources

    • DEPENDENCIES: ‘pre’ and ‘post’ dependencies

    • BUGS

    • AUTHOR

    • SEE ALSO

  • pro_declaration_component_comp.tpl.cin – PAN component template


Packaging components files 3 3

Packaging components: files (3/3)

  • Extra data files (template config files, etc): Can be stored in a subdirectory called ‘DATA’

    • Example: see ncm-state/DATA

    • The file is copied into the directory @NCM_DATA_COMP@ (typically, /usr/lib/ncm/config/name/)

  • Extra documentation files: Can be stored in a subdirectory called ‘DOC’

    • The file is then dropped into the standard RPM package documentation directory

  • Extra template files: Can be stored in a subdirectory called ‘TPL’

    • All files are then dropped into the standard RPM package doc directory

    • The pro_declaration_component… template can be stored either in the top level directory or in the ‘TPL’ directory

    • All templates are copied to a common directory as well (QTTR_DOCDIR/pan-templates, typically /usr/share/doc/pan-templates)


Packaging components commands

Packaging components: commands

  • Generating packages out of checked out sources

    • ‘make rpm’ – generates RPM (on RHLinux)

    • ‘make pkg’ – generates PKG (on Solaris)

  • ‘make (release|minorversion|majorversion)’ – generate new version

    • checks in modified files to CVS

    • Prompts for ChangeLog entry (one line)

    • Increases release / minorversion / majorversion in config.mk

    • Generates a new CVS tag for the component

    • Note: ensure your files are declared in CVS (ie. ‘cvs add’)

  • make clean

    • Removes temp files generated during package build process


Exercises

Exercises


Exercise environment i

Exercise environment (I)

The demo setup has been simplified compared to the real production environment.

  • Only base component functionality is showed in the exercises.

    • The rest is left as exercise to the students !

  • No “real” CDB client/server flow

    • … but wrapper script provided for generating and updating profiles locally


Exercise environment ii

Exercise environment (II)

  • Documentation:

    See

    http://quattor.org/documentation/tutorials -> CERN

    for this slides (and solutions)

  • Man pages:

    • ncm-query(1), ncm-ncd(1)

    • ncm-component(8)


Exercise environment iii

Exercise environment (III)

  • Checking the ncm tutorial directory contents:

    • ~/ncm-tutorial/bin:

      • cdb_wrapper.sh: wrapper for running ‘pan’ and CCM

      • pan.rh7: ‘pan’ binary – (do not use directly)

    • ~/ncm-tutorial/templates:

      • *.tpl: the exercise ‘pan’ template files for our NCM configurations

    • ~/ncm-tutorial/component-cvs:

      • quattor-build-tools: tools for building packages

      • ncm-state: our example component

    • ~/ncm-tutorial/xml:

      • Temporary dir for XML profiles generated by cdb_wrapper.sh


Ncm exercises

NCM exercises

First: Quick intro to the demo NCM CBD/pan template structure and contents

Exercises:

  • Compile default profile

  • Modifying the profile

  • Running a component and changing its configuration

  • Extending a component (ncm-state)


Cdb templates for ncm components

CDB templates for NCM components

  • Please cd into ~/ncm-tutorial/templates

  • The templates provided serve the configuration of 3 NCM components (see next two slides)

  • Every time a template is updated: ‘pan’ and CCM need to be invoked using a wrapper to use the right directories and includes.

    ~/bin/cdb_wrapper.sh


Quattor ncm components introduction tutorial

Template hierarchy for

NCM config tutorial

pro_declaration_component_type

pro_declaration_system

grub

state

testcomp

pro_declaration_component_<x>

pro_default_tutorial

“cluster” or “service” level config values

Types & structures

data templates

profile_dummy

Templates to be edited

Node level config values.

Inherited values can be overwritten


Cdb templates for ncm contents

CDB templates for NCM: contents

  • $ less pro_default_tutorial.tpl

  • Components can have “private” configuration entries:

    /software/components/name/active (bool)dependencies/pre (string[])dependencies/post(string[])foo/... (component specific)bar/... (component specific)

  • Components can access configuration information anywhere in the node profile (/system/.., /software/.., /hardware/..)

    • Useful to share common configuration entries between components

    • Eg. /system/state, /system/kernel/version

  • All components need to declare their “private” config data types

    • pro_software_component_type: common definitions

    • pro_software_component_name: added definitions


Exercise 1

Exercise 1

Generate default profile, run NCM

  • ‘cd’ into ~/ncm-tutorial/templates

  • Generate the profile with ../bin/cdb_wrapper.sh

  • Use ncm-query for visualizing the configured component list, and to visualize all configuration information:

    ncm-query –-components

    ncm-query –-dump /

  • Run now all active components:

    ncm-ncd –-configure

    (logfiles are generated under /var/log/ncm/..)

  • Run now a specific component:

    eg. ncm-ncd –-configure testcomp

    What other component is run, and why?

    Note: ncm-ncd –-help and ncm-query –-help provide you with all options and default values. Try also man ncm-ncd or ncm-query.


Exercise 2

Exercise 2

Modify default profile, run NCM

  • Modify the default profile to deactivate ‘testcomp’, re-run cdb_wrapper, re-run ncm-ncd –-configure

    (modify the pro_default_tutorial template)

  • What happens if ‘grub’ is deactivated in the profile, but ‘testcomp’ is activated, and ncm-ncd is run?


Exercise 3

Exercise 3

Modify a component’s configuration

  • Read the component’s man page for ‘grub’.

    man ncm-grub

  • Examine which kernels are installed on your node, and modify the kernel version to be used on the next reboot

    What kernel is currently configured? (using ncm-query --dump)

    Available kernels: cat /etc/grub.conf or ls /boot/vmlinuz*

    Don’t reboot the node! :-)

  • Try out what happens if you configure a non-existing kernel.

  • What happens if you run ncm-ncd –-unconfigure grub and why?

  • Restore the previously configured kernel.

    Note: The component code is found under /usr/lib/perl/NCM/Component/grub.pm


Exercise 4

Exercise 4

Modify the ncm-state component to add a ‘reason’ field to /etc/state.desired

  • Add a ‘reason’ field to the state NCM declaration template

  • Add the wanted value to the ‘data’ template

  • Regenerate the profile and verify with ncm-query that the new field is there

  • Add handling code to the component source to read out the new field, and to update /etc/state.desired

  • Generate the ncm-state rpm after increasing the ‘minorversion’ (not manually)

  • install the rpm (with rpm –Uvh <filename> - in the ‘real’ world this would be done via SPMA)

  • Run the component, and verify that /etc/state.desired contains the new data field

  • Did you forget to ‘activate’ the component?


Quattor ncm components introduction tutorial

http://quattor.org


  • Login