scs computing facilities software installation on facilitized linux machines l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
SCS Computing Facilities: Software Installation on Facilitized Linux Machines PowerPoint Presentation
Download Presentation
SCS Computing Facilities: Software Installation on Facilitized Linux Machines

Loading in 2 Seconds...

play fullscreen
1 / 38

SCS Computing Facilities: Software Installation on Facilitized Linux Machines - PowerPoint PPT Presentation


  • 289 Views
  • Uploaded on

SCS Computing Facilities: Software Installation on Facilitized Linux Machines. Mark Held, Ken Tew and Jeff Baird Carnegie Mellon University April 15, 2008. How Software Is Installed. These slides (and past SCS Admins Presentations) are available under http://www.cs.cmu.edu/~markh.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'SCS Computing Facilities: Software Installation on Facilitized Linux Machines' - daniel_millan


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
scs computing facilities software installation on facilitized linux machines

SCS Computing Facilities:Software Installation on Facilitized Linux Machines

Mark Held, Ken Tew

and Jeff Baird

Carnegie Mellon University

April 15, 2008

how software is installed
How Software Is Installed
  • These slides (and past SCS Admins Presentations) are available under http://www.cs.cmu.edu/~markh

CMU / SCS Computing Facilities

how software is installed3
How Software Is Installed
  • SUP (Software Update Program)
    • System files
  • Depot
    • Application Files (/usr/local)
  • Misc
    • Uses Depot for distribution
    • Details on how application software is prepared for distribution via Depot in the SCS Environment

CMU / SCS Computing Facilities

sup software update program
SUP (Software Update Program)
  • SUP runs nightly
    • Starting at 2:00 AM
      • With random delays of between 0 and 5.5 hours
      • Delays spread out load on servers
  • SUP updates “system” files
    • Not /usr/local
    • Kernel, /usr/bin, /bin, /lib, /usr/lib, /etc

CMU / SCS Computing Facilities

sup software update program5
SUP (Software Update Program)
  • Uses timestamps to determine what needs to be updated on the target machine
  • Basically a single collection is updated
    • fac.<AFS_SYSTYPE>
      • fac.amd64_f7 For Example
  • What has been updated is recorded in /usr/adm/fac/sup/fac.<AFS_SYSTYPE>/last
  • /usr/adm/fac/sup/fac.<AFS_SYSTYPE>/when is a binary timestamp

CMU / SCS Computing Facilities

sup software update program6
SUP (Software Update Program)
  • Trigger mechanism can invoke a script when a particular file has been updated.
    • See /usr/adm/fac/log/sup.log for examples
    • Merge of local and global printcap files into /etc/printcap (and then restart cups) is an example
    • .global and .local conventions

CMU / SCS Computing Facilities

sup software update program7
SUP (Software Update Program)
  • .global Convention
    • Used by printcap, inetd.conf, services
    • See the “last” file for all examples
    • Modern applications may not require a merged file
      • Read all files in a directory, so merge unnecessary
      • Examples: Cron.d, Pam.d, Init.d, Xinetd.d

CMU / SCS Computing Facilities

sup software update program8
SUP (Software Update Program)
  • Only One SUP Server in use
  • Security Updates are SUP’ed out
    • RPMs downloaded, then run
  • SUP will NOT delete local additions
    • Even overwritten files are not touched if the timestamp is newer than the file in the repository.

CMU / SCS Computing Facilities

sup software update program9
SUP (Software Update Program)
  • To halt SUP (and Depot) updates
    • Touch /etc/disableupdates (as local root)
    • Dangerous, as you will not get security updates
    • Machine could get into a “bad” state from which it cannot recover without intervention

CMU / SCS Computing Facilities

sup software update program10
SUP (Software Update Program)
  • FC5 Versus F7
  • The Fedora folks are including more packages with their OS distribution
    • The “C” means (meant) “Core”
  • Consequently some applications have migrated from /usr/local to /usr/bin and /bin

CMU / SCS Computing Facilities

depot
Depot
  • Depot maintains the /usr/local directory hierarchy
    • AFS-based repository of applications copied to local disks of all facilitized machines
    • 2nd part of the nightly ‘dosupdepot’ process

CMU / SCS Computing Facilities

depot12
Depot
  • Depot can be used to integrate local software into the /usr/local hierarchy
  • Process is relatively easy

CMU / SCS Computing Facilities

depot integration into usr local
Depot – Integration into /usr/local
  • First, install your software into:
    • /usr0/local
  • Do not leave “cruft” in the /usr0/local directory structure
  • Make sure that all paths are relative or if absolute refer to /usr/local (and not /usr0/local)

CMU / SCS Computing Facilities

depot integration into usr local14
Depot – Integration into /usr/local
  • Second, In your /usr/local/depot/depot.pref.local file add the following line:   path  mylocal  /usr0/local
  • This will tell depot to look in /usr0/local for a collection called 'mylocal' and link all of files found there into the /usr/local hierarchy. 

CMU / SCS Computing Facilities

depot integration into usr local15
Depot – Integration into /usr/local
  • Note that you can install multiple packages into /usr0/local and depot will link all of them into your /usr/local environment.
  • The install directory does not have to be /usr0/local, and directory will do.
    • However, /usr0 is backed up.

CMU / SCS Computing Facilities

depot integration into usr local16
Depot – Integration into /usr/local
  • Third, as root, run the “dosupdepot” command. 
  • Save the output somewhere, such as: dosupdepot >> /tmp/depot.log 2>&1
    • Then “tail –f /tmp/depot.log” to see the output as it is written.
  • Nightly runs will store log output in the /var/adm/fac.log directory

CMU / SCS Computing Facilities

depot integration into usr local17
Depot – Integration into /usr/local
  • Fourth, if there are ‘conflict’ errors in the log file and depot did not successfully complete, add the following to your depot.pref.local file:
  • In this example I am assuming that the 'java' collection is the one that is in conflict with your software install override  mylocal  java

CMU / SCS Computing Facilities

depot integration into usr local18
Depot – Integration into /usr/local
  • Depot finds conflict errors one at a time.
    • Once a conflict is encountered, depot exits
  • So, you have to keep running depot, and adding “overrides” until depot runs without finding conflicts
    • So a depot.pref.local file with multiple overrides would look like:
      • override mylocal java,foo,bar

CMU / SCS Computing Facilities

depot integration into usr local19
Depot – Integration into /usr/local
  • Installing directly in /usr/local
    • This can also be done, but you need to tell depot to ignore the new application
    • By default, depot will remove ANYTHING in /usr/local of which it is unaware

CMU / SCS Computing Facilities

depot integration into usr local20
Depot – Integration into /usr/local
  • To protect files under /usr/local
    • You simply have to add a “specialfile” directive into the depot.pref.local file
  • The “specialfile” directive can protect hierarchies or individual files
    • specialfile lib/foo
    • specialfile lib/firefox-2.0.0.5/plugins/libflashplayer.s

CMU / SCS Computing Facilities

misc applications
Misc Applications
  • Misc Applications are ‘depot-ed’ out to all machines with each depot run.
  • Depot integrates Misc Applications into /usr/local on facilitized Linux Machines
    • Not “all” Misc Applications, some must be requested
    • Ex: httpd, or wwwserv
  • Most Misc collections are maintained by volunteer maintainers
    • When you complain about missing software you are given the opportunity to provide it to the User community

CMU / SCS Computing Facilities

misc applications22
Misc Applications
  • Misc Maintainers receive
    • AFS Disk Space for Source and Binaries
    • Control over releases of their software
      • Alpha, Beta and Omega Environments
    • Ability to build and test new releases
    • Integration of their offering into the /usr/local hierarchy

CMU / SCS Computing Facilities

misc applications23
Misc Applications
  • Misc Applications’ releases are defined by the coll.conf file
    • All Misc Applications have a “common” directory
      • Usually this is an AFS mountpoint to a “common” volume
      • /afs/cs/misc/<APPLICATION-NAME>/common/conf/coll.conf

CMU / SCS Computing Facilities

misc applications coll conf
Misc Applications – coll.conf
  • Coll.conf file tells depot where to find packages.
  • stored in: /afs/cs/misc/<collection/common/conf/coll.conf
  • template in: /afs/cs/misc/tools/src/samples/coll.conf

CMU / SCS Computing Facilities

misc applications coll conf25
Misc Applications – coll.conf
  • Minimum information required:
    • sys_types  all the systems the collection is released for (any versions)
    • alpha, beta, omega and default definitions
      • these should be aliases

CMU / SCS Computing Facilities

misc applications coll conf26
Misc Applications – coll.conf
  • Other information
    • named releases for each version
    • specific sys_type releases.

CMU / SCS Computing Facilities

misc applications coll conf27
Misc Applications – coll.conf
  • Named releases:
    • contents <release-name> <relative path(s) to code>
    • Examples:
      • contents 4.2.4 @sys/4.2.4 common/4.2.4
        • Note: release-name 4.2.4 does NOT have to be same as pathname 4.2.4
      • contents 4.3.2 @sys/4.3.2 common/4.3.2
      • contents empty common/empty

CMU / SCS Computing Facilities

misc applications coll conf28
Misc Applications – coll.conf
  • Using aliases for named releases
    • alias alpha 4.3.2
    • alias beta  4.2.4
    • alias omega empty

CMU / SCS Computing Facilities

misc applications coll conf29
Misc Applications – coll.conf
  • Platform specific overrides
    • Syntax:
      • on <sys_type> alias <alias-name> <release-name or other alias>
    • Example:
      • on amd64_fc3 alias omega 4.2.4

CMU / SCS Computing Facilities

misc applications psmake
Misc Applications – psmake
  • Building misc collections use “psmake”
    • Psmake is a tool developed to efficiently work in the SCS environment
    • psmake allows you to build on multiple platforms from a single afs source tree
  • Facilities provides machines of each sys_type for building misc collections:
    • <sys_type>.sp.cs.cmu.edu

CMU / SCS Computing Facilities

misc applications psmake31
Misc Applications – psmake
  • Building for each platform will simply require:
    • Logging into build machine of the desired sys_type
    • Moving to misc src area
    • Running:
      • psmake init all (initialize and build package)
      • psmake check (test build - if package provides)
      • psmake install (install to afs misc sys_type area) Building misc collections

CMU / SCS Computing Facilities

misc applications psmake32
Misc Applications – psmake
  • Details on using psmake
    • Requires PSMakefile
    • Sample file available from:

/afs/cs.cmu.edu/misc/tools/src/samples/PSMakefile-autoconf

CMU / SCS Computing Facilities

misc applications psmake33
Misc Applications – psmake
  • For new misc collections:
    • Untar the source in the misc source area
    • Copy the sample PSMakefile to the root of the source area as PSMakefile

CMU / SCS Computing Facilities

misc applications psmake34
Misc Applications – psmake
  • Edit the following in the PSMakefile: 1) Comment out second line: ## This is a sample; you need to edit it for it to be useful. die "Read and edit the PSMakefile first!\n"; 2) Set the package (misc collection) name ## Set to the package name as it appears in directory names: $_package = 'sample';

CMU / SCS Computing Facilities

misc applications psmake35
Misc Applications – psmake
  • Edit the following in the PSMakefile (Cont’d): 3). Set the version pattern: ## Uncomment one of the following two lines.  Convention says you should ## use the first variant unless there is more than one package in the same ## collection. # $ReleasePat =~ s/omega/$_version/; # $ReleasePat =~ s/omega/$_package-$_version/; You should always pick 1st pattern unless multiple software packages are under one misc collection – (ex. httpd)

CMU / SCS Computing Facilities

misc applications psmake36
Misc Applications – psmake
  • Edit the following in the PSMakefile (Cont’d): 4) If you need to add ./configure options for ALL sys_types add them here: ## Fill in any additional platform-independent configure options here. @configure_args = ( '--prefix=/usr/local' ); Or just below this section for system dependent arguments.

CMU / SCS Computing Facilities

misc applications psmake37
Misc Applications – psmake
  • Once the PSMake file is configured it can usually be re-used for newer releases.
  • Create a README.cmu file in the misc/<name>/src which includes:   - Where the code came from   - Any problems you encountered in building   - Any special configuration options needed   - Commands you ran to build and install the code

CMU / SCS Computing Facilities

misc applications psmake38
Misc Applications – psmake
  • What to do if code already installs under /usr/local
    • Depot will take the contents of the /afs/cs/misc/<name>/<sys_type>/<version> and put it under    /usr/local
    • The simplest solution is to create a /afs/cs/misc/<name>/<sys_type>/<version>/depot.conf with the   contents: usr/local/bin     bin usr/local/lib     lib usr/local/lib64   lib64 usr/local/man     man usr/local/share   share
    • And so on….

CMU / SCS Computing Facilities