The evolving environment at nccs
1 / 8

The Evolving Environment at NCCS - PowerPoint PPT Presentation

  • Uploaded on

The Evolving Environment at NCCS. Vickie Lynch. Compute Node Linux. Jaguar will be down November 1-5 to change from Catamount to Compute Node Linux (CNL). CNL is a stripped-down version of Linux on the compute nodes.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'The Evolving Environment at NCCS' - glenna

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Compute node linux
Compute Node Linux

  • Jaguar will be down November 1-5 to change from Catamount to Compute Node Linux (CNL).

  • CNL is a stripped-down version of Linux on the compute nodes.

  • Try the system running CNL that is accessible to all jaguar users:

  • Jaguarcnl contains 2,976 dual core XT3 compute nodes each running CNL.

  • Eight additional cabinets will be removed from jaguar and added to jaguarcnl increasing it from 32 cabinets to 40 cabinets (7440 cores).

Changes for compute node linux
Changes for Compute Node Linux

  • Currently only the /tmp/work/$USER lustre directory is visible to compute nodes.

  • There are several important differences between Catamount and CNL, and users are encouraged to view the details of those differences at

  • Due to the differences between CNL and Catamount, users should recompile on the CNL system prior to running.

  • The batch option “-lsize” specifies cores under CNL.

  • aprun replaces yod as the job launcher under CNL.

  • Each user is currently allowed only one job in a running state.

  • The maximum walltime limit is 12 hours.

Cnl batch example
CNL batch example

#PBS -A STF006

#PBS -N test

#PBS -j oe

#PBS -q debug

#PBS -l walltime=0:50:00,size=4

cd /tmp/work/vel/cqlptr

export PGPLOT_FONT=/tmp/work/vel/pgplot_xt3/grfont.dat

aprun -n4 ./cqlp

Pathscale compiler
Pathscale compiler

  • The pathscale compiler is now available on jaguar and jaguarcnl.

  • Before compiling:

    • module swap PrgEnv-pgi PrgEnv-pathscale

  • Use ftn –O3 instead of ftn –fast for compiler optimization

  • Use -Wl,--allow-multiple-definition for the ldr flag if you get too many warnings about multiple definitions

  • Use a SuperLU library compiled with pathscale:

    • /spin/home/vel/lib/SuperLU_DIST/libsuperlu.a

Quad core upgrade
Quad-core Upgrade

  • Jaguar shutdown for upgrade: December 4, 2007- ?

  • During this time users will have access to the remaining 40 XT cabinet transition system (7440 cores).

  • Jaguar will be upgraded to 250 teraflops.

  • The dual-core processors will be replaced with quad-core processors.

  • Memory will be doubled.

  • This transition will involve significant down time.

  • Consider using OpenMP across the cores of a processor

Petaflop system
Petaflop system

  • A petaflop system will be installed at the NCCS in 2008.

  • The LCF Petaflop system will also be quad-core.

  • It will come with a new interconnect that should enable better scalability than the current XT4.

  • This new interconnect is called Gemini and it will allow you to effectively perform Co-Array Fortran, UPC and Global Arrays.