The evolving environment at nccs
Sponsored Links
This presentation is the property of its rightful owner.
1 / 8

The Evolving Environment at NCCS PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

The Evolving Environment at NCCS. Vickie Lynch. Compute Node Linux. Jaguar will be down November 1-5 to change from Catamount to Compute Node Linux (CNL). CNL is a stripped-down version of Linux on the compute nodes.

Download Presentation

The Evolving Environment at NCCS

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

The Evolving Environment at NCCS

Vickie Lynch

Compute Node Linux

  • Jaguar will be down November 1-5 to change from Catamount to Compute Node Linux (CNL).

  • CNL is a stripped-down version of Linux on the compute nodes.

  • Try the system running CNL that is accessible to all jaguar users:

  • Jaguarcnl contains 2,976 dual core XT3 compute nodes each running CNL.

  • Eight additional cabinets will be removed from jaguar and added to jaguarcnl increasing it from 32 cabinets to 40 cabinets (7440 cores).

Changes for Compute Node Linux

  • Currently only the /tmp/work/$USER lustre directory is visible to compute nodes.

  • There are several important differences between Catamount and CNL, and users are encouraged to view the details of those differences at

  • Due to the differences between CNL and Catamount, users should recompile on the CNL system prior to running.

  • The batch option “-lsize” specifies cores under CNL.

  • aprun replaces yod as the job launcher under CNL.

  • Each user is currently allowed only one job in a running state.

  • The maximum walltime limit is 12 hours.

CNL batch example

#PBS -A STF006

#PBS -N test

#PBS -j oe

#PBS -q debug

#PBS -l walltime=0:50:00,size=4

cd /tmp/work/vel/cqlptr

export PGPLOT_FONT=/tmp/work/vel/pgplot_xt3/grfont.dat

aprun -n4 ./cqlp

Pathscale compiler

  • The pathscale compiler is now available on jaguar and jaguarcnl.

  • Before compiling:

    • module swap PrgEnv-pgi PrgEnv-pathscale

  • Use ftn –O3 instead of ftn –fast for compiler optimization

  • Use -Wl,--allow-multiple-definition for the ldr flag if you get too many warnings about multiple definitions

  • Use a SuperLU library compiled with pathscale:

    • /spin/home/vel/lib/SuperLU_DIST/libsuperlu.a

Quad-core Upgrade

  • Jaguar shutdown for upgrade: December 4, 2007- ?

  • During this time users will have access to the remaining 40 XT cabinet transition system (7440 cores).

  • Jaguar will be upgraded to 250 teraflops.

  • The dual-core processors will be replaced with quad-core processors.

  • Memory will be doubled.

  • This transition will involve significant down time.

  • Consider using OpenMP across the cores of a processor

Petaflop system

  • A petaflop system will be installed at the NCCS in 2008.

  • The LCF Petaflop system will also be quad-core.

  • It will come with a new interconnect that should enable better scalability than the current XT4.

  • This new interconnect is called Gemini and it will allow you to effectively perform Co-Array Fortran, UPC and Global Arrays.


  • Login