Migration of larc icing products to columbia supercomputer
This presentation is the property of its rightful owner.
Sponsored Links
1 / 12

Migration of LaRC Icing Products to Columbia Supercomputer PowerPoint PPT Presentation


  • 63 Views
  • Uploaded on
  • Presentation posted in: General

Migration of LaRC Icing Products to Columbia Supercomputer. Louis Nguyen, Patrick Minnis, John Murray, and Rabindra Palikonda NASA Langley Research Center Hampton, Virginia, USA. Objectives.

Download Presentation

Migration of LaRC Icing Products to Columbia Supercomputer

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Migration of larc icing products to columbia supercomputer

Migration of LaRC Icing Products to Columbia Supercomputer

Louis Nguyen, Patrick Minnis, John Murray,

and Rabindra Palikonda

NASA Langley Research Center

Hampton, Virginia, USA


Objectives

Objectives

Transition and Integrate the Langley’s Satellite Products to Operations in the operational National Weather Service Aviation Weather Center CIP product

  • Improve processing system to produce reliable icing products

  • Increase temporal and spatial resolution of these products

  • Improve latency time

  • Port and migrate code to Project Columbia Supercomputer

  • Transition research code to operational code


Outline

Outline

  • Overview of LaRC Icing processing environment

  • Project Columbia Supercomputer

  • Migration plan

  • Utilize parallel processing

  • Benefits from improved computer capability

  • Summary and future plans


Overview of larc processing environment

Overview of LaRC Processing Environment

Processing Environment

•Uses the Man computer Interactive Data Access System (McIDAS, developed by SSEC Univ. Wisc)

•McIDAS is a programmable suite of software and interactive data visualization tool used for displaying, analyzing, and acquiring datasets

• Runs on SGI Unix platform

Real-time Data Acquisition System

•GOES-East & West data from via McIDAS ADDE server (SSEC)

• RUC data obtained from SSEC and NOAA Forecast Systems Lab

• Snow maps from NOAA via FTP site

• Transition to Operations, utilize the NOAA NESDIS operational 24/7 ADDE servers to ingest satellite and model data


Overview of larc icing processing con t

Overview of LaRC Icing Processing (con’t)

Source Code

•Code Modularization (R. Palikonda)

-satellite modules for GOES (8-12), MSG, AVHRR, MODIS

- sounding module for RUC, ECMWF, GFS/AVN

- helps isolates sections of the algorithm to improve cloud mask, corr-k, cloud parameterization, etc.

- most modules are written in C

•Main driver program is in McIDAS environment

•Cloud algorithms are written in Fortran

• Data inputs and outputs utilizes McIDAS API

• Approximately 75,000 lines of code


Overview of larc icing processing con t1

Overview of LaRC Icing Processing (con’t)

Current Computer Resources Used for ASAP

•SGI Orgin 3800 with 40 CPUs

•Only 8 CPU is available for Icing (other CPUs are used by CERES & CALIPSO project)

Limitation of Current Computer Resources

•Run code every half hour during the day and hourly at night

•Process GOES-West (10) at 8km res: ~30 mins

•Process GOES-East (12) at 8km res: ~35 mins

Need Additional Resources

•Run at full 4km resolution every 15 mins

•Would take ~70 min to process GOES-West and ~80 min for GOES-East at full resolution

•Project Columbia Supercomputer offers needed resource


Project columbia supercomputer

Project Columbia Supercomputer

  • NASA’s newest supercomputer is one of the world’s fastest

  • Supports the agency’s 4 mission directorates (Aeronautics, Exploration, Science, & Space), NASA Engineering and Safety Center (NESC), and Vision for Space Exploration

  • Support general science community: GSFC runs a General Circulation Model to generate real-time numerical weather prediction (NWP) to improve hurricane tracking and intensity forecast for Hurricane Center

  • Funds for Columbia pre-paid for

  • thru FY05-07 from the 4 mission

  • directorates and NESC

  • Requested 35,000 cpu hrs FY05-06

  • from Aviation Safety Program

  • allotment


Project columbia supercomputer1

Project Columbia Supercomputer

Technical Hardware Description

  • 20 SGI Altix 3700 superclusters, each with 512 processors

  • 10,240 Intel Itanium-2 processors running at 1.5 GHz

  • Capable of 51.9 teraflops, ranked second behind DOE Blue Gene

  • 20 Terabyte of Memory

  • Linix based operation system

  • 40 TB of fiber channel RAID storage

  • Capable of storing 10 petabytes of archive storage


Migration to project columbia

Migration to Project Columbia

  • NASA Advance Supercomputing (NAS) accounts requested

  • McIDAS software installation

    • Over 1000 programs and subroutines

    • Testing the software components

  • Port LaRC Icing code to Columbia platform

    • Data ingest system (satellite, model, input maps)

    • Processing script and aux program package

    • LaRC cloud algorithms

  • Test and check for consistency of Icing products produced from LaRC and Columbia

  • Possible security/firewall issues related to ingest and dissemination

  • of data and products via ADDE and FTP ports


Parallel processing

Parallel Processing

  • GOES-12 & GOES-10 are

  • processed using single CPU

  • at 8km takes 30 mins

  • at 4km takes 80 mins

  • Time includes 5-8 mins for

  • Input and output

GOES-10 Phase

GOES-12 Phase

  • Two parallel processing approach:

  • run sub-domains using multiple CPUs (use same binary)

  • multi-threaded process for looping through each grid box

  • Parallel processing will use 16 CPUs (8 for ea GOES)

  • Columbia CPU are ~2.5x faster


Parallel processing con t

Parallel Processing (con’t)

Full Resolution 80 min process should be reduced to 9-12 mins


Summary and future plans

Summary and Future Plans

  • LaRC Satellite products available near real-time in several formats (McIDAS ADDE server, RUC Gridded netCDF, and LaRC Packed Binary)

  • LaRC Icing code will be adapted to run in parallel processing mode

  • Parallel processing will allow GOES Icing products to run at full resolution every 15 min

  • Porting to Project Columbia Supercomputer to be completed by end of FY05 (ASAP project milestone)

  • Work with the NASA Applied Sciences Program, the FAA and NOAA to integrate and transition the LaRC Icing products to the operational NWS Aviation Weather Center CIP product by end of FY06

This research was support by the NASA Aviation Safety Program.


  • Login