1 / 131

Earth System Modeling Framework

Earth System Modeling Framework. ESMF Introductory Tutorial. ESMF v 3.0.1 http://www.esmf.ucar.edu. Introduction and Orientation. http://www.esmf.ucar.edu. Outline. Administrivia Background Building ESMF HPC terminology How to create an ESMF component Coupling

ifama
Download Presentation

Earth System Modeling Framework

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Earth System Modeling Framework ESMF Introductory Tutorial ESMF v 3.0.1 http://www.esmf.ucar.edu

  2. Introduction and Orientation http://www.esmf.ucar.edu

  3. Outline • Administrivia • Background • Building ESMF • HPC terminology • How to create an ESMF component • Coupling • Setting up a parallel environment • ESMF infrastructure: Data structures • ESMF infrastructure: Utilities

  4. Rough Timeline • [8:30-9:00] Introduction and orientation • [9:00-10:30] Building and linking exercise • [10:30-10:45] Break • [10:45-noon] Terminology, superstructure, coupling exercise • [noon-1 pm] Lunch on your own • [1:00-2:20] Parallel environment, data structures, methods, and code review • [2:20-2:35] Break • [2:35-3:00] Utilities

  5. Outline • Administrivia • Background • Building ESMF • HPC terminology • How to create an ESMF component • Coupling • Setting up a parallel environment • ESMF infrastructure: Data structures • ESMF infrastructure: Utilities

  6. Administrivia • Bathrooms down the hall • Cafeteria in next building over • Refreshments in the hall • No stupid questions, ask anytime • You can tell us to speed up or slow down • During exercises let us know if you are having problems or don’t understand

  7. Outline • Administrivia • Background • Building ESMF • HPC terminology • How to create an ESMF component • Coupling • Setting up a parallel environment • ESMF infrastructure: Data structures • ESMF infrastructure: Utilities

  8. What is ESMF? • What is a framework? • Framework vs. library • A toolbox for modelers • Turn model codes into components • Couple model components • Provide code infrastructure • Time management • Log management • Etc.

  9. Superstructure vs. Infrastructure • Superstructure: The means by which models are converted into components and coupled • Infrastructure: Tools that modelers can use within their own code • Data structures: Array, Field, etc. • Utilities: TimeMgr, LogErr, etc. • These are independent, but work together

  10. ESMF Structure GridComp Land, ocean, atm, … model CplComp Xfers between GridComps State Data imported or exported Superstructure Infrastructure Regrid Computes interp weights Bundle Collection of fields Field Physical field, e.g. pressure Grid LogRect, Unstruct, etc. PhysGrid Math description DistGrid Grid decomposition F90 Array Hybrid F90/C++ arrays DELayout Communications Route Stores comm paths C++ Utilities Virtual Machine, TimeMgr, LogErr, IO, ConfigAttr, Base etc. Communications Data

  11. What’s New in ESMF 3 • Simplified build and link • Public DistGrid - stores grid topology • New Array Class – extensions to native arrays • Regridding w/ ESMF_ArraySparseMatMul() • User provided interpolation weights • Parallel sparse matrix multiply

  12. Support • Mailing lists to join: • esmf_jst@ucar.edu: Technical discussion • esmf_info@ucar.edu: Quarterly newsletter • esmf_community@ucar.edu: Community announcements • Location to join: http://www.esmf.ucar.edu > Users • Support: • Send email to: esmf_support@ucar.edu

  13. Web • Main site: http://www.esmf.ucar.edu

  14. Reference Manual • Go to http://www.esmf.ucar.edu/ • Click Download tab • Click View All Releases link (left nav bar) • Scroll down to v3_0_1 • Click on the HTML version of the Reference Manual in the table

  15. Downloading • Go to http://www.esmf.ucar.edu/ • Click Download tab • Latest public release here • Click View All Releases link (left nav bar) • Scroll down to v3.0.1 • Click to download and accept license • Alternate: Go to http://cdp.ucar.edu • Must register and join ESMF group • All releases are available • Source is a gzipped tar file - must be unpacked

  16. Feature Prioritization • Issues and feature requests arrive via esmf_support@ucar.edu • SourceForge tracker ticket is created • CRB reviews issues and establishes release schedule: • http://www.esmf.ucar.edu/plans/releases/index.shtml • Prioritization can be influenced by participating in JST telecons

  17. Future Development • New Grid development • Support for structured grids • Support for unstructured meshes. • Automated regridding • C framework interfaces

  18. Questions?

  19. Outline • Administrivia • Background • Building ESMF • HPC terminology • How to create an ESMF component • Coupling • Setting up a parallel environment • ESMF infrastructure: Data structures • ESMF infrastructure: Utilities

  20. Building ESMFHands-On

  21. Section Outline • Requirements to build • Set environment variables • Build library • Common build problems • Directory structure • Build and run tests

  22. Requirements to Build • Fortran 90 compiler (g95, ifort, …) • C++ compiler (gcc, icc, …) • MPI libraries (MPICH, LAMMPI, ...) • gnu’s make gmake (not make) • prompt$ make -v • indicates type • Perl (for running test scripts)

  23. Environment Variables: Review • Typical values for Linux machines • ESMF_DIR: Top level ESMF directory path • ESMF_BOPT: ‘g’ for debug, ‘O{0,1,2,…}’ optimize • ESMF_COMM: mpich, mpich2, lam, openmpi, intelmpi, or mpiuni for serial mode. • ESMF_COMPILER: intel, xlf, absoft,lahey, pgi, pathscale, g95 • ESMF_ABI: (Application binary interface) System pointer word size/memory model; 32 or 64 • See the README file and “Environment Variables” in the “Quick Start” section of the User’s Guide for more info

  24. Required Environment Variables • Environment variables must be set prior to building the source. • export ESMF_DIR=absolute_path • export ESMF_BOPT=g • export ESMF_COMM=mpich2 • export ESMF_COMPILER=g95 • Check the settings with • make info

  25. Building the Framework • Download the tarball (already done for tutorial) • Unpack and build the framework • prompt$ gunzip esmf_3_0_1_src.tar.gz • prompt$ tar -xvf esmf_3_0_1_src.tar • Set environment variables • prompt$ more set* • prompt$ export ESMF_DIR=absolute_path • prompt$ export ESMF_BOPT=g • prompt$ export ESMF_COMM=mpich2 • prompt$ export ESMF_COMPILER=g95 • Build: prompt$ cd ~/esmf; make

  26. Review Build Commands • Clobber any previous build • make clobber • Build the framework • make • Display build settings • make info • NOTE about gnumake: On many Linux systems gnumake or gmake is renamed make. Typically this is not the case on non-Linux platforms.

  27. Common Build Problems • Old versions of required software • gcc • gnumake • Environment variable settings • Library paths • 32 bit machines set to 64 bit • Software install • Compiler build options • MPI build options

  28. Modify Build Files • prompt$ cd ~/esmf/build_config • prompt$ ls • Linux.absoftintel.default/ • Linux.pgigcc.default/ • Linux.g95.default/ • Linux.gfortran.default/ • Linux.xlf.default/ • Linux.intel.default/ • See the “Porting to New Platforms” section of the User’s Guide

  29. Directory Structure (1 of 2) • Change to the framework directory • prompt$ cd ~/esmf/ • README: Overview ESMF of build & run • application/: Example of a standalone application code • build/: Make helper file common.mk • build_config/: Config file for building on different platforms • lib/: Location of ESMF library (esmf.mk & libesmf.a)

  30. Directory Structure (2 of 2) • makefile: Top level makefile for building and running • mod/: Shared modules • quick_start/: Toy example of a coupled model • src/: Source code • demo/ • Infrastructure/ • Superstructure/ • system_tests/

  31. Starting MPI • Required to run • Some flavors of MPI, such as LAM & MPICH2, require starting an external process manager before running programs. • mpdboot MPICH2 • Run the something here!!!!! • run_something • Halt MPI • mpdallexit MPICH2

  32. Verify the Build • Build the unit tests • prompt$ make build_unit_tests • Run the unit tests • prompt$ mpdboot • prompt$ make run_unit_tests

  33. What Did We Learn? • Environment variables are key • gnumake info can helpful in debugging • The library build takes time • Where to find things in the source directories • Running the tests can confirm the build

  34. Exercise: Installing ESMF and Building and Linking to a Stand-Alone Application

  35. ESMF Installation • Extracts the minimum library and helper files needed to link any stand-alone application to the framework • Allows these files to be placed in a public space • Eliminates need for keeping the entire source tree

  36. ESMF Installation • prompt$ make info • Examine the installation environment variables • Set installation environment variables • ESMF_INSTALL_PREFIX=~/ESMF301 • ESMF_INSTALL_MODDIR=mod • ESMF_INSTALL_LIBDIR=lib • prompt$ more stand* • prompt$ make install • prompt$ make installcheck • See the “Building and Installing” section of the User’s Guide for more info

  37. The Stand-Alone Code • prompt$ mkdir ~/exercise • prompt$ chmod 755 -R ~/esmf • prompt$ cd ~/esmf/application • prompt$ ls • Makefile • esmf_application.f90 • prompt$ cp * ~/exercise • prompt$ cd ~/exercise • prompt$ more Makefile

  38. Setting up the Makefile(1 of 3) • Step 1: Completing the Makefile path • The Makefile requires a path to an esmf.mk • There can be multiple versions if you have built with different flags • Different versions represent different build choices • Debug or Optimize • Compiler choice • MPI choice • Step 2: • prompt$ cd ~/esmf/lib/ • prompt$ ls • Explore the directories created by your build choices

  39. Setting up the Makefile(2 of 3) • Step 3: Choose the path to esmf.mk consistent with the desired system configuration • Example path: • ~/esmf/lib/libg/Darwin.absoft.32.lam.default/ • libg => set up in debug mode • Darwin => operating system • Absoft => compiler • 32 => ESMF_ABI • lam => MPI flavor • What is your path?

  40. Setting up the Makefile(3 of 3) • Step 4: prompt$cd ~/ESMF301/lib • Step 5: prompt$ ls • Notice the alternative path to esmf.mk • Save this path • Step 6: prompt$ cd ~/exercise/ • Step 7: Open the Makefile in an editor • Replace the dummy path with the captured path, to end of the variable ESMF_INSTALL_LIBDIR_ABSPATH on line 14 • Step 8: Save and exit

  41. Building & Running the Application • Build • prompt$ cd ~/exercise • prompt$ make • Running • prompt$ mpdboot • prompt$ mpirun -np 4 esmf_application • prompt$ mpdallexit

  42. What Did We Learn? • All we need to link to the ESMF library with a stand-alone application is the appropriate esmf.mk file • We don’t need the environment variables to build and link to the the stand-alone application once the framework is built.

  43. Questions?

  44. Outline • Administrivia • Background • Building ESMF • HPC terminology • How to create an ESMF component • Coupling • Setting up a parallel environment • ESMF infrastructure: Data structures • ESMF infrastructure: Utilities

  45. HPC Terminology

  46. Serial vs. Parallel • Future computing platforms will consist of multiple processors • Often don’t share memory • Can have multiple threads of execution per processor • MPI & OpenMP are the standards • Programs can run in a serial fashion, w/ one thread or, • Parallel using multiple threads

  47. PETs: Persistent Execution Thread • We call threads PETs • In this introductory tutorial, we’ll stick with one PET per processor • Serial applications run on one PET, parallel applications run on multiple PETs

  48. Decomposition Elements (DEs) • A data decomposition is represented as a set of Decomposition Elements (DEs) • In many applications there is one DE per PET

  49. Simple Decomposition Illustration

  50. Sequential Execution • Components run one after another on the same set of PETs

More Related