Overview of ESMF in the Community Climate System Model (CCSM)

1. Overview of ESMF in the Community Climate System Model (CCSM) Erik Kluzek NCAR -- CCSM Software Engineering Group (CSEG)

2. Outline • Introduction to CCSM Projects using ESMF • ESMF Stage-1 Code Development • Stage-1 Requirements and Overall Design • Simple pseudo-code examples of Stage-1 design • River Transport Model Project • Data-7 Model Project • ESMF Infrastructure in CCSM • Conclusion Erik Kluzek -- NCAR

3. Introduction to CCSM Projects using ESMF • ESMF Stage-1 Code Development -- Implement a sequential driver using ESMF that replaces stand-alone CAM. • River Transport Model (RTM) Re-Gridding--Use ESMF to make the CCSM river-routing model distributed. • Data-7 Model Project -- Rewrite of CCSM data-models designed with the ability to use ESMF superstructure from the get-go. • ESMF utilities in CCSM -- Plan on using ESMF utilities such as: time-manager, ConfigAtt, LogErr and I/O. Erik Kluzek -- NCAR

4. ESMF Stage-1 Code Development • Goal: Implement a single-executable sequential version of CCSM that replicates the answers from stand-alone CAM. • Process: Developed and approved evaluation plan, project requirements and design documents. Staged implementation, frequent iterative design/review. • Status: Currently modifying CAM toward desired product with staged commits on development version. • Motivation: Initially work from a simpler non-concurrent version of CCSM, that can show replication of answers on different configurations of tasks/threads using a more widely used model than CCSM. Erik Kluzek -- NCAR

5. ESMF Stage-1 Overall Design • New SPMD, sequential CCSM: Replace stand-alone CAM with a single-executable fully sequential implementation of CCSM (which we currently do not have). • Reduce code-duplication: Will reduce code duplication between CAM and CCSM. • Plug and Play: Active, data, and dead components can be interchanged in the system. • Easy to make driver concurrent for Stage-2: After Stage-1 a concurrent system will be developed that only involves changing the hub and driver components. The underlying separable model components won’t change. Erik Kluzek -- NCAR

6. Overview of ESMF Stage-1 Project Requirements • Keep backwards compatibility with both CCSM and all stand-alone CAM configurations. • Hub-spoke design of CCSM used with five components: atmosphere, land, ocean, sea-ice, and hub. • List of fields exchanged -- same as CCSM3. • Amount of model code modified should be kept to the minimum. • Top level driver referred to as Sequential CCSM. • Capability to easily swap different types of components in (active, data or dead). • Confine ESMF to a thin wrapper layer. • Sequential and concurrent mode should share as much code as possible. Erik Kluzek -- NCAR

7. Example of ESMF Layer Erik Kluzek -- NCAR

8. Top Level Sequential CCSM Erik Kluzek -- NCAR

9. Top level Plug and Play Sequential Driver Do while( .not. ESMF_IsStopTime( sync_clock ) ) call ESMF_CplCompRun( cpl=atm2hub_cpl, import=hub_export, export=atm_import ,& clock=sync_clock, phase=1 ) ! Run Atmosphere phase 1 call ESMF_GridCompRun( gc=atm_gc, in=atm_import, out=atm_export, sync_clock, phase=1 ) call ESMF_CplCompRun( cpl=atm2hub_cpl, in=atm_export, out=hub_import , phase=2 & clock=sync_clock ) ! Run hub to prepare land import call ESMF_GridCompRun( gc=hub_gc, in=hub_import, out=hub_export , phase=1, clock=sync_clock ) call ESMF_CplCompRun( cpl=hub2lnd_cpl, in=hub_export out=lnd_import , phase=3,& clock=sync_clock ) ! Run Land call ESMF_GridCompRun( in=lnd_import, out=lnd_export , phase=1, clock=sync_clock ) . . Similarly for ice, and ocean and second phase run-methods . End do Erik Kluzek -- NCAR

10. Plug and Play ESMF Component Module Atm_gridcomp Use cam_comp, only: cam_input_t, cam_output_t, cam_init, cam_run1, cam_run2, cam_final Contains ! Register methods with ESMF Public atm_register ! ! Following are called only through ESMF ! Public :: atm_init, atm_run1, atm_run2, atm_final ! Convert between ESMF import/export states into CAM input/output derived types Private :: convert_import2camin, convert_output2export . End module Erik Kluzek -- NCAR

11. Model Component non-ESMF Layer Module cam_comp Public data types :: cam_input_t, cam_output_t Contains Subroutine cam_alloc( cam_in, cam_out ) Subroutine cam_init( cam_in, cam_out ) Subroutine cam_run1( cam_in, cam_out ) Subroutine cam_run2( cam_in, cam_out ) Subroutine cam_finalize( cam_in, cam_out ) End module cam_comp Erik Kluzek -- NCAR

12. River Transport Model Re-gridding • Prototyping the use of ESMF re-gridding infrastructure for the eventual creation of a high-resolution parallel biogeochemical river transport model. • Found and resolved problems in ESMF for conservative re-grid on widely differing grid scales. Erik Kluzek -- NCAR

13. Data-7 Model Project • Total re-write of CCSM data-model components. • Requirements for utilization of ESMF superstructure (initial, run, final phases) is being done from the ground up. • We will prototype the new system with ESMF superstructure and concurrency in the late fall. Erik Kluzek -- NCAR

14. Use of ESMF Infrastructure in CCSM • Currently use prototype ESMF time-manager in two CCSM components, we will update to ESMF2 in the fall independent of Stage-1 evaluation plan. • CCSM is also interested in the use of ESMF ConfigAtt and LogErr utilities in CCSM. Erik Kluzek -- NCAR

15. Conclusions • CCSM has several active projects utilizing ESMF: Stage-1, RTM, Data-7, and Infrastructure. • The Stage-1 Project is taking considerable time and resources, but will payoff in having a well-designed flexible and extensible system built around ESMF. • The Data-7 project will be able to evaluate ESMF concurrency. • The RTM project has helped to find and resolve issues with ESMF re-gridding. Erik Kluzek -- NCAR