coupler design issues from the modular ocean data assimilation project n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Coupler Design Issues from the Modular Ocean Data Assimilation Project PowerPoint Presentation
Download Presentation
Coupler Design Issues from the Modular Ocean Data Assimilation Project

Loading in 2 Seconds...

play fullscreen
1 / 30

Coupler Design Issues from the Modular Ocean Data Assimilation Project - PowerPoint PPT Presentation


  • 97 Views
  • Uploaded on

Coupler Design Issues from the Modular Ocean Data Assimilation Project. Dr. Richard Loft Computational Science Section Scientific Computing Division National Center for Atmospheric Research Boulder, CO USA. Outline. Project Introduction goals / pariticipants / scope http//:iom.asu.edu

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Coupler Design Issues from the Modular Ocean Data Assimilation Project' - dea


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
coupler design issues from the modular ocean data assimilation project
Coupler Design Issues from the Modular Ocean Data Assimilation Project

Dr. Richard Loft

Computational Science Section

Scientific Computing Division

National Center for Atmospheric Research

Boulder, CO USA

outline
Outline
  • Project Introduction
    • goals / pariticipants / scope
    • http//:iom.asu.edu
  • IOM: weak variational data assimilation
    • Inputs/outputs
    • Data components
    • Functional components
  • IOM Coupling design
    • Strategy
    • Details
    • Role of Domain Specific Languages (DSL)
modular ocean data assimilation
Modular Ocean Data Assimilation
  • NSF ITR/AP
  • Focus on the OSU Inverse Ocean Model(IOM) system for ocean data assimilation.
    • Variational data assimilation system
    • Weak assimilation
    • Iterative algorithm for solving nonlinear assimilation problems
    • Suite of diagnostics (posterior error statistics)
    • 20 years in development by Bennett, et al.
objectives of moda project
Objectives of MODA Project
  • Enhance the IOM System with modern information technology.
    • Modular software design
    • Hybrid parallel implementation
    • Coupled model strategy
    • Automated code generation
  • Distribute it to the ocean modeling community.
    • 5 ocean modeling partners
  • Facilitate application of the system to coastal oceans, ocean basins and the global ocean.
nsf itr moda collaboration
NSF ITR MODA Collaboration

Institution

Researcher

Arizona State U. Muccino

U. Colorado Moore

NCAR Loft

NCSA Baker

U. North Carolina Leuttich

Oregon State U Bennett, Egbert, Erwig

Rutgers U. Arango, Haidvogel

UCSD ( Scripps) Cornuelle, Miller

iom team members
IOM Team Members
  • IOM-PEZ Parallelism
    • PI: Rich Loft, NCAR
    • Component: parallel super and Infra-structure

(hybrid F90 module framework)

  • Domain Specific Languages
    • PIs: Martin Erwig & Zhu Fu, OSU
    • Component: automated code generator (Haskell)
  • Visualization
    • Pis: Polly Baker, NCSA/IU
    • Component: VisBenchToolVisualization Software
participating model teams
Participating Model Teams
  • PEZ (Primitive Eq. Z-coordinate)model
    • Description: 3D, free-surface, z-coordinate
      • Bryan-Cox
    • Grid: spherical polar, “B” grid
    • Language: F90
    • Parallelism: SPMD, MPI
  • ROMS (Regional Ocean Model System)
    • Description: 3D, free-surface, S-coordinate
    • Grid: Horizontal orthogonal curvilinear coordinate, “C” grid
    • Language: F90
    • Parallelism: SPMD, OpenMP, MPI, SMS
participating model teams1
Participating Model Teams
  • SEOM (Spectral Element Ocean Model)
    • PI: Dale Haidvogel, Rutgers
    • Description: 3D, free-surface, S-coordinate
    • Grid: h-p finite element, quadrilateral element
    • Language: F90
    • Parallelism: SPMD, MPI
  • ADCIRC (Advanced Circulation) model
    • PIs: Julia Muccino, ASU; Rich Luettich, UNC
    • Description: 3D, free-surface, sigma-coordinate
    • Grid: finite element, linear triangular element Language: F90
    • Parallelism: SPMD, MPI
participating model teams2
Participating Model Teams
  • OTIS (Internal Tides)
    • PI: Gary Egbert, OSU
    • Description: Laplace tidal equations plus 10 years of TOPEX/Poseidon
    • Description: Solid Earth mageto-tellurics (magnetic field of earth’s crust)
data assimilation checklist inputs
Data Assimilation Checklist: Inputs
  • observing system
    • e.g. measured sea surface temperatures, isotherm depths and surface winds
  • dynamics
    • e.g. the hydrostatic primitive equations
  • hypothesis concerning the error
    • covariances of errors in the
      • initial conditions
      • boundary conditions
      • and forcing
  • estimator
    • space-time integrated weighted sum of squared errors
  • optimization algorithm
    • iterative indirect representer algorithm
data assimilation checklist outputs
Data Assimilation Checklist: Outputs
  • state estimate
    • hindcast over the period in the data collection
    • space-time fields for state variables
  • data residuals, dynamic residuals
    • Space-time minimum residuals
  • posterior error statistics
    • space-time covariances
  • test statistics
    • chi square internal consistency tests
  • model improvements
    • e.g. suggested from the distribution of errors
  • observing system assessment
inverse system data components
Inverse system data components…
  • Vector of data to be assimilated
  • Trajectories
    • multi-variate physical space-time fields
    • generated by tangent linear/adjoint model
    • user prescribed
  • Parameters
    • covariance matrices
    • user prescribed
inverse system functional components
Inverse system functional components…
  • Pure data space components
    • iterative solver
  • Physical space components
    • tangent linear/ adjoint models
  • Components that map between the two spaces.
    • measurement operator
    • impulse operator
  • Looks like a coupled system
coupler macro design iom coupler ocean models
Coupler macro design: IOM + coupler + ocean models

Fwd & adjoint

Ocean model

IOM

grids

State

Data

Data space

Physical space

  • Ocean Models:
    • PEZ
    • ROMS
    • SEOM
    • ADCIRC
    • OTIS
  • Data Assimilations:
    • IOM core

Coupler

  • Both spaces
  • Impulses / measurements:
    • DSL autogeneration
    • Control info
data physical space interactions
Data / Physical Space Interactions

Interpolations

IOM

PEs

model

PEs

IOM

Iterative Solve

(data space)

Ocean Model

Time Integration

(physical space)

iom system pseudo code
IOM System Pseudo-code

d = data

uF = meas. first guess

h = misfit

 = representer coefficients

Key:

IOM(){

read d

first( ; uF)

h = d - uF

solve(h ; )

final(  ; )

}

^

^

^

Physical Space / user supplied

Both / Auto-generated

Data Space/IOM supplied

Subroutine Notation: foo( in ; out )

Pure Data Space…

iom first guess code
IOM First Guess Code

first(;uF){

read F

tanlin( F(x,t) ; UF(x,t) )

measure(UF(x,t) ; uF)

}

F(x,t) = Initial / boundary / interior forcing

UF(x,t) = response to F

uF = measured first guess

iom inner solver code
IOM Inner Solver Code

^

(x,t) = adjoint

(x,t) = forward

D(x,t) = Dirac comb

 = representer coefs

 = measured 

 = scratch

 = scratch

solve(h ;  ){

= h

while (  ≠ ){

comb(  ; D(x,t))

adjoint( D(x,t) ; (x,t) )

convolve( (x,t) ; (x,t))

tanlin( (x,t) ; (x,t) )

measure((x,t) ; )

stabilize( ,  ; )

call precongrad(  ; )

}

 = 

}

^

^

iom final sweep
IOM Final Sweep

^

final(  ; ){

comb(  ; D(x,t))

adjoint( D(x,t) ; (x,t) )

convolve((x,t) ; (x,t))

tanlin((x,t) + F(x,t); U(x,t) )

write U(x,t)

}

D(x,t) = Dirac comb

U(x,t) = optimal estimate

(x,t) = optimal adjoint

^

^

^

^

iom system architecture
IOM System Architecture

IOM Coupling Layer

IOM Solver

DSL

Generated Code

Tangent

Linear

Model

Adjoint

Model

Parallel Infrastructure

External libraries: MPI, NetCDF, …

iom parallel module support
IOM Parallel Module Support

F90 Module Heirarchy:

PEZ

9pt stencil

Solver

global sums

stencils

parallel

IOM

broadcast

buffers

process

thread

virtual

topology

iom coupling design details
IOM Coupling Design Details
  • Key object: the observation
    • Simplest case: a point observation
    • is a distribution:  (-', -', t-t' )
    • In general smeared out over space-time.
  • The observation (self describing)
    • F90 derived type
    • type of observation
    • units
  • Associated methods must be supplied for mapping observations onto gridded state variables.
iom coupling design details1
IOM Coupling Design Details
  • Fortran/Unix specific (applications / code generator)
  • Preferred mode: separate executables.
    • IOM + coupler + ocean models (constant functions)
    • IOM-PEZ components currently merged into one executable
  • IOM supports parallel components (MPI/OMP/hybrid)
  • Fixed number of processors
    • MPI_SPAWN and MPI_SPAWN_MULTIPLE not supported on many major platforms (e.g. IBM) ).
  • Coupled component execution does not overlap.
    • Strategy adopted to enable easy interfacing with diverse ocean codes, with different internal forms of parallelism.
  • Solver checkpoint/restarts IOM system, no internal state saved on TLM/adjoint model side.
automatic generation of model specific iom tools
Automatic Generation of Model-Specific IOM Tools

Martin Erwin and Zhe FuOregon State University

Idea:

  • Specify ocean modeling tools once
  • Identify model-dependent parameters
  • Generate Fortran programs from specification and values for parameters
  • Ocean modelers:
  • select simulation tools
  • provide parameters for their models
  • obtain a customized variational system
iom dsl system

p1 = ......pk = ...

p1 = ......pk = ...

module tool_PEZ ...

module tool_ROMS ...

IOM DSL System

Tool Specification

ModelConfigurations

p1 = ......pk = ...

tool(p1,...,pk):obj[n] = ......

PEZ

ROMS

Compiler

Fortran Code

iom dsl modeler interactions

IOM

Developers

Computer

Scientists

Ocean

Modelers

IOM-DSL-Modeler Interactions

Configuration

DSL

p1 = ......pk = ...

tool(p1,...,pk):obj[n] = ......

DSL-Compiler

Fortran