slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Plan PowerPoint Presentation
Download Presentation
Plan

Loading in 2 Seconds...

play fullscreen
1 / 33

Plan - PowerPoint PPT Presentation


  • 160 Views
  • Uploaded on

Plan. Code coupling How CERFACS came into play OASIS: Historical development and scientific use at CERFACS An example of active collaborations Community today PRISM The OASIS4 coupler PALM: Contexte, principe, utilisation L’interface graphique pre_PALM

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Plan' - dewei


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide2

Plan

  • Code coupling
  • How CERFACS came into play
    • OASIS:
    • Historical development and scientific use at CERFACS
    • An example of active collaborations
    • Community today
    • PRISM
    • The OASIS4 coupler
  • PALM:
    • Contexte, principe, utilisation
    • L’interface graphique pre_PALM
    • L’analyse de performance de pre_PALM
    • Le développement, les utilisateurs
  • OASIS et PALM: en résumé
  • En conclusions
code coupling

What:

    • Manage execution of the codes
    • Ensure synchronised data exchange between the codes
  • How:
    • 1- Merge the codes:
    • Efficient, but…
    • Integration problem (common, units, …)
    • Memory problem
    • Low flexibility

prog1 & prog2

Program prog1

call sub_prog2(in, out)

end

Program prog2

Subroutine sub_prog2(in,out)

end

Prog1

Prog2

données

Code coupling
  • Why:
    • Treat a global system (ocean-atmosphere, fluid-structure, etc.)
    • Create modular applications by coupling elementary component
code coupling1

How:

    • 2- Use communication protocol:
    • PVM, MPI, file exchange
    • More or less efficient
    • Non generic
    • Non flexible
    • Complex to implement

Prog1

Prog2

Program prog1

if ( )

call x_send(data, tag, target)

endif

end

Program prog2

if ( )

call x_recv(data, tag, source)

endif

end

  • 3- Use a coupler:
  • Maybe less efficient than 1-
  • User external configuration
  • Generic, flexible
  • Portable
  • Interpolation by coupler
  • Redistribution by coupler

PALM/OASIS

Prog1

Prog2

Program prog1

call palm/prism_put(data, time)

end

Program prog2

call palm/prism(data, time)

end

Code coupling
how cerfacs came into play oasis

Models at relatively low resolution (~10000-20000 pts)

  • Small number of 2D coupling fields (~10)
  • Low coupling frequency (~once/day)
How CERFACS came into play: OASIS
  • The need in 1991:
  • To build a French ocean-atmosphere coupled model based on existing GCMs developed independently.
  • Allow component model intercomparison.
how cerfacs came into play oasis1

The answer :OASISOceanAtmosphereSeaIceSoil

O

A

O

A

OASIS

O

A

O

Static coupling: all coupling parameters are fixed initially by the user in an input file namcouple

Flexibility, modularity, (efficiency)

How CERFACS came into play: OASIS
  • 1 monoprocess coupler:
  • synchronisationof component models
  • exchangeof coupling fields
  • interpolationof coupling fields
  • Arbitrary number of (parallel) models and coupling fields
  • Different coupling frequencies for the different fields
  • Particular transformations for each field
slide7

OASIS: Historical development

and scientific use at CERFACS

  • 1993: OASIS1.0 (CRAY pipes and files) 10 year tropical coupling
  • 1995: OASIS2.0 (PVM3) CATHODE: distributed coupling
  • 1996: OASIS2.1 SIDDACLICH (EU): global warming scenarios
  • 1998: OASIS2.2 (SIPC) DEMETER (EU): seasonal forecasting
  • 1999: OASIS2.3 (GMEM - NEC) SINTEX (EU)
  • 2000: OASIS2.4 (MPI2) PREDICATE (EU) : climate natural variability
  • 2004: OASIS3 (MPI1 + PSMILe + I/O) PRISM + DYNAMITE (EU)
  • 2005: OASIS4 (full parallelisation) PRISM + GEMS (EU)
slide8

MPI-M

MPI-M&D

NEC-CCRLE

NEC-HPCE

RPN

SGI

SMHI

UKMO

CGAM

CRAY

ECMWF

EDF

FECIT/Fujitsu

IDRIS

IPSL

Météo-France

OASIS: An example of active collaborations

slide9

OASIS community today

  • CERFACS (France) ARPEGE3 - ORCA2LIM ARPEGE3 - OPA 8.1 ARPEGE3 - OPAICE
  • METEO-FRANCE (France)ARPEGE4 - ORCA2 ARPEGE medias -OPAmed ARPEGE3 - OPA8.1 ARPEGE2 - OPA TDH
  • IPSL- LODYC, LMD, LSCE (France) LMDz - ORCA2LIM LMDz - ORCA4 LMDz - OPA ATL3/ATL1 IFS - OPA 8.1 ECHAM4 - ORCA2
  • MERCATOR (France) PAM(OPA)
  • MPI - M&D (Germany) ECHAM5 - MPI-OM ECHAM5 - C-HOPE PUMA - C-HOPE EMAD - E-HOPE ECHAM5 - E-HOPE ECHAM4 - E-HOPE
  • ECMWF(UK)IFS Cy23r4 - E-HOPE IFS Cy15r8 - E-HOPE
      • IFS - CTM
slide10

OASIS community today

  • CGAM-Reading (UK) HadAM3 - ORCA2
  • Southampton University (UK) Inter. Atm - OCCAM lite
  • UCL (Belgium) LMDz - CLIO
  • SMHI (Sweden) ECHAM - RCA RCA(region.) – RCO(region.)
  • U. of Bergen (Norway) MM5 - ROMS
  • KNMI (Netherlands) ECHAM5 - MPI-OM
  • DMI (Danemark) ECHAM - HIRLAM
  • INGV (Italy) ECHAM5 – MPI-OM
  • IRI (USA) ECHAM4 - MOM3
  • JAMSTEC (Japan) ECHAM4 - OPA 8.2
  • BMRC (Australia) BAM - MOM4 BAM3 - ACOM2
  • U. of Tasmania (Australia) Data Atm. - MOM4
  • RPN-Environment Canada (Canada) MEC - GOM
slide11

OASIS development today: PRISM

OASIS3 and OASIS4 currently developed within PRISM (PRogram for Integrated Earth System Modelling)

  • PRISM is organised under ENES (European Network for Earth System modelling):
    • «Think tank » to organize Earth System modelling in Europe
    • 50 members so far; open to any institute/industry (MoU)
  • PRISM was a European project
    • 22 partners funded in 2001-2004 for 4.8 M€
    • Coordinators: Guy Brasseur (MPI, Hamburg) and Gerbrand Komen (KNMI, DeBilt); Director: Reinhard Budich (MPI)
    • Provide a standard software infrastructurefor Earth system modelling based on state-of-art European components models
slide12

PRISM continues with the PRISM Support Initiative

  • 12 partners
  • own manpower for a total of about 9 py/y
  • support and develop the PRISM software (2005-2008):

- MPI/M&D - UK Met Office

- CNRS - CERFACS

- ECMWF - SMHI

- CGAM - CCRLE

- CRAY - NEC-HPCE

- SGI - SUN

OASIS development today: PRISM

slide13

Need to optimise and parallelise the coupler

The OASIS4 coupler

  • Situation today:
    • Higher resolution, parallel and scalable models
    • Higher coupling frequencies desirable
    • Higher number of models and (3D) coupling fields
    • Massively parallel platforms
  • OASIS4 is composed of:
    • a Driver
    • a Transformer
    • a new PRISM System Model Interface Library (PSMILe)
slide14

OASIS4: Some key notes

  • Developers: CERFACS; NEC CCRLE; SGI; NEC HPCE
  • Public domain; open source license (LGPL)
  • Programming language: Fortran 90 and C
  • Public domain libraries; vendor versions may exist:
    • MPI1 and/or MPI2; NetCDF/parallel NetCDF; libXML
    • mpp_io; SCRIP
slide15

OASIS4: the Driver

  • Manages a static coupling:
    • launches models at the beginning of the run (MPI2) or
    • models started directly in the run script (MPI1)
  • Participates to communicator set-up
  • Reads the XML user-defined configuration info and distributes it to the component PSMILes
slide16

C

C

C

C

C

C

OB

OB

O2

O1

OB

OB

C

C

O2

O1

OB

OB

C

C

T

T

  • Different grid and decomposition
  • via parallel Transformer:
  • Same grid, different decomposition
  • direct repartitioning:

OASIS4: the communication (1/2)

  • Model interface library: PSMILe based on MPI
  • Parallel communication including repartitioning:
    • based on geographical description of the partitions
    • parallel calculation of communication patterns in source PSMILe
slide17

file

A

file1

A

A

A

file

A

A

file2

A

A

A

file3

single

parallel

distributed

OASIS4: the communication (2/2)

  • end-point communication (prism_put, prism_get)
  • parallel 3D search in each source process PSMILe
  • parallel multigrid search algorithm: high efficiency
  • extraction of useful part of source field only
  • one-to-one, one-to-many
  • parallel I/O : GFDL mpp_io, parNetCDF
slide18

For grids:

  • Regular in lon, lat, vert: lon(i), lat(j), height(k)
  • Irregular in lon and lat, regular in vert: lon(i,j), lat(i,j), height(k)
  • Irregular in lon, lat, and vert: lon(i,j,k), lat(i,j,k), height(i,j,k)
  • Gaussian Reduced in lon and lat, reg in vert: lon(npthor), lat(npthor), height(k)

OASIS4: the field transformations

  • source time transformations (below prism_put):
    • average, accumulation
  • target time transformations (below prism_get):
    • time interpolation (for I/O only)
  • statistics
  • local transformations:
    • addition/multiplication by scalar
  • interpolation/regridding (3D):
    • nearest-neighbour, bilinear, bicubic, trilinear
slide19

OASIS4: the users

  • OASIS4 was tested and run with toy models on:
    • Intel Pentium 4 Workstation Cluster,
    • SGI O3000/2000,
    • SGI IA64 Linux server Altix 3000,
    • NEC SX6,
    • AMD 2800 Cluster,
    • IBM Power 4
  • OASIS4 starts being used in real applications:
    • GFDL: MOM4-toy atmosphere
    • SMHI: regional coupling
    • IFM-GEOMAR (Kiel): interpolation of high-resolution fields.
    • GEMS: operational global monitoring of atm chemistry and dynamics
slide20

MERCATOR:océanographie opérationnelle (1997)

Spatial

observations

Model

Forecast

In-situ

observations

PALM: le contexte

  • Implémentation modulaire de chaînes d’assimilation de données
    • - recherche de méthodes
      • flexibilité

- exploitation opérationnelle

      • performances
slide21

Couplage dynamique

PALM: le principe

  • Les différentes techniques d’assimilation de données peuvent se décomposer en opérations élémentaires
  • Un enchaînement donné de ces opérations élémentaires (unités) génère une technique d’assimilation particulière
  • L’outil PALM assure (suivant la description faite par l’utilisateur)
    • l’enchaînement et la synchronisation des unités
    • les échanges efficaces d’objets entre les unités
slide22

PALM: l’utilisation

  • 1. Décomposer l’application en unités de traitement:
  • décrire les données produites (output) ou requises (input) de l’unités :objets
  • implémenter dans l’unité les requêtes d’objets input :PALM_GET
  • implémenter dans l’unité les mises à disposition d’objets output:PALM_PUT
  • 2. Utiliser l’interface graphique pré-palm pour:
  • décrire l’enchaînement dynamique de ces unités (for, while, if , select): les branches
  • décrire les échanges d’objets entre les différentes unités
  • 3. Compiler, lancer l’application et laisser PALM assurer
  • l’enchaînement et la synchronisation des unités décrits avec pre-Palm
  • les échanges efficaces d’objets entre les unités
slide25

Pre_

  • Interface utilisateur
  • Version SPMD (basee sur MPI1) :
  • Version MPMD (basee sur MPI2):

_research

_MP

>Disponibles gratuitement pour la recherche

http://www.cerfacs.fr/~palm

PALM: le développement

slide26

PALM: les utilisateurs

  • Projet MERCATOR:
    • assimilation de données en océanographie opérationnelle
  • Projet Européen ASSET:
    • assimilation de données en chimie atmosphérique (ENVISAT)
  • Projet ADOMOCA (PNCA):
    • PALM choisi comme outil fédérateur pour l’assimilation de données en chimie atmosphérique en France
  • EDF:
    • assimilation de données dans les cœurs de réacteurs nucléaires
  • SNECMA:
    • couplage 3D fluide–radiatif–solide (simulation des températures de paroi des chambres de combustion de moteur d’avion).
  • elsA (CERFACS):
    • couplage fluide-structure
  • Projet OPAVAR (CERFACS):
    • assimilation variationnelle 3/4DVAR dans l'océan
  • CESBIO, SCHAPI, TURBOMECA, IUSTI, IMFT,…
slide27

OASIS et PALM: en résumé

  • OASIS: coupleur statique, interpolation parallèle -> modélisation climatique
    • Utilisé par plusieurs groupes internationalement
    • OASIS3 -> OASIS4 (version « grand public » prévue d’ici 1 an)
  • PALM: coupleur dynamique -> assimilation de données + autres couplages dynamiques
    • Communauté grandissante d’utilisateurs
    • Quelques améliorations (objets dyn, IO parallèles) en développement
  • OASIS et PALM répondent à des besoins différents de communautés différentes mais voisines
  • Fusion des coupleurs à + ou – long terme?
    • .
slide28

En conclusion

  • Beaucoup d’interaction avec la communauté des utilisateurs
  • Nombreuses collaborations
  • Bénéfices de l’approche « open-source » (OASIS)
  • Besoin d’une intégration centrale forte
  • Au CERFACS, la coexistence d’activités de développement de logiciels et d’actions de recherche au sein d’une même équipe est extrêmement bénéfique!
    • .
slide30

OASIS4: component model description

  • Application and component description (XML files):
  • For each application (code):

one Application Description (AD):

      • possible number of processes, components included, etc.
  • For each component in the application:

one Potential Model Input and Output Description (PMIOD)

      • component general characteristics: name, component simulated, …
      • grid information: domain, resolution(s), grid type, …
      • potential I/O or coupling variables:
        • local name, standard name (NetCDF CF convention)
        • units, valid min and max
        • numerical type
        • associated grid and points
        • intent –input and/or output
slide31

OASIS4: coupled model configuration

  • Coupled model configuration (XML files):
  • (Through a GUI,) the user produces
  • a Specific Coupling Configuration (SCC):
    • start date and end date
    • start mode (MPI1, MPI2)
    • applications, components for each application
    • host(s), number of processes per host, ranks for each component
  • For each component,
    • a Specific Model Input and Output Configuration (SMIOC)
      • grid information: chosen resolution, …
      • I/O or coupling variables:
        • local and standard name, units, valid min max, numerical type, grid
        • activated intent –input and/or output
        • source and/or target (component and/or file)
        • coupling or I/O dates
        • transformations/interpolations
slide32

ATM-LAND AD

Driver

ATM PMIOD

V3: in, metadata V3

V4: out, metadata V4

V5: out, metadata V5

user

user

OCE AD

user

OCE PMIOD

V1: out, metadata V1

V2: in, metadata V2

user

  • SCC
    • ATM:...
    • OCE:...
    • LAND:...

OCE

OCE SMIOC

V1 : to ATM V3, T1

to file V1

V2 : from ATM V4, T2

ATM

ATM SMIOC

V3 : from OCE V1, T1

V4: to OCE V2, T2

V5 : to LAND V6

T

LAND

V4

V2

V5

V6

V3

V1

LAND PMIOD

V6: in, metadata V6

V7: in, metadata V7

user

V7

fileV1

fileV7

Definition Phase

LAND SMIOC

V6 : from ATM V5

V7 : from fileV7

Composition Phase

Deployment Phase

slide33

x x x x x x x x x x x x x x x x x x x x

o o o

o o o

o o o

3/ EPIOS and EPIOT definition

2/ Detailed neighbourhood search

1/ Envelopexchange

x x x x x xx x x

x x x x x x x x x

o o o

o o o

o o o

o o o

EPIOS1

EPIOT1

x x

x x

x x

x x

x x

x x

o o o

o o o

o o o

o o o

EPIOS2

EPIOT2

x x x x x

x x x x x

Oasis4: communication (2/3)

Parallel calculation of communication patterns in source PSMILe

For each pair of source and target processes: