1 / 34

The Realization of GRAPES Model on China Meteorological Application Grid

The Realization of GRAPES Model on China Meteorological Application Grid. Xuesheng YANG Chinese Academy of Meteorological Sciences China Meteorological Administration yangxs@cma.gov.cn 2005/09/16. Characteristics of Numerical weather prediction.

wilda
Download Presentation

The Realization of GRAPES Model on China Meteorological Application Grid

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Realization of GRAPES Model on China Meteorological Application Grid Xuesheng YANG Chinese Academy of Meteorological Sciences China Meteorological Administration yangxs@cma.gov.cn 2005/09/16

  2. Characteristics of Numerical weather prediction • Meteorology is one of the main components in the grid application • US: • LEAD (Linked Environment for Atmospheric Discovery) • MEAD (Modeling Environment for Atmospheric Discovery) • ECMWF: ECAccess • NWP system includes preprocessing, analysis, numerical model, post processing, visualization: • execution processes are dynamically connected • enormously complex even if run individually • Involves in data management, remote collaborative research, heavy computation, massive data and intensive services • to ensure the proper operation of a NWP: • massive computing resources needed

  3. Existed issues • tough issues meteorologists encountered: • organization of the NWP workflow and data • execution of model simulations • management of the resulting large volumes of data • subsequent data analysis • visualization • model developers and computing resources stationed in different locations • telephone, mobile, e-mail, … • inadequate for conducting a research project that involves in huge amount of simulations and extensive on-line discussions and consultations • Computing resources can’t satisfy the research requirements • divert researchers’ focus on how to acquire computing resources instead of addressing the scientific issues

  4. Existed computing resources at CMA • Available operational computing resources: • IBM Cluster 1600 • IBM SP • Available computing resources for research: • IBM • Clusters: Sunway, Dawning, Oscar, Ocean • CAMS • NMIC • Guangdong • Shanghai • Guangxi • etc. • do not get full utilized • Computing resources • manpower

  5. 2. China Meteorological Application Grid(CMAG) • grid portal • GridWeather • a NWP workflow control interface • GRAPES meso-scale model • user management sub-system • CVSExplorer • a code management system • visualization

  6. ProjectManager CodeExplorer CVSManager CVSExplorer Portlets Modules CMAGPortal Page RainForecast CityForecast ElementForecast PrecipitionObserved Forecast Portlets Portlets Modules GridSphere Container IE browser LoginPortlet ProfileManager PortletApplication LayoutManager CredentialRecieve JobSubmission JobMonitoring JobPosttreatment FileManager FileTransfer ResourceBrowser CredentialRequest CredentialSign CredentialDelete GridUserMap Tomcat5.28 Application Server Base Portlets GridWeatherportlets Portlets Modules GridUserPortlets Portlets Modules Ganglia GridServices Myproxy GRAPESservice Database Server Mysql CVS Server Grid Software(GT2,GT3,GT4,GOS) Job Scheduler (OpenPBS,LSF,Conder) Grid Resources(Cluster,expensive instrument) architecture of CMAG Portal

  7. Security and user management • provides a single-log-on access interface for all grid services and grid resources • authenticated users can access to computing, storage, data, software resources as well as all kinds of services

  8. Security and user management • SimpleCA of Globus toolkit is adopted to ensure the security of the grid resources • based on GSI and supports X.509 inter-certification mechanism • CMAG Certification Authority signs each certificate • In actual implementation, a CA center at CMA is set up and CA distribution packages are installed at all CMAG nodes or provincial meteorological bureaus • User certifications managed by Myproxy library

  9. CMAG Portal • Resource Access Portal • to access geographically distributed computing resources • NWP Products Browse Portal • to disseminate NWP products to the public and professional clients • Remote Collaborative Portal (RCP) • to offer a universal platform for scientists to develop NWP modules, to share source codes, to register modules within the grid and to perform NWP experiments.

  10. CMAG Portal • Globus 4.0 as the communication and tool-kit for collaborative research • authentification and grid-job management is performed based on Gridsphere Portal Framework and Java COG tool-kits.

  11. Infrastructure of CMAG Portal

  12. Job Submission and resource monitoring • based on the pre-wsrf-gram and wsrf-gram of Globus 4.0 • Each grid resource has its own job scheduling software: • SSC: • PLATFORM’s LSF to perform job scheduling • Others: • OpenPBS • Ganglia to display dynamically the load of computing resource : • such as CPU load, availability of both memory and hard-disk, as well as network throughput

  13. The load report of grid resources displayed dynamically by Ganglia within CMAG

  14. NWP executing control interface --- GridWeather • NWP system involves complicated process control and massive parameters setting • traditional implementation method: • write a control routine • effective in local mode familiar with both every part of NWP system and system environment. • Gridweather: • based on Portlet including GRAM and GridFTP of CoG as well as Globus toolkit

  15. interface for research: GARPES model improvement • to select the appropriate physical parameterization schemes • to designate forecast domain, available numerical schemes, etc.

  16. Flowchart of GRAPES model improvement on CMAG job-submission strategy : firstly, select the computing resources already with special experimental data, secondly, select machines that can run the GRAPES model

  17. NWP operational executing interface

  18. Flowchart of operational GRAPES meso-scale model on CAMG • job-submitting strategy: • selecting the computing resources that have had data already • selecting idle machines, then local machine, or any available high-performance computing capabilities machines.

  19. CVSExplorer • NWP system upgrading • frequent modifications • diversity of program versions • intensive interactions between model developers and inter-comparisons between numerical experiments • it is necessary to create a suitable platform for joint NWP software development based on WinCVS

  20. Browse source codes

  21. GRAPES improvement

  22. Configuration of CMAG • portal server: • deployed on a DELL server at CAMS, includes: • Web server Tomcat • Portal Framework Gridsphere • Web service software Java SOAP • grid application and development package CoG, GMETAD • web Front-end Ganglia • management node: • implemented at OSCAR Cluster at CAMS • Hierarchically structured MDS is installed • GIIS is available at each managing node and GRIS at other nodes.

  23. Computing platform on CMAG CADP: CMA CA Distributed Packages STI: Shanghai Typhoon Research Inst.

  24. Real-time weather forecasts • GRAPES meso-scale model 30KM/15KM • Mainly for rain forecast • runs every day on CMAG • Real-time forecast products: • http://grid.cma.gov.cn:8080/ • Also computing resources and NWP technique comparatively less developed regions can utilize the grid resources or even run the NWP models to satisfy their forecast service requirement: • Yantai • Qinghai

  25. GRAPES-Meso Scale model • new NWP system developed by Research Center for Numerical Prediction at CAMS • Features: • fully compressible primitive equations • switchable between hydrostatic and non-hydrostatic mode • semi-implicit and semi-Lagrangian advection schemes • latitude and longitude grid • horizontal Arakawa C grid • terrain-following coordinates • Charney-Phillips staggering vertically • physical parameterization package

  26. prepocessing Analysis Forecast Postprocessing 3D Variation Analysis GRAPES Model real-time forecasts Post- processing Forecast products ATOVS Data Preprocess Initial 48hrs forecast Quality control visualization Conventional Data preprocess Global model lateral B.C. Quality control 6hrs cycling Flowchart of meso-scale GRAPES model on CMAG

  27. 500 hPa Geopotential height forecast over China (2005090900UTC)

  28. forecast

  29. Hourly rain forecast for Beijing, 2005080800UTC Typhoon Matsa

  30. Conclusion • Distributed computing resources at Beijing, Shanghai, Guangdong and Guangxi aggregated • real-time GRAPES executions on CMAG feasible • a virtual collaborative R/D center established: • provides researchers at different locations with a common R/D environment aimed at further improvement of the GRAPES

  31. Thanks for listening!

More Related