WCRP Community-wide Consultation on Model Evaluation and Improvement

1. WCRP Community-wide Consultation on Model Evaluation and Improvement Sandrine Bony, Jerry Meehl, Anna Pirani (WGCM) Christian Jakob, Martin Miller (WGNE) Ben Kirtman (WGSIP), Stephen Griffies (WGOMD), Tony Busalacchi (WCRP) Background and Goal : - Model errors and biases are key limitations of the skill of model predictions over a wide range of time and space scales ; - Not a new story. The increase of resolution and the addition of complexity in ESMs have not solved the problem. - How to tackle the problem ? What should we do? What can we do ? -> Bottom-up consultation of NWP/climate modeling groups, CLIVAR WGs/panels, WCRP/WWRP/IGBP projects - Restructuring of WCRP : an opportunity to put recommendations into action.

2. History • Idea of a survey discussed at WGNE-23 (2007) and approved at WGNE-24 (2008) • Independently CLIVAR/WGCM were planning similar activity • Merged ideas into WCRP-wide call, but with focus on ALL applications - NWP, seasonal, decadal, climate • expanded to WWRP feat. THORPEX community in Dec 2009

3. Who? To: - NWP and Seasonal Forecasting Centers - World Climate Modeling Centers - WGCM and associated MIPs (PMIP, CFMIP, C4MIP, etc) - CLIVAR modeling groups (WGOMD, WGSIP) - CLIVAR regional and monsoon panels - US CLIVAR panels and working groups - WCRP Task force on Regional Climate Downscaling - WCRP Projects (CLIC, SPARC, GEWEX) - THORPEX, WWRP - IGBP/AIMES From: Sandrine Bony, Gerald Meehl, Anna Pirani (WGCM), Christian Jakob, Martin Miller (WGNE), Ben Kirtman (WGSIP), Stephen Griffies (WGOMD), Tony Busalacchi (WCRP)

4. Preamble • Identifying these errors and understanding their root cause constitutes a prerequisite for the planning of model improvement activities. For this purpose, we propose to initiate a "bottom-up survey" about the key deficiencies of NWP and climate models. This survey includes problems identified in operational NWP and seasonal prediction centers as well as deficiencies that climate modelers and analysts of CMIP3 simulations have identified for the current generation of models

5. The questions : Q1. Please state your particular area of interest (e.g. global/regional climate modeling, NWP, ...) Q2. Given your interest, what would you consider/identify as the KEY uncertainties/deficiencies/problems of current models ? What do you think should be evaluated/improved as a priority in models in terms of parameterization and/or interactions among processes? Q3: Do you see a particular gap (in knowledge, in observations or in practice) that would need to be filled, or a particular connection between different modeling communities or between modeling, process studies and observations that should be made a priority? Q4: Do you see any particular resource or opportunity within the modeling/process study/observational/theoretical community (e.g. new results, new observations) that would be particularly useful and should be exploited to tackle this problem? Q5 What would best accelerate progress on the topics raised in questions 1-4? Do you have suggestions for new initiatives (new process studies, field campaigns, or new collaborative approaches, eg international Working Groups, Climate Process Teams)? Q6: Any other suggestions/issues to be raised?

6. Overview over responses • Received 110 independent responses so far • approximately 50 responses from people at modelling centres, 60 from other sources • Interests stated: NWP-18; Seasonal-33; Decadal & Climate-93 • Analysis of the results is ongoing

7. General issues • hot new topics vs long-standing errors -> imbalance in visibility and effort • resolution often portrayed as the solution to everything -> untrue and new problems arise • imbalance in the maturity and size of efforts in evaluating model components (e.g. atmosphere vs biogeochemistry) • need for more interdisciplinary interactions

8. Some highlights so far • Key issues/deficiencies (Q2): • Tropical biases and variability (double ITCZ, cold tongue, ENSO, MJO,...) • Clouds, moist processes and associated feedbacks • Carbon cycle and land surface/ocean - atmosphere coupling in general • Troposphere-stratosphere interaction • Physics in high-resolution modeling

9. Some highlights so far • Key gaps (Q3) • Methods to identify the key players in model error at process level -> relating processes to climate • not enough people • weak links between bottom-up and top-down model development • better link obs and process studies • better link NWP and climate approaches (seamless)

10. Some highlights so far • Resources and Approaches to use (Q4) • Combine top-down and bottom-up approaches • use short-range forecasts to confront models • reduce fragmentation in community • focus on understanding rather than describing model differences • improve leadership in model community to integrate different approaches

11. Some highlights so far • Ways to accelerate (Q5) • Bring process and application groups together (e.g. latest WGNE development) • Correct over-investment into satellites and field experiments relative to model development • Specific activities, rather than more working groups • Form new working group on tropical biases • more flexible programs rather than entrenched acronyms • focus more on research than publicity

12. Some highlights so far • Other issues (Q6) • highlight need for model development as key to all other activities • how to convince policy makers that previous statement is true? • Should the US have a single seamless modelling system? • Redefine scientific merit to encourage more model developers • Objective techniques to prioritize model development • Stop going after new things before old big problems are solved

13. An early attempt at synergy • Promote growth of the model development community • Organize coordinated investigations into the link between model error and prediction error • Reduce the gaps between modelling, process and observations communities • Reduce the gaps between NWP/seasonal/climate communities • Observation-related issues: maintain networks, simulators, OIPs • Facilitate sharing of resources (cf. CMIP) • Focus on the questions, not the acronyms!

14. Where to from here? • Preliminary report to WCRP JSC -> Done • Authors of survey analyze results independent of each other and exchange their results -> Ongoing • Use results to help organize a model/parametrization development workshop in early 2011 - Goal: Plans how to solve top-priority problems identified by the survey • Encourage CMIP5 analysis based on key issues highlighted in survey (e.g., special diagnostic sub-projects) • BAMS (or so) paper on the outcomes of the survey (Christian and Sandrine, Q3 2010) • Create a web-based resource for the community based on the responses