B.Ritter (DWD) & R.Pincus (Uni of Colorado)

1. Monte Carlo Spectral IntegrationA computationally efficient alternative to sparse radiative transfer calculations in COSMO NWP simulations B.Ritter (DWD) & R.Pincus (Uni of Colorado)

2. Radiative transfer in a plane parallel, horizontally homogeneous atmosphere can be described by the monochromatic radiative transfer equation (RTE): scattering of direct beam incoming radiance emission scattering of diffuse radiance components with L monochromatic directional radiance  optical thickness ,µ0 cosine of zenith angle for diffuse resp. direct radiation  azimuth angle single scattering albedo B Planck function P Phase function for scattering ,0 scattering angle for diffuse resp. direct solar radiation S0 solar constant      

3. The efficiency problem of radiative transfer parameterisation • even though the RTE in the form shown previously already contains severe approximations (e.g. considerung only one dimension), further simplifications are required to find a solution which is at all feasible within the constraints of operational NWP • the most critical issue in this context originates from the need to integrate solutions of the RTE over all energetically relevant wavelengths

4. The efficiency problem of radiative transfer parameterisation • integration over wavelength is severly hampered by the huge spectral variability of gaseous absorption coefficients

5. The efficiency problem of radiative transfer parameterisation • replacing the numerical integration over wavelength by the so-called k-distribution method leads to affordable, but still very expensive solutions of the RT problem (cf. Fu and Liou, 1992)

6. The efficiency problem of radiative transfer parameterisation • simulation of radiative transfer in the atmosphere for every column at each model time step provides the spatio-temporal distribution of corresponding fluxes as: where the summation over spectral bands reflects the fact that the grouping of gaseous absorption coefficients is carried out within intervals where the optical properties of other constituents (e.g. cloud droplets) are considered to be constant • BUT: RT simulation as described above is still far too expensive for NWP! • Question:What means to save CPU time are left?

7. Making interactive radiation in NWP models affordable The usual compromise • Compute radiative heating rates every N time steps, apply these uniformly in time and/or employ a coarser grid for RT calculation than for other processes simulated leads to the usual disadvantages • flux and heating rate errors are correlated with the flow (largest where flow/development is most vigorous, smallest in quasi-stationary situations) • no theoretical basis: no guidelines for choosing ‘optimal’ radiation time step and or spatial resolution, and no way of knowing when this choice is affecting the solution • difficult to assess whether optimal computational efficiency (i.e. a good cost/benefit ratio) is achieved

8. Making interactive radiation in NWP models affordable a schematic illustration of the effects of reduced temporal sampling change of atmospheric state between radiation time steps is not reflected in fluxes, leading to sub-optimal interaction with other processes substantial bias may occur (e.g. time lag between radiative fluxes and diurnal cycle of cloud field)

9. An alternative: Monte Carlo spectral integration • Why are heating rate calculations so expensive? It’s the broadband integration - the double sum over bands and g-points! • We use the roadblock as a springboard: (cf. Pincus&Stevens, 2008) This is a Monte Carlo sample of the calculation we’d like to do but can’t afford. A single estimate is (very) noisy but many estimates converge to the right answer.

10. A schematic depiction of the MCSI approach

11. Classical approach: low frequency RT calculations

12. MCSI approach: high frequency random sampling

13. Making interactive radiation in NWP models affordable a schematic illustration of advantages of the MCSI approach over classical approach - low frequency, deterministic sampling MCSI approach introduces noise in fluxes but responds to changes in atmospheric state immediately no bias occurs for sufficiently large sample size

14. Is MCSI a valid approximation for atmospheric simulations?Can the approach be used in the COSMO NWP model? • Pincus&Stevens, 2008 demonstrate the successful application of the MCSI approach in LES model simulations of the evolution of a nocturnal stratocumulus fields. • The radiative transfer scheme of the COSMO model (cf. Ritter and Geleyn, 1992) can be modified easily so that an MCSI-like behaviour is achieved. • Some proof-of-concept tests demonstrate that MCSI may be used successfully in COSMO in order to overcome problems associated with low temporal frequency radiation calculations Pincus and Stevens, 2008

15. Convergence test for the MCSI approach implemented in RG92 RT-scheme Example: • The ‚soft‘ version of MCSI: • retains the loop over spectral bands • is less noisy than original MCSI version • is less efficient than original, but still much faster than standard RT calculations

16. Experiments with COSMO-DE (Version 4.14) Initial date: 20100808 12 UTC Experiment 1: Operational DWD configuration, i.e. hincrad=0.25,lradf_avg=.true. Experiment 2: as 1), but lradf_avg=.false., i.e. RT is calculated at every grid point Experiment 3: as 2), but soft MCSI approach instead of full spectral integration Experiment 4: as 3), but nincrad=1, i.e. ‚radiation time step‘ = ‚dynamics time step‘ Experiment 5: as 2), but nincrad=1, i.e. ‚radiation time step‘ = ‚dynamics time step‘ • Experiment 5 can be considered as ‚reference‘ !

17. Comparison of hourly precipitation rates Experiment 4, i.e. MCSI at each time step and grid point Reference

18. Comparison of hourly precipitation rates Experiment 3, i.e. MCSI at each grid point, but nincrad=36 Reference

19. Comparison of hourly precipitation rates Experiment 1, i.e. operational COSMO-DE configuration Reference

20. Comparison of T2m at end of forecast range Reference Experiment 4 MCSI introduces some small scale noise, but no bias

21. Comparison of T2m at end of forecast range Experiment 1, operational configuration Experiment 2, no spatial averaging Impact of ‚coarse radiation grid‘ is at least as large as that of MCSI

22. Computational efficiency • Standard RT calculations at each time step and grid point would blow the computational budget available for operational NWP • RT calculations employing the ‚soft‘ MCSI approach are approximately 5 times faster than the full RG92 scheme • through further code optimization a theoretically possible speed-up factor of ~10 may be achieved • Using the MCSI approach and a fairly small additional investment of CPU time, we could avoid the downscaling & call radiation more often

23. Pitfalls resulting from naive use of RNG in parallel architectures Code like x=random_number() will provide a useable series of random numbers on a serial machine, but • every random number generator keeps track of its ‚state‘ via a (set of) global variable(s) • in a distributed memory architecture, the Single Program – Multiple Memory concept for parallelisation will lead to identical sequences of random numbers for each processor if no attention is paid to the seeding/initialization of the RNG • in a shared memory architecture, where both the program and the state of the RNG are shared between processors, the sequence of random numbers obtained by an individual processor may depend on the work balance between processes and/or the domain decomposition, leading to non-reproducible results If undesirable side effects are to be avoided, care in the choice and application of the RNG are essential!

24. Pitfalls resulting from naive use of RNG in parallel architectures: here ‚careless use on distributed memory machine‘ Identical random number sequences on each task (=sub-domain) are definitely not what we want for MCSI!

25. Pitfalls resulting from naive use of RNG in parallel architectures Proper seeding for each task individually avoids identical sequences, but reproducibility is only ensured, if domain decomposition is not changed!

26. Use of NEC random number generator for MCSI in COSMO-DE • individual seeding for each task • check of ‚randomness‘ in space&time of random numbers obtained in RT scheme

27. Summary and Conclusions MCSI provides an opportunity to overcome some shortcomings of classical approaches to deal with the efficiency problem of RT calculations Applying the MCSI approach in the framework of the COSMO NWP model demonstrated: • the introduction as a variation of the RG92 radiation scheme poses no major problem • no evidence of significant deterioration of critical forecast products was found in a forecast experiment • even the ‚soft‘ version MCSI is much faster than the standard RT scheme • special attention is necessary to ensure that the mechanism employed for the generation of random numbers does not lead to undesirable side effects A closer inspection of this approach appears to be worth the effort!