New Approaches to Data Assimilation

1. New Approaches to Data Assimilation Greg Hakim1 & Pierre Gauthier2 1University of Washington 2Meteorological Service of Canada THORPEX Pacific Predictability Experiment workshop 6 June 2005 Contributors: Jeff Anderson, John Derber, Brian Etherton, Geoff DiMego, Josh Hacker, Tom Hamill, Jim Hansen, Steve Lord, Sharan Majumdar, Michael Morgan, Rebecca Morss, Carolyn Reynolds, Chris Snyder, & Istvan Szunyogh.

2. What DA advances would most impact THORPEX? How can THORPEX accelerate progress in DA? Goals & Background Answer depends upon whether THORPEX aims for: • incremental improvements to existing systems OR • forward-looking research tonext-generation systems? Main issues: • model error. • targeted data assimilation. • observing network design. • needs for OSSEs/OSEs and field campaigns. Hakim & Gauthier---New Approaches to Data Assimilation

3. Research & Operational Communities Complementary strengths and agendas. THORPEX provides an opportunity for greater interaction between these two communities: faster progress Hakim & Gauthier---New Approaches to Data Assimilation

4. Strengths / Role / Interests of the Research Community • basic science • new DA techniques. • predictability • DA as a tool for dynamics. • idealized testing & rapid prototyping (e.g. DART). • pseudo-operational DA on small scale. • interest mainly in ensemble filters • 3dvar (mainly for testing new ob types) • 4dvar too hard for academic community? Hakim & Gauthier---New Approaches to Data Assimilation

5. Strengths / Role / Interests of the operational community • controlled, incremental changes to operational systems. • obs gathering, quality control, and error specification. • forward operators. • satellite radiance assimilation. • supercomputing facilities. • OSE/OSSEs with operational systems. • Main interest: 4dvar • minority interest in ensemble filters. Hakim & Gauthier---New Approaches to Data Assimilation

6. Generic DA Future Issues • multi-scale obs & increments. • convective scale ---> planetary scale. • model error. • observation data deluge. • automatic data selection "crucial" for future. • non-Gaussian errors. • coupled DA; chemistry DA. Hakim & Gauthier---New Approaches to Data Assimilation

7. observation assimilation model runs AnalysesForecastsAnalyses Ensemble Filters Essential aspect: ensemble estimate of background error covariance matrix Hakim & Gauthier---New Approaches to Data Assimilation

8. MSLP ob: 500 hPa Analysis Increment 3DVAR EnKF Hakim & Gauthier---New Approaches to Data Assimilation

9. Mesoscale Example: cov(|V|, qrain) Hakim & Gauthier---New Approaches to Data Assimilation

10. 4dvar background Essential aspect: fits observations over a time period that satisfy model dynamics. analysis observations t0 t Hakim & Gauthier---New Approaches to Data Assimilation

11. Ensemble Filter Issues • model error: background. • sampling error. • balance. • serial obs. processing • some solutions exist. • sequential update problematic for predominantly asynchronous obs. • tighter coupling at expense of modularity? • filter wrapping in e.g. Python. Hakim & Gauthier---New Approaches to Data Assimilation

12. 4DVAR Issues • model error: background & obs estimates. • adjoint model • linearity (multi-scale). • ~fixed background errors (multi-scale). • initial time. • deterministic analysis. • NCEP: cost-effective 4dvar • limited run-time availability. • “situation dependent” background errors. Hakim & Gauthier---New Approaches to Data Assimilation

13. 4DVAR-Ensemble Filter Fusion • Ensemble background errors in 4dvar: • ETKF + 4dvar update for mean. • 4dvar update on members. • EnKF/3DVAR hybrids. • Kalman smoothers. • “Analysis of record” Hakim & Gauthier---New Approaches to Data Assimilation

14. Key DA Research & Applications • [Operational systems]. • Model error. • Targeted data assimilation. • Dynamics. • Observing network design. Hakim & Gauthier---New Approaches to Data Assimilation

15. Model Error • model calibration. • e.g. parameter estimation. • systematic, conditional, development. • new changes conditioned on existing model structure. • use DA to construct frameworks. • e.g. state-dependent model error. • non-Gaussian errors. Hakim & Gauthier---New Approaches to Data Assimilation

16. Targeted Data Assimilation • metric-dependent filtered obs stream. • not just satellite data thinning! • e.g. ob usage depends on forecast lead time. • e.g. ob usage depends on forecast metric. • required for THORPEX wide-ranging time/space scales? Hakim & Gauthier---New Approaches to Data Assimilation

17. balance equations ensemble inversion recovered divergence Dynamics • scale interactions. • predictability error growth. • errors conditioned on flow structure. • new tool for dynamics. • e.g. ensemble potential vorticity inversion. Hakim & Gauthier---New Approaches to Data Assimilation

18. Observing Network Design • transition the “organically grown” network. • where to move old obs and/or when to use. • e.g. radiosondes. • new fixed optimal obs sites & types. • basic science and practical aspects. • entanglement with DA approach & norms • try many. Hakim & Gauthier---New Approaches to Data Assimilation

19. Summary • THORPEX and DA need each other. • basic science & operational advances. • Ensemble filters, 4dvar, fusion. • Pacific campaign • Model error. • best obs for state-dependent calibration. • Targeted data assimilation. • Oversampling for OSEs. Hakim & Gauthier---New Approaches to Data Assimilation