Multi-Model Fusion and Uncertainty Estimation for Ocean Prediction

1. Multiple Models Observing System A useful methodology: • must be capable of operating with a small sample of validating events, possibly 1 event • Must be designed to work with sparse (in space and in time) and limited observational data • A Methodology for Multi-Model Forecast Fusion • Bias Correction Suitable for Ocean Prediction • Uncertainty Estimation for Ocean Prediction • Multi-Model Fusion and Uncertainty Estimation • for Ocean Prediction

2. The described optimization problem can be shown to yield the following solution: Bayesian Multi-Model Fusion • Seek central forecast as a linear combination of the individual forecasts, with spatially varying weights • Weight matrices D found to ensure that central forecast has minimum error variance and is unbiased Logutov, O. G. Multi-Model Fusion and Uncertainty Estimation for Ocean Prediction.Ph.D. dissertation, Harvard University, 2007

3. Forecast, deg. C Forecast, deg. C Uncertainty, deg. C Uncertainty, deg. C EmployingBayesian Multi-Model Fusionfor Integration of Multiple Models Into a Single Ocean Prediction System Example: 24 hour HOPS/ROMS SST forecast, valid Aug 28, 2003 ? HOPS ROMS Central Forecast, deg. C • Consists of combining the individual forecasts based on their relative uncertainties • minimizes error variance of central forecast • spatially varying weights have clear interpretation of Bayes factors associated with the individual models HOPS ROMS

4. Unconstrained ML estimation: Given q realizations of forecast error the unconstrained Maximum-Likelihood error covariance estimate has Wishart distribution Observational data are limited and spatially/temporally sparse Data collected at different locations for different events Volume of observational data changes with time Traditional practice of using unconstrained ML (right panel) leads to estimates that are unbiased but with large estimation error. Not suitable when the number of error realizations is small Sequence of validation events Data-Model Misfits Data-Model Misfits Data-Model Misfits Two required steps prior to Bayesian Multi-Model Fusion: • Bias correction • Uncertainty estimation (main diagonal of s) • Bias correction • Uncertainty estimation (main diagonal of s) Data-model misfits is an important source of information for identifying uncertainty and bias parameters

5. Look for uncertainty estimate in the form • MSE of can be estimated by expressing it via expectation and variance of quadratic forms in normal variables • Given consider a quadratic form in x • Find expectation and variance of y: • Find lambdas that minimize the MSE of Proposed Method of Uncertainty Estimation: Example of Uncertainty Estimate for AOSN-2 Obtained from 3 validation events • The constrained (e.g. constant on fixed depth levels) error variance has bias coming from a stringent misspecified structural assumption, but little estimation error. On the contrary,  the unconstrained estimates are asymptotically unbiased but have a lot of estimation error.   • Seek uncertainty as a linear combination of spatially constrained and unconstrained estimates The main idea of the method is to find an optimal balance of detail and generality in the output uncertainty spatial structure • Mean-Squared Error (MSE) of estimator B

6. By assuming error variance of ensemble-generated uncertainties, we can combine the approach of uncertainty estimation from data-model misfits with the ensemble-based uncertainty modeling • Outstanding problem: how can we go about estimating error variance of uncertainty estimates in ensemble-based uncertainty modeling? From misfits From ESSE ensemble analysis Uncertainty, deg. C Uncertainty, deg. C Uncertainty, deg. C We need not only uncertainty, but alsouncertainty of uncertainty. By analyzing expectation and variance of quadratic forms we can compute error variance of uncertainty estimates generated from data-model misfits

7. Example of Central Forecast for AOSN-2 • Central Forecast • HOPS • ROMS • 24-hour T forecast for Aug 14, 2003

8. RMS of HOPS, ROMS, and Central 24-hour Temperature Forecasts • Z=10 m • Z=150 m