Tag: schevenhoven

Training a supermodel with noisy and sparse observations: a case study with CPT and the synch rule on SPEEDO – v.1

Schevenhoven, F., Carrassi, A. 2022: Training a supermodel with noisy and sparse observations: a case study with CPT and the synch rule on SPEEDO – v.1. Geosci. Model Dev. https://doi.org/10.5194/gmd-15-3831-2022

Summary: As an alternative to using the standard multi-model ensemble (MME) approach to combine the output of different models to improve prediction skill, models can also be combined dynamically to form a so-called supermodel. The supermodel approach enables a quicker correction of the model errors. In this study we connect different versions of SPEEDO, a global atmosphere-ocean-land model of intermediate complexity, into a supermodel. We focus on a weighted supermodel, in which the supermodel state is a weighted superposition of different imperfect model states. The estimation, “the training”, of the optimal weights of this combination is a critical aspect in the construction of a supermodel. In our previous works two algorithms were developed: (i) cross pollination in time (CPT)-based technique and (ii) a synchronization-based learning rule (synch rule). Those algorithms have so far been applied under the assumption of complete and noise-free observations. Here we go beyond and consider the more realistic case of noisy data that do not cover the full system’s state and are not taken at each model’s computational time step. We revise the training methods to cope with this observational scenario, while still being able to estimate accurate weights. In the synch rule an additional term is introduced to maintain physical balances, while in CPT nudging terms are added to let the models stay closer to the observations during training. Furthermore, we propose a novel formulation of the CPT method allowing the weights to be negative. This makes it possible for CPT to deal with cases in which the individual model biases have the same sign, a situation that hampers constructing a skillfully weighted supermodel based on positive weights. With these developments, both CPT and the synch rule have been made suitable to train a supermodel consisting of state of the art weather and climate models.

Link to publication. You are most welcome to contact us or the corresponding author(s) directly, if you have questions.

Training of supermodels in the context of weather and climate forecasting

Schevenhoven, Francine (2021-02-08). Training of supermodels in the context of weather and climate forecasting (PhD thesis, University of Bergen, Bergen, Norway). https://bora.uib.no/bora-xmlui/handle/11250/2727454 .

Summary: Given a set of imperfect weather or climate models, predictions can be improved by combining the models dynamically into a so called `supermodel’. The models are optimally combined to compensate their individual errors. This is different from the standard multi-model ensemble approach (MME), where the model output is statistically combined after the simulations. Instead, the supermodel can create a trajectory closer to observations than any of the imperfect models. By intervening during the forecast, errors can be reduced at an early stage and the ensemble can exhibit different dynamical behavior than any of the individual models. In this way, common errors between the models can be removed and new, physically correct behavior can appear.
In our simplified context of models sharing the same evolution function and phase space, we can define either a connected or a weighted supermodel. A connected supermodel uses nudging to bring the models closer together, while in a weighted supermodel all model states are replaced at regular time intervals (i.e., restarted) by the weighted average of the individual model states. To obtain optimal connection coefficients or weights, we need to train the supermodel on the basis of historical observations. A standard training approach such as minimization of a cost function requires many model simulations, which is computationally very expensive. This thesis has focused on developing two new methods to efficiently train supermodels. The first method is based on an idea called cross pollination in time, where models exchange states during the training. The second method is a synchronization-based learning rule, originally developed for parameter estimation.
The techniques are developed on low-order systems, such as Lorenz63, and later applied to different versions of the intermediate-complexity global coupled atmosphere-ocean-land model SPEEDO. Here the observations are from the same models, but with different parameters. The applicability of the method to real observations is tested using sensitivity to noisy and incomplete data. The characteristics the individual models should have in order to be combined together into a supermodel are identified, as well as which physical variables should be connected in a supermodel, and which ones should not. Both training methods result in supermodels that outperform both the individual models and the MME, for short term predictions as well as long term simulations. Furthermore, we show that the novel use of negative weights can improve predictions in cases where model errors do not cancel (for instance, all models are too warm with respect to the truth). A crucial advantage of the proposed training schemes is that in the present context relatively short training periods suffice to find good solutions. Although the validity of our conclusions in the context of real observations and model scenarios has yet to be proved, our results are very encouraging. In principle, the methods are suitable to train supermodels constructed using state-of-the art weather and climate models.

Link to publication. You are most welcome to contact us or the corresponding author(s) directly, if you have questions.

Improving weather and climate predictions by training of supermodels.

Schevenhoven, F., F. Selten, A. Carrassi, Keenlyside, N. 2019: Improving weather and climate predictions by training of supermodels. Earth Syst. Dynam., 10, 789–807. https://doi.org/10.5194/esd-10-789-2019

Summary: Recent studies demonstrate that weather and climate predictions potentially improve by dynamically combining different models into a so-called “supermodel”. Here, we focus on the weighted supermodel – the supermodel’s time derivative is a weighted superposition of the time derivatives of the imperfect models, referred to as weighted supermodeling. A crucial step is to train the weights of the supermodel on the basis of historical observations. Here, we apply two different training methods to a supermodel of up to four different versions of the global atmosphere–ocean–land model SPEEDO. The standard version is regarded as truth. The first training method is based on an idea called cross pollination in time (CPT), where models exchange states during the training. The second method is a synchronization-based learning rule, originally developed for parameter estimation. We demonstrate that both training methods yield climate simulations and weather predictions of superior quality as compared to the individual model versions. Supermodel predictions also outperform predictions based on the commonly used multi-model ensemble (MME) mean. Furthermore, we find evidence that negative weights can improve predictions in cases where model errors do not cancel (for instance, all models are warm with respect to the truth). In principle, the proposed training schemes are applicable to state-of-the-art models and historical observations. A prime advantage of the proposed training schemes is that in the present context relatively short training periods suffice to find good solutions. Additional work needs to be done to assess the limitations due to incomplete and noisy data, to combine models that are structurally different (different resolution and state representation, for instance) and to evaluate cases for which the truth falls outside of the model class.

Link to publication. You are most welcome to contact us or the corresponding author(s) directly, if you have questions.