Second SRNWP Workshop on Short-range EPS
7-8 April 2005, Bologna (Italy)
Highlights of the final discussion
Chair: Roberto Buizza
As the workshop final discussion was a free discussion not following a plan, the following summary is to be seen as a list of the highlights of the discussion. It must be understood that the perception of what is a highlight is very subjective and personal. Thus nobody will claim that the following summary contains all the important statements and ideas expressed.
Quality of probabilistic forecasts
With the EPS technique, we could be tempted to think that it is no longer imperious to improve the different aspects of the classical NWP, as the data assimilation, the model physics or the model dynamics. But Roberto Buizza was very clear: it is an illusion to think that we could make good probabilistic forecasts with bad analyses and bad models.
Bad analyses and bad models could enlarge the spread, but the spread must be centred over the true solution, over the reality. It has even been said that "the quality of the data analysis and of the model are more important than the way perturbations are created".
The Breeding Method
This method is loosing momentum. At NCEP it is already decided to abandon it. Its main advantage was surely its relative practical simplicity, but its theoretical background is lean which does not allow a good control of the perturbations, its main deficiency being that there is no control on the orthogonality of the perturbations.
DA and EPS
If you admit - and the trend seems to go into this direction - that the future of numerical forecasting will be given by systems able to explicitly account for uncertainties at the analysis and as well as at the forecast times, you marry DA and EPS with each other. The new problem is to choose them such that they are consistent with each other.
Ensemble Data Assimilation (EDA)
Work in Europe on EDA is starting to take momentum, with the Met Office experimenting the use of EnKF techniques to generate ensemble initial perturbations, and ECMWF starting to revisit earlier work that tested the use of ensemble data assimilation to generate ensemble initial conditions.
Ensemble Kalman Filter and Ensemble Transform Methods
The developments around these methods are presently booming in America, but not so much in Europe. The difference is striking: almost the whole European effort is devolved to the variational methods (with 4D-Var as final aim) when in the United States both paths - Var and KF - are followed.
The classical KF method has undergone several developments recently leading to the ETKF (Ensemble Transform Kalman Filter) and the LEKF (Local Ensemble Kalman Filter). The NCEP ET (Ensemble Transform) has been praised by Mozheng Wei as being the best choice.
For the global models:
Today bred-vectors are still more largely used than SV-based methods.
But SVs are becoming more popular. For example, Environment Canada and the Met Office are starting to experiment their use in ensemble systems.
It is very probable that the SV method will supplant in the future the bred-vectors.
For the LAMs:
The general opinion was that this method is not yet "ripe" for use with the high resolution LAMs.
Perturbation incompatibility in LAMs:
If you choose to perturb simultaneously analysis and boundary conditions in the way that each member will differ by its initial condition and by its boundary conditions during the
forecast, you have to be careful that these perturbations are compatible.
It has been said that it would not be the best - at least from a theoretical point of view - to perturb both independently, for example to produce the analysis perturbations from random perturbations of the observations and to take the boundary conditions from members of an independent global EPS.
It has been claimed that the ideal LAM EPS would be produced by a unified model, that is with a LAM very similar to the global model where the BC would come from and that each LAM EPS member would be forced by a unique corresponding member of the global ensemble: initial condition and boundary conditions all from the same integration of that global member.
With this scheme, each LAM EPS member would perform a pure dynamical downscaling of its corresponding member of the global EPS.
Where to put the uncertainties?
Ensembles are today primarily defined by putting the uncertainties in the analyses. Much less work has been done on the model uncertainties. One colleague expressed the wish that before we put a lot of effort into the model uncertainties, it would be valuable to have an idea about the respective share in the probabilistic forecasts between the model uncertainties and the uncertainties in the initial conditions.
But the problem is not easy because the analysis is model dependent. Therefore the analysis errors are influenced by model errors. This inter-connection makes it extremely difficult to clearly separate the role of initial and model uncertainties.
At the workshop there has been very little discussion on stochastic physics. The main contribution in this field has been in the presentation of the Meteorological Office with the random use of different coefficients for some physical schemes as convection, boundary layer and gravity wave drag.
How to account for the intrinsic model uncertainties?
One of the basic intrinsic uncertainties in a numerical model is due to the numerical discretisation in space and time. This uncertainty should be dealt with by the use of pure stochastic perturbations.
The use of different physical parameterizations between members (for example some members with mass-flux, others with Kain-Fritsch, others with adjustment) does not account for the intrinsic model uncertainties. This technique can enlarge the spread, but favours clustering of the members around the different parameterizations.
When parameterizations are randomly changed in the course of integration, the different model configurations are not all of the same quality: to enlarge an ensemble by introduction of "second quality physics" cannot be a strategy for the long term.
Ideally, we should have true stochastic parameterizations instead of randomizing parameterizations.
Ensemble size for a LAM EPS
This very important question has been asked by the Chairman, but raised up in the audience only one reaction. It must be concluded that we do not yet know the answer.
But we have to be careful not to continue to think "the bigger, the better". In the ECMWF presentations, it has been shown that there is a threshold in the number of members above which the improvement gained by adding members is marginal. For Robert Buizza a limited number of good members is better than a large number of average members.
Use of short-range LAM EPS
It has been generally admitted that in the Weather Services forecasters know today what the method is about and - more important - begin to use it when looking for information when confronted with an uncertain weather development.
In Europe, no Weather Service issues today weather forecasts in probabilistic terms. But in the French speaking part of Switzerland, the TV and a major newspaper accompany each forecast - presented in deterministic terms - with a confidence index (between 10 and 1) based on a clustering algorithm applied to the ECMWF EPS.
But it seems that the general public is no yet ready to accept probabilistic forecasts.
Franco Molteni indirectly confirms that by indicating that neither Meteo-France nor the Meteorological Office present probabilistic forecasts on their respective web site
(also true for the DWD).
It was a wish of Tiziana Paccagnella that we discuss (and find!) possibilities of collaborations. The SRNWP Programme Coordinator thought that it would be wrong to define at the end of the meeting a deep, large-scale collaboration. This would very probably be unsuccessful. More promising in his view would be to start with something small and to enlarge it if it suscitates interest.
It has been decided to create a chapter on "Short-range EPS" in the web site of the SRNWP Programme. As first step, information will be collected from the NWS active in short-range EPS in order to give a synoptic view on the state of this technique in Europe.
As second step, we should try to agree on some standard products and some verification scores.
It will be rather easy in my view to find agreement on the scores (Brier, ROC and Talagrand) but the very different spatial resolution of the European EPS will still make comparisons delicate.
Connection with THORPEX-TIGGE
In view of the importance attached by the First TIGGE Workshop on "the interfacing of global ensemble prediction systems with LAM ensembles", the Workshop has proposed that the Executive Director of the THORPEX/WMO Project (Dr Burridge) should be informed that the European Short-Range EPS Community
- strongly supports TIGGE activities
- thinks that it should be represented at any forthcoming planning meeting when appropriate.
The Manager of the SRNWP Programme will write to Dr. Burridge on behalf of the Workshop participants.
Next SRNWP Workshop on Short-Range EPS
When closing the meeting, Massimo Capaldo announced on behalf of the UGM that the next workshop will take place in Italy in the spring 2007.
For the report, reviewed and supplemented by Roberto Buizza:
SRNWP Programme Manager