Third SRNWP Workshop on Statistical Adaptation
1-2 June 2005, Vienna (Austria)
Summaries of the Presentations
Session A: Ensemble Forecasts
J. Kilpinen, FMI
Calibration of EPS winds
Experiments with very recent data (October 2004 - March 2005)
Correction of the 10m winds not only for the deterministic winds but also for the EPS winds.
Determination of probabilistic forecasts from deterministic forecasts by use of two methods:
- forecast error distribution
- neighbourhood method (the Suzanne Theis method).
No calibration of the EPS in the traditional way where a large sample of past forecasts and observations are needed. Calibration is made by Kalman filtering. First, the ensemble mean is Kalman filtered and after each member is Kalman filtered using the same coefficients.
Kalman filtering is able to reduce biases and produce better probability forecasts for most of the 6 stations considered in terms of ROC curves, ROC areas and Brier Skill Scores.
Analysing and downscaling ensemble forecasts with topology-preserving clustering methods
The Institute for Artificial Intelligence and Meteorology of the University of Cantabria (Spain) has been with 3 high-level presentations one of the major contributors of the Workshop.
A principal component analysis of ECMWF grid point values of several parameters at several levels over Iberia is realized in order to drop the number of predictors from more than 6000 to about 600. Then a clustering based on these principal components (PC) is performed. For each member of the cluster, a mean precipitation amount (mm/24h) has been computed from the observations.
In operational mode, there is a sharp drop in the quality of the precipitation forecasts after day +3 due to the loss of skill of the ECMWF forecasts. To alleviate this problem, the downscaling has been applied to the 51 members of the ECMWF EPS. Each EPS member finds its place in one member of the cluster in the PC space. Believing that the error is directly correlated with the spread of the EPS members, the authors want to use the spread a measure of predictability. In their case, the spread is immediately given by the distribution of the 51 members over the cluster. To measure the dispersion over the cluster, they use the Eckert-Cattani "Self-Organising Map" algorithm and the "Generative Topographic Map" concept which is a probabilistic re-formulation of the Self-Organizing Map algorithm.
D. Cattani and T. Comment, MeteoSwiss
Short presentation about confidence index
How to inform the public about the uncertainty of a weather
forecast? Or: How much trust should the public give to a particular weather
The solution of MeteoSwiss lies in the definition of a "confidence index".
The confidence index (CI) is based on the dispersion of the ECMWF EPS members in an adaptive table of 144 weather situations. The weather situations are defined by their respective H500hPa and T850hPa patterns over Europe and Eastern Atlantic.
The confidence index is determined with respect to a climatological dispersion. Is the EPS dispersion larger than the climatological one, the ECMWF deterministic forecast is considered totally unreliable and receives the mark zero. A confidence index of 10 corresponds to a certainty.
But since this CI is related to a large area of Europe, a low index may still be related to the same kind of weather over the small Swiss area, or a high index to different types of weather over Switzerland. In order to regionalize the CI, we consider the dispersion of the EPS for 3 specific parameters (temperature, cloudiness, precipitation), for 3 grid points over Switzerland. Those dispersions are again compared to climatology. A new CI is then calculated, as a weighted sum of the CI over Europe and the CI of each parameter.
M. Vallée, Environment Canada, RPN, Dorval (Canada)
Verification of weather elements forecast by the Canadian EPS
It has become a nice tradition that scientists of Environment Canada, more precisely from "Recherche en Prévision Numérique" in Dorval, cross the pond to visit the different EUMETNET SRNWP Workshops.
For the first time in the series of our Statistical Adaptation Workshops was RPN represented. Marcel Vallée reviewed the past and present EPS works of RPN and informed us about their future plans.
He also presented statistical scores for the verification of the EPS forecasts which are not used in Europe (at least not by the National Meteorological Services) as the "Probability Score", the "Reduce Centered Random Variable" and the "Continuous Ranked Probability Score".
Session B: Dynamical Adaptation
T. Haiden, ZAMG
INCA - High resolution downscaling of NWP-model forecasts
Thomas presented the "Integrated Nowcasting through Comprehensive Analysis" (INCA) system presently in development at the ZAMG.
Some characteristics of this system:
- Effort has been made to derive the best possible precipitation analyses by combination of the radar and rain gauge information
- Precipitations up to 6 hours are forecast by an extrapolation (of the precipitation areas of the analyses) based on motion vectors determined by precipitation pattern correlation.
Tests show that this forecast method is better than the Aladin precipitations till +4 / +5 hours.
The 2-D cloudiness analyses are determined by using the cloud type analyses produced operationally by EUMETSAT (from MSG) and the sunshine intensities measured by the Austrian automatic observing network TAWES.
The 3-D temperature analyses are based on short-range ALADIN forecasts which are modified with respect to the 1km orography of the INCA grid. The INCA temperature forecasts are clearly better than the Aladin ones till +7 / +8 hours and remain slightly better till +12 hours.
The 1-km winds are determined in the following way: after a dynamical adaptation of the 9.6-km winds from the Aladin forecasts to a 2.3-km mesh, the winds of the TAWES observing stations are nudged simultaneously with a downscaling to the 1-km INCA grid.
P. Crochet, Icelandic Meteorological Office
Precipitation mapping in Iceland using the linear theory model of orographic precipitation
A comparison of the yearly accumulated precipitations between the observations and ERA-40 shows large difference in Iceland according to the type of stations. For station in open terrain, ERA-40 is very good, but it shows large precipitation overestimates for station in rain shadow. For stations where precipitations are enhanced by orography, this enhancement is too weak for the ERA-40 precipitations.
It must not be forgotten that a rain gauge network systematically underestimates precipitation due to evaporation, plashing and aerodynamical effect, primarily for snow.
In order to improve the ERA-40 precipitations over Iceland and, consequently, the ECMWF precipitations forecasts, the ERA-40 precipitations have been corrected by the use of a linear model for orographic precipitations [Smith and Barstad, 2004]. The first results look very promising.
F. Wimmer and T. Haiden, ZAMG
Analysis of residual errors in T2m nowcasting
For the station Vienna, in February 2003, for the temperature, the persistence is better than the Aladin DMO till +4 hours and the adapted climatology is better till +10 hours. This shows that there must be room for improvement by use of an adequate method.
By introduction of an advection algorithm, the number of better forecasts by use of this algorithm largely exceeds the number of worse forecasts for the period Nov. 2004 - March 2005.
Session C: Operational Applications
S. Roquelaure and T. Bergot, Météo-France
Predictability of fog and low clouds at Paris CDG airport
Use of the 1-D version of the ISBA soil model and of the 1-D COBEL model.
The initialization of the low clouds and fog is made by explicit introduction into the analysis.
Several initial conditions have been tested and the results of the integrations compared:
- with initialization of low clouds and fog
- without initialization of low clouds and fog
- with initialization of low clouds and fog and cloudiness from the Aladin model
- with initialization of low clouds and fog and persistence of the observed cloudiness at initial time.
These tests showed a large spread for the downward IR fluxes. The best set-up has been the last one (persistence of the observed cloudiness) and the worst the one without initialization (its bias was 20 times larger).
It is interesting to note that the weakest forecasts are for 3 UTC in the afternoon: highest false alarm rate and lowest hit rate.
M. Rohn et al., DWD
Objective optimisation of local forecast guidances
The DWD presented a very comprehensive review of adaptation and optimisation of its NWP forecasts results.
From their global model GME: 3500 point forecasts worldwide, each with 21 surface weather elements, most of them MOS processed.
They also work on objective optimisation and one of the methods used is to combine model results by weighted averages. Example for 2 models (global and regional): the weights of each model will vary during the integration as a function of the range.
They also intend to use a Bayesian Model Average scheme in order to use the best model - not known in advance - for any forecast range.
DWD works also intensively in the nowcasting range. Already developed is the "SatelliteWeather": analysis with METEOSAT and SYNOP; forecast by extrapolation.
Planned is BlitzMOS whose predictand will be the probability of lightning.
H. Hoffmann and V. Renner, DWD
Interpretation of the new DWD high resolution LMK
Present configuration of the LMK: 2.8 km grid resolution, 50 levels, forecast lead time: 18 hours.
High-resolution numerical weather forecasts include noticeable stochastic elements already in the short range. Therefore direct model output for deterministic forecasts should be transformed in order to suppress essentially unpredictable small scale structures.
Probability information for the exceedance of given thresholds should be derived by statistical means.
The following methods are planned to reach these aims:
- Derivation of statistical (probabilistic) forecasts from a single (deterministic) model integration by means of the neighbourhood method (NM)
- As there will be 8 LMK integrations per day (every 3 hours), use of time-lagged ensembles.
First results based on two short periods of time:
= Deterministic precipitation forecasts:
Neighbourhood averaging (space and time) not better than simple spatial averaging over 5x5 or 15x15 grid points. Averaging over 5x5 points better than over 15x15.
For some of the elements (e.g. precipitation, gusts) a re-calibration of the distributions of the smoothed field towards the distribution of the original field will be done.
= Probabilistic precipitation forecasts:
Quality improves by increasing the size of the spatio-temporal neighbourhood. The optimal neighbourhood has not yet been determined.
Increasing the temporal neighbourhood size leads to better results than increasing the spatial neighbourhood (with the same number of points in the spatio-temporal neighbourhood).
H. Petithomme, Météo-France
New operational methods and forecast parameters at Météo-France
The author reviewed the operational production at Météo-France.
Postprocessing is made for 2500 sites in France and up to 6000 worldwide. The method used depends on the parameter: linear regression + filtering for T2m and U; linear discrimination for dd/ff, vis and N.
Computed for 374 sites over France. 2 thresholds: 28 kt and 43 kt.
Several methods tested: regression, linear discriminant analysis, pseudo PPM, direct wind gust computation. In the predictor selection procedure, the surface wind stress came first, which gives to the turbulence produced by wind shear the main cause for wind gusts.
We can say that
- higher threshold (43 kt) much more difficult to predict
- regression gives poor results
- high HR can be achieved (for example with linear discriminant analysis) but they are always linked with high FAR.
Five different classes are forecasted with linear discriminant analysis at 16 locations over France.
F. Schubiger, MeteoSwiss
Short-report of some operational adaptations at MeteoSwiss with aLMo
MeteoSwiss presented an overview of its presently operational statistical and dynamical adaptations applied to its high-resolution model aLMo.
- Kalman filtering: T2m, Td2m
- Verification of the precipitations by averaging over 5 and 13 grid points around observation point
- Verification of the cloudiness by averaging over all grid points up to 30 km of observation point
- Wind gust computation with the Brasseur method
For airplane icing warnings: production of
- liquid water content charts at 700 hPa
- temperature charts at 700 hPa.
Super-cooled cloud droplets play a very important role in aviation safety as they are the cause of air-plane icing.
Forecast maps showing the amount of cloud liquid water can be of great importance for the aviation. It is by temperatures of -2/-3 degrees that icing is the most dangerous, when glazed, transparent ice forms. At -6/-7 degrees, rimed, opaque ice forms, which is less dangerous as it remains on the edges of the wings and does not spread on the wings as the glazed ice.
Session D: Statistical Adaptation / Neural Networks
Short term rainfall forecasts from sounding-derived indices using neural networks
In the plain of Friuli-Venezia-Giulia in Northern Italy, there is a radio-sounding (RS) station (Udine-Campoformido) and 15 automatic observing stations.
The RS station makes 4 soundings a day. Each sounding is associated with the maximum of the 15 6-hourly accumulated precipitations measured by the automatic stations (the 6-hourly periods start at sounding launch times). Only precipitation events with at least 5mm in 6h at at least one of the 15 stations are considered.
For each RS of Udine-Campoformido, a very large number of indices (as CAPE and K index) and indications (as height of the tropopause and lifting condensation level) are recorded.
These indices will form the input of a neural network whose output will be the maximum of precipitation to be expected for the next 6 hours at at least one of the 15 observing stations.
The set-up and the computation of the neural network are explained with great details in the presentations.
An interesting point of the work is to look at the ranking of the indices chosen as input by the forward selection algorithm for all the precipitation cases (>=5mm/6h).
1. mean relative humidity in the lower 500hPa
2. maximum buoyancy
3. mid-level density weighted wind (v-component)
4. mean water vapour flux (v-component) at the previous sounding (6 hour before).
The same work has been done for the forecast of two precipitation classes:
one class is >20 mm / 6h at at least one station; the other class is >40mm / 6h at at least one station.
For the 20 mm class as well as for the 40 mm class, the forward selection algorithm picked up in both cases the same indices as inputs and in the same (decreasing) order:
1. mean water vapour flux (v-component)
3. standard deviation of the radiosonde vertical velocity.
But the forecasts given by the neural network for this last class are very poor. They are better for the 20mm class and, according to the author, good for RR> 5mm/6h.
C. Sordo and J. Gutiérrez, University of Cantabria (Spain)
A comparison of PCA and CCA predictors for wind speed downscaling using logistic regression and neural networks
PCA = Principal Components
CCA = Canonical Correlation Analysis
The authors have started their presentation by showing for a simple example (temperature in Santander to be deduced from the temperature of the ERA nearest grid point) that a linear relationship (regression) is totally inappropriate. Thus nonlinear relationship between predictands and predictors must be used.
The problem is the determination of the probabilities of wind speeds > 50 kt at 11 stations in the North of Spain using as input T, Z, U, V and H at 27 grid points of ERA-40 for the period January 1977 - August 2002.
The first step is to reduce the amount of data of the predictors.
Two methods are used:
- Principal Component Analysis (PCA). With only 10% of the PCs, they can reconstruct fields presenting a RMSE of 2 % only.
- Canonical Correlation Analysis (CCA): an output data vector on a lower dimensional space which should have the maximum correlation with the equally reduced input data vector must be defined.
The PCA and the CCA data are used as input for two models:
- Logistic Regression
- Neural Network.
The best results measured by the Brier Skill Score are given by the Neural Network using as input 10 Principal Components. The use of a larger number of PCs has not improved the results significantly.
J. Vehovar, Environment Agency of the Republic of Slovenia
Temperature forecasting in terms of quantiles
Instead of having as predictand for a given range and location only the value of the parameter we are interested in (for example temperature), the method will yield a distribution of its possible values in terms of quantiles. For temperature, the regressions to be performed will give the temperature of the different quantiles, for example 5%, 25% 50%, 75% and 95%, and the median. As predictors, DMO (Aladin/SI) and observations have been used.
J. Bremnes, met.no
Quantile forecasting in practice using local quantile regression
The aim is to forecast quantiles. For example: for a given probability of rain (say: 40%), which will be to forecast the probabilies of precipitation more than a given amount. The method defines a weight for each data point; the most similar data to historical ones should get most weight.
In the training phase, evaluate the quantile function at predictor values by looking at the corresponding observed precipitations.
H. Seidl, ZAMG
Experiences with PPM-methods applied to forecast local and areal amounts of precipitation in Austria
PPM = Perfect Prognosis Method
Firstly the observed precipitations (6-hourly and 12-hourly accumulated) are interpolated onto a regular grid with height correction (the increase of precipitation with height is deduced from the seasonal climatology of Austria).
Method for the Areal PPM:
1. Determination of a multiple regression equation for each area.
Predictors values are taken from ECMWF archived analyses for the grid point most representative of the area considered. Corresponding areal precipitations are taken from the observation data base. Then the regression coefficients are computed. This work must be done once for each area.
2. Operational procedure: From the ECMWF forecast fields, the same predictors are extracted and inserted into the multiple regression equation.
Next to this PPM for area precipitation forecasts, ZAMG has also in a similar way developed a PPM for forecasting local amounts of precipitation, but the method has a tendency to underestimate severe precipitations. This has been alleviated with the "MAXMIN method" which, according to the values of the parameters (predictors), will compute the precipitations either in a maximum mode or in a minimum mode. This method deteriorates the classical scores as MAE or RMSE, but the HR for strong precipitations is increased, concomitantly with an acceptable increase of the FAR.
G. Csima, HMS
Using multiple linear regression for post-processing model output data
At the HMS, forecasts of the 00 and 12 UTC of the ECMWF and ALADIN models are 3-hourly processed between +12 / +60 for ECMWF and +3 / +48 for ALADIN.
The post-processing is done by multiple linear regression at every SYNOP station. There is a different set of regression equations for each month.
The predictors are: MSLP, T2m, RH2m, U10m, V10m, N. At 925, 850,700 and 500 hPa: Z, T, U, V, RH. Together 26 predictors.
The selection of the predictors for the linear regression equations has been done by forward selection. The quality of the procedure has been assessed by the ANOVA method (ANalysis Of VAriance).
In simple linear regression mode for the T2m, it is with 13-14 predictors that the reduction of the variance of the errors - the difference between the computed temperatures and their corresponding observations - is maximum. By adding supplementary predictors, the reduction of variance decreases.
For the full test period (Jan, Feb, Mar 2005) for the T2m as predictand, the most significant model corrections took place in mountainous and hilly terrain.
P. Crochet, Icelandic Meteorological Office
Prediction of T2m and 10m wind-speed in Iceland using a Kalman filter
A precise mathematical description of the Kalman filter technique has been presented by the author. The author presented results obtained with quantification of the noise statistics (in many NWS, Kalman filter is used with an observation noise sets equal to zero).
A. Persson, SMHI
From 2-D to 3D Kalman filtering of NWP output
The author claims that Kalman filtering of a two-dimensional expression (2D-KF) not only corrects the bias but also the variance of the forecasts.
This can be understood if we consider the correction C = A + Bx where x is the forecast and A (the "bias") and B (the "slope" and therefore the variance) which are recursively estimated by the Kalman filter. The improvement in variance meant that an underforecasting of extreme events such as temperature < -20° was almost avoided.
This has been very well shown by the HIRLAM 2m temperature and 10m wind speed corrections for different observing stations in Sweden. The author has also compared the application of a 2D- versus a 3D-KF to the temperatures of a very cold observing station. Account was now taken also of the forecast with respect to the last available observation. This type of 3D-KF decreased the RMSE, but had - compared to the 2D-KF - a dampening influence with fewer forecasts with temperature < -20°.
If the anomalies are used instead of the full values, a large change in B will not necessarily imply large changes in A as well and the Kalman filter will achieve a quicker adaptation.
J.M. Gutierrez et al., University of Cantabria (Spain)
Multisite Spatial Analog Downscaling Methods with Bayesian Networks
The main point of this presentation was to show the use of the Bayesian Networks (BN) as a non-linear statistical downscaling method linking the ERA-40 grid point values over Iberia with the local observations.
BM are very popular in several fields (as biology and medicine), but are just starting to be used in meteorology.
BN have the advantage to be global in the sense that only joint probabilities encompassing all the variables (large-scale and small-scale) can be computed. The graphs defining the dependence structures do not separate for the downscaling process the large scale (ERA-40 grid-point values) from the local observations, as other downscaling methods - for example the multiple regression - do it.
The authors have not restricted themselves to the case of singly connected networks over Spain (ie. networks with only one path between any two nodes). This limitation would have rendered the search of a solution for the joint probabilities much easier.
This presentation can be considered as a break-through in the field of the meteorological statistical downscaling.