Adrian E Raftery
Department of Statistics
University of Washington
We consider the problem of calibrated probabilistic mesoscale forecasting of
a single future meteorological quantity. We define this as the specification
of a probability distribution of the quantity of interest which is both
calibrated and sharp. By calibrated, we mean that if we define a probability
interval, such as a 90% probability interval, then on average in the long
run, 90% of such intervals contain the true value. By sharp, we mean that
the distribution is more concentrated than forecast distributions from
climatology alone.
Mass and colleagues have been developing
an ensemble mesoscale forecasting system based on runs of MM5 initialized
using different global models. They have established a clear spread-skill
relationship, but the resulting forecast intervals are not calibrated; they
are too narrow.
We apply Bayesian Model Averaging (BMA; Hoeting et al, 1999, Statistical
Science), to develop calibrated probability forecasts using the same
ensemble that underlies the Mass system. Bayesian Model Averaging is a
formal statistical framework for combining probability forecasts from
different models in a calibrated way, taking account of the models' past
forecasting performance. The theory of Bayesian Model Averaging explains and
predicts two of the empirical findings from the Mass group: the spread-skill
relationship, and the fact that the intervals from the Mass ensemble are too
narrow on average.
We calculate BMA forecast distributions of temperature and sea-level
pressure 48 hours ahead in winter in the Pacific Northwest region for
January-June 2000. The resulting forecasts are well calibrated and are also
substantially sharper than calibrated forecasts from climatology. These
preliminary results suggest that BMA is successful at providing
probabilistic forecasts that are both calibrated and sharp. We will mention
plans to optimize various aspects of the method's implementation, and to
incorporate it in the UW online ensemble system.
This is joint work with Tilmann Gneiting, Fadoua Balabdaoui and Michael
Polakowski.