Forecast skill

[ecomodeller]

Verif

is a command-line tool that lets you verify the quality of weather forecasts for point locations. It can also compare forecasts from different forecasting systems (that have different models, post-processing methods, etc).

The program reads files with observations and forecasts in a specific format (see "Input files" below). The input files contain information about dates, forecast lead times, and locations such that statistics can be aggregated across different dimensions. To ensure a fair comparison among files, Verif will discard data points where one or more forecast systems have missing forecasts. Since Verif is a command-line tool, it can be used in scripts to automatically create verification figures.

Features

• Deterministic metrics such as MAE, bias, correlation, RMSE (e.g. -m mae)
• Threshold-based metrics such as the false alarm rate, ETS, EDI, Yule's Q (e.g. -m ets)
• Probabilistic metrics such as brier score, PIT-histogram, reliability diagrams (e.g. -m bs)
• Special plots like Taylor diagrams (-m taylor), quantile-quantile plots (-m qq).
• Plot scores as a function of date, lead time, station altitude/lat/longitude (e.g. -x date)
• Show scores on maps (-type map)
• Subset the data by specifying a date range and lat/lon range (-latrange 58,60)
• ...

[jsmariegaard]

Super nice, but out-of-scope for this library IMO.