-
Notifications
You must be signed in to change notification settings - Fork 8
Open
Description
I was testing Quantile Skill Score in the reports and came across some results that initially confused me. Table below shows metrics for 4 pairs of forecasts
Forecast | BS | QS | BSS | QSS |
---|---|---|---|---|
Table Mountain Boulder CO Day Ahead GEFS ghi Prob(f <= x) = 0.0% | 0.46 | 46.3 | -0.113 | 0.653 |
Table Mountain Boulder CO Day Ahead GEFS ghi Prob(f <= x) = 40.0% | 0.282 | 39.6 | -0.161 | 0.688 |
Table Mountain Boulder CO Day Ahead GEFS ghi Prob(f <= x) = 50.0% | 0.25 | 44.7 | 0.00e+00 | 0.643 |
Table Mountain Boulder CO Day Ahead GEFS ghi Prob(f <= x) = 100.0% | 0.264 | 93.4 | 0.55 | 0.201 |
Table Mountain Boulder CO Hour Ahead Prob Persistence ghi Prob(f <= x) = 0.0% | 0.415 | 129 | nan | nan |
Table Mountain Boulder CO Hour Ahead Prob Persistence ghi Prob(f <= x) = 40.0% | 0.243 | 122 | nan | nan |
Table Mountain Boulder CO Hour Ahead Prob Persistence ghi Prob(f <= x) = 50.0% | 0.25 | 121 | nan | nan |
Table Mountain Boulder CO Hour Ahead Prob Persistence ghi Prob(f <= x) = 100.0% | 0.585 | 112 | nan | nan |
I came away from this wanting a few things in the brier score documentation
- a statement about how the Arbiter computes o for non-event forecasts (is f <= x?).
- some discussion about brier score vs. quantile score for quantile forecasts (like all of our reference forecasts as of today).
Metadata
Metadata
Assignees
Labels
No labels