Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

As a user, I don't expect to see an UNKNOWN domain axis title when using event detection #390

Open
HankHerr-NOAA opened this issue Jan 24, 2025 · 4 comments
Labels
bug Something isn't working

Comments

@HankHerr-NOAA
Copy link
Contributor

The specific situation is that observed are NWM v3.0 retrosim, predicted are WRDS RFC forecasts, and covariates list one covariate for HEFS observations. The declaration is below. This resulted in the image shown in the comment,

#382 (comment)

The domain axis label is, "UNKNOWN [UTC]". In my opinion, that is sort of equivalent to an NPE: it should never happen. Hence, this ticket. If I'm wrong, and unknown is reasonable for this axis label, please explain.

All data for this evaluation is in the GitHub_382 folder within `WRES Redmine Ticket Large Data Sets' on Google Drive. The retrosim data is a bit large, but I can attach the other two files to this ticket if desired.

Thanks,

Hank

==========

label: Testing Event Based
observed:
  label: "19161749 RetroSim CSVs"
  sources:
  - /home/ISED/wres/nwm_3_0_retro_simulations/wfo/OAX/19161749_nwm_3_0_retro_wres.csv.gz
  variable: streamflow
  feature_authority: nwm feature id
  type: simulations

predicted:
  label: AHPS Streamflow Forecast
  sources:
    - uri: https://WRDS/api/rfc_forecast/v2.0/forecast/streamflow
      interface: wrds ahps
  variable: QR
  type: single valued forecasts

covariates:
  - label: OBS streamflow covariate
    sources: /home/ISED/wres/wresTestData/issue92087/inputs/ABRN1_QME.xml
    variable: QME
    feature_authority: nws lid
    type: observations
    time_scale:
      function: mean
      period: 24
      unit: hours

features:
 - {observed: '19161749' , predicted: 'ABRN1' }
 - 
time_scale:
  function: mean
  period: 24
  unit: hours

reference_dates:
  minimum: 2000-08-07T23:00:00Z
  maximum: 2015-08-08T23:00:00Z

event_detection: covariates

metrics:
  - pearson correlation coefficient
  - sample size
  - name: time to peak error
    summary_statistics:
    - median
    - minimum
    - maximum
    - mean

output_formats:
  - csv2
  - png
@HankHerr-NOAA HankHerr-NOAA added the bug Something isn't working label Jan 24, 2025
@HankHerr-NOAA
Copy link
Contributor Author

Saw this in #382:

Odd, the UNKNOWN means that the reference time type is ReferenceTimeType.UNKNOWN for some reason, not ReferenceTimeType.ISSUED_TIME, for example, so unsure why the type was not correctly resolved for these forecasts.

Oh. Then "UNKNOWN" is not unreasonable if the reference time type is actually unknown. In this case, the predicted data are from WRDS, so it should in theory be known.

Hank

@HankHerr-NOAA
Copy link
Contributor Author

Now I'll be curious if you will struggle to reproduce, given I'm presenting the data in a JSON instead of obtaining it from WRDS.

Hank

@HankHerr-NOAA
Copy link
Contributor Author

I dug up the time to peak error results for an evaluation of WRDS RFC forecasts against HEFS observations using event detection, and the domain access was correctly labeled as "ISSUED_TIME".

The domain axis was also correct if I had HEFS observations for observed and HEFS observations repeated for covariates and used the covariates for event detection.

It must somehow be related to the use of NWM v3.0 retrosim evaluations as observed, though I'm not sure why. I think that is the only change from the previously described evaluation.

Hank

@james-d-brown
Copy link
Collaborator

Probably fix here is twofold: 1) determine why the reference time type is not being resolved for these time-series; and 2) when this situation arises, at least qualify the UNKNOWN a little better, i.e., REFERENCE TIME.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants