Skip to content

Validation fails silently in Anemoi training when time-range does not span the data #511

@manvendra9099

Description

@manvendra9099

Describe the bug
We found that validation doesn't run at all (without any warning/error) when dataloader.validation.start/end dates don't overlap with the actual data in the dataset

Example:

dataloader:
  validation:
    start: 2020-07-01 00:00:00   # <-- worked
    # start: 2021-01-01 00:00:00 # <-- silently skips validation
    end: 2020-07-31 23:59:00 
    # end: 2021-12-31 23:59:00

Run training with this config. The validator does not run and logs no validation metrics or warnings.

Actual behaviour:
Training runs normally
No validation occurs
No warning or error is logged

Expected behavior:
When the validation range yields zero samples: Throw an error or at least a warning early in the dataloader setup
Include in the message:

  • requested validation.start/end
  • dataset actual coverage (tmin/tmax)

Recommended fix
Raise a ValueError in _dates_to_indices if the requested date range yields zero samples, with a message showing the requested and dataset date ranges.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Status

To be triaged

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions