Skip to content

fix(create): restore --test flag for dataset creation#570

Open
pontuselmrin wants to merge 3 commits intoecmwf:mainfrom
pontuselmrin:fix/test_flag_cli
Open

fix(create): restore --test flag for dataset creation#570
pontuselmrin wants to merge 3 commits intoecmwf:mainfrom
pontuselmrin:fix/test_flag_cli

Conversation

@pontuselmrin
Copy link

Description

This fix restores the --test flag for anemoi-datasets create which was removed during a prior refactor. By using this flag, the dataset that is being created is reduced to 4 dates; the grid is coarsened and any ensembles or constants are trimmed. This allows for quick validation of the create pipeline without large data downloads.

What problem does this change solve?

Bugfix. The --test flag was removed during a refactor of create.py. This restores the flag in the CLI by re-instating the set_to_test_mode() function and adds unit tests for its functionality.

What issue or task does this change relate to?

#546

Additional notes

  • Added two unit tests
  • The change only affects the init part of the pipeline for creating a dataset; hence multiple GPUs scenario doesn't apply
  • The code is reinstated from previous commits, ie I tried to reproduce the previous state.
  • The values for grid, constants, ensemble and number of tests in the set_to_test_mode() are hard-coded values. Perhaps this could be improved in a future feature

By opening this pull request, I affirm that all authors agree to the Contributor License Agreement.

@github-project-automation github-project-automation bot moved this to To be triaged in Anemoi-dev Mar 11, 2026
@pontuselmrin pontuselmrin changed the title Fix/test flag cli fix(create): restore --test flag for dataset creation Mar 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: To be triaged

Development

Successfully merging this pull request may close these issues.

1 participant