Feature/gcafsv1 requirements

[bbakernoaa]

Description

This pull request introduces a new pyproject.toml file to establish standardized Python packaging and dependency management for the project. It defines core and optional dependencies, project metadata, and testing and documentation requirements. Additionally, it updates the requirements_gcafs.txt file to align package naming conventions with PEP 503 and removes local editable installs for certain packages.

Python packaging and dependency management:

• Added a new pyproject.toml file, specifying project metadata, core dependencies, optional dependency groups, test and documentation requirements, and build system configuration. This enables modern Python packaging workflows and standardizes dependency management.
• Defined package structure and test configuration in pyproject.toml, including package directories, package list, and pytest options.

Dependency updates and cleanup:

• Updated requirements_gcafs.txt to use PEP 503-compliant lowercase package names (e.g., jinja2, markupsafe, netcdf4, pyyaml) for consistency and compatibility.
• Removed jcb and wxflow from requirements_gcafs.txt, as these are now handled as local editable installs per the note in pyproject.toml

This gives us the maximum flexibility for our development and CI, as well as a way to generate requirements.txt files for NCO implementations

pyproject.toml usage

In the pyproject.toml there are several sections to install. If you simply do a pip install . it will only install the core dependencies. This can be seen in https://github.com/bbakernoaa/global-workflow/blob/feature/gcafsv1_requirements/pyproject.toml#L20-L34

The optional sections can be seen here https://github.com/bbakernoaa/global-workflow/blob/feature/gcafsv1_requirements/pyproject.toml#L36-L69

• To install all dependencies for all applications you can simply do pip install .[all].
• To only install a specific application pip install .[gcafs]
• To install multiple extra dependencies pip install .[gcafs,testing,docs]

Example for transition

To generate the requirements files use pip-compile within pip-tools. For an example of executing this for deployment

pip-compile --extra gcafs -o versions/requirements_gcafs.txt pyproject.toml --no-header --no-annotate

This generates a new file in versions/requirements_gcafs.txt.

Going forward, if applications need a specific requirement that isn't needed by other applications they can add it under their specific section.

Type of change

• Bug fix (fixes something broken)
• New feature (adds functionality)
• Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

• Is this change expected to change outputs (e.g. value changes to existing outputs, new files stored in COM, files removed from COM, filename changes, additions/subtractions to archives)? YES/NO (If YES, please indicate to which system(s))
  • GFS
  • GEFS
  • SFS
  • GCAFS
• Is this a breaking change (a change in existing functionality)? YES/NO
• Does this change require a documentation update? YES/NO
• Does this change require an update to any of the following submodules? YES/NO (If YES, please add a link to any PRs that are pending.)
  • EMC verif-global
  • GDAS
  • GFS-utils
  • GSI
  • GSI-monitor
  • GSI-utils
  • UFS-utils
  • UFS-weather-model
  • wxflow

How has this been tested?

Checklist

• Any dependent changes have been merged and published
• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have documented my code, including function, input, and output descriptions
• My changes generate no new warnings
• New and existing tests pass with my changes
• This change is covered by an existing CI test or a new one has been added
• Any new scripts have been added to the .github/CODEOWNERS file with owners
• I have made corresponding changes to the system documentation if necessary