Update to latest submodule versions and changes to develop branch#183
Draft
fmalatino wants to merge 77 commits intoNOAA-GFDL:benchmarkfrom
Draft
Update to latest submodule versions and changes to develop branch#183fmalatino wants to merge 77 commits intoNOAA-GFDL:benchmarkfrom
develop branch#183fmalatino wants to merge 77 commits intoNOAA-GFDL:benchmarkfrom
Conversation
…mutually incompatible versions (NOAA-GFDL#84)
* updated generate_eta_files * remove notebook, add script * lint --------- Co-authored-by: mlee03 <[email protected]>
* Updates to allow use of python3.11 * Removed main_unit_tests.yml as it is covered by tests in containers * Updated images for unit tests with openmpi and mpich
* updated generate_eta_files * remove notebook, add script * lint * correct path * update constraints * python3 execute * Update main_unit_tests_mpich.yaml * Update main_unit_tests_mpich.yaml * update input directory * get ..s correct * lint --------- Co-authored-by: mlee03 <[email protected]>
…A-GFDL#89) * test * tet * fix test dir * test * checkout pace * rename * test * rm * remove dir * fix name * test * pyshield * test * fix trigger mistake * test * change lint version * remove push * lint --------- Co-authored-by: mlee03 <[email protected]>
…tdated pull request template (NOAA-GFDL#86) * Changed workflows to use python 3.11.7, added pytest-subtests to requirments_dev.txt * Matched package dependencies between main and develop branches * Updating constraints file * Updating Dockerfile to use 3.11-slim-bookworm base python image
* fix trigger mistake * fix spacking * update yaml files * rename lint.yaml * remove backup yaml file * simply pull request trigger * beloved lint --------- Co-authored-by: mlee03 <[email protected]>
…FDL#88) * add build script for intel oneapi and gcc compiler separately * update the python version and remove the link time optimization option for gcc
…NOAA-GFDL#92) * Removed test_eta.py, moved to NDSL/tests/grid
…FDL#99) * Updated README to include directions for generating eta files * Linting * Updated baroclinic_c2.yaml to generate zarr output
…DL#101) * Updated submodules as of 21 Jan 2025, and added baroclinic run to github workflow
* Update netcdf version * also upgrade pip * update ndsl
* Updated submodules to versions as of 15 Jan 2025 * Updated submodules as of 21 Jan 2025, and added baroclinic run to github workflow * Updated submodules as of 5 09:49 March 2025
…OAA-GFDL#105) * Added configurable options for precision of NetCDF output * Added in method for setting NetCDF output precision * Updated NDSL to use NetCDF precision setting feature
* Unrelated: consistent markdown formatting Consistent markdown formatting in the top-level README file. * More md cleanups - consistent whitespace around headers and code blocks - fenced code blocks - for shell commands, only include `$` if you also show the output --------- Co-authored-by: Roman Cattaneo <[email protected]>
Starting numpy 1.25.0, `np.product()` is deprecated and `np.prod()` should be used instead. Co-authored-by: Roman Cattaneo <[email protected]>
…ositories (NOAA-GFDL#109) * Don't use constraints.txt in NDSL pipeline * Avoid installing zarr v3 zarr v3 comes with breaking API changes that will fail tests. * Updated list of contributors --------- Co-authored-by: Roman Cattaneo <[email protected]>
* Don't upgrade pip, setuptools, wheels * WIP: dependencies handling * fix linting workflow * fix linting even more * remove editable installs of NDSL/pyFV3/pySHiELD in dev setup * Use pip install -e .[test,lint] directly instead of requirements_dev.txt * forgot to push some changes * re-adding constraints.txt file * add constraints files for dev and test setup * fix typo in readme * Remove hard-coded paths from constraints file * Keep docs/scripts consistent * no need for pip list anymore (we have constraints file now) * actually install extras ;) * Remove constraints* files again * for easier (future) debuggging: print versions of installed packages * one last (tm) update to the readme --------- Co-authored-by: Roman Cattaneo <[email protected]>
…anges (NOAA-GFDL#119) * Changes to Dockerfile and Makefile to reflect most recent changes to install method * Setting version of matplotlib in Dockerfile to be 3.10.0 * Installing using test hook in Dockerfile
* removed direct gt4py imports * linting * added nametag to gtscript imports * linting * changed @function to @gtfunction * new NDSL version * new NDSL branch * new NDSL version, removing extra imports in ndsl.dsl.gt4py that causes crash during tests * new NDSL version... again * Added name to CONTRIBUTORS.md
Newer GT4Py versions come with vendored-in boost headers. NDSL picked up this newer boost version and thus pace doesn't need boost in the install notes anymore. Co-authored-by: Roman Cattaneo <[email protected]>
* tests: Use pathlib.Path for eta_files * Udpate more paths from str to Path * Revert changes to pace/comm.py and pace/configs/comm.py --------- Co-authored-by: Roman Cattaneo <[email protected]>
Removes `pace/config/comm.py` in favor of `pace/comm.py`, which look like duplicates, where `pace/comm.py` is the one actually used and developed. Also deleting `pace/config/__init__.py` since it was the only file left in the folder. Co-authored-by: Roman Cattaneo <[email protected]>
…CTools (NOAA-GFDL#127) * Updating external grid read-in method to match output from FRENCTools * Updating external grid read-in method to take correct data chunk from file per rank * Using the correct order of dims and indices for chunking data, instead of previous method for external grid data read in * Changing external grid data test configs data file names to match number of grid points used * Added check for ordering of input grid data file * Fixing base file name in test_external_grid * Adding annotations regarding the need to stride input grid data due to supergrid format
* let's see if any PR will break CI * front-load numpy * replace miniconda with plain python 3.11 * cleanup and todo for further cleanup --------- Co-authored-by: Roman Cattaneo <[email protected]>
… PyFV3 PR NOAA-GFDL#88] (NOAA-GFDL#152) * Tmp dycore dacite config related to PyFV3 PR NOAA-GFDL#88 + NDSL Issue NOAA-GFDL#64 * test_config.py tweak for type checks --------- Co-authored-by: Florian Deconinck <[email protected]>
… `dataclass` (NOAA-GFDL#154) * Use `fom_tile_params` instead of direct constructor of dataclass * Typo * Typo + one more normalizing * Missing `layout` kwarg
…OAA-GFDL#153) * Tmp physics dacite config related to NDSL Issue NOAA-GFDL#64 Clean up after pyFV3 PR NOAA-GFDL#88 * Updating pyFV3 submodule * Updating pyFV3 submodule after PyFV3 PR NOAA-GFDL#90
…DL#156) This PR updates NDSL to a few commits after version 2025.10.00 and syncs pyFV3 and pySHiELD to their current `develop` branch. The release version of NDSL was raisning type issues that were only caught in pace. We are working on streamlining this experience for future releases of NDSL. Co-authored-by: Roman Cattaneo <[email protected]>
* refactor: extra_dim_lengths -> data_dimensions
Change `extra_dim_lengths` -> `data_dimensions` when creating a
`SubtileGridSizer`. We can remove empty dictionaries `{}` because that's
now the default. Previously that default wasn't re-entrant so many
places specified the option explicitly. Now there's no need anymore for
this.
* refactor: drop dependency on deprecated ndsl.filesystem
---------
Co-authored-by: Roman Cattaneo <[email protected]>
* Removing ndsl.Namelist * Cleanup from pySHiELD PR 68
…OAA-GFDL#159) * Fixes for Dockerfile to specify NDSL version and black specification * Testing Dockerfile changes, will be undone * Undoing changes to test Dockerfile build
* Update to 2025.11.00 RC * Update to 2025.11.00 released (tag)
NOAA-GFDL#163) * Amending Makefile to match testing data downloads of submodules * Downloading all necessary files through the Makefile now * Separating getting standard and physics data, and creating method for getting both. * Simplifying get_test_data in Makefile
…DL#167) `get_tile_number()` is deprecated and replaced by the zero-index version called `get_tile_index()`. This removes a `FutureWarning`. Co-authored-by: Roman Cattaneo <[email protected]>
Prefer the new constructor of `QuantityFactor` over the deprecated call to `QuantityFactor.from_backend(...)`. This removes a bunch of deprecation warnings in tests. Co-authored-by: Roman Cattaneo <[email protected]>
On `Quantities`, we currently allow both `backend` and `gt4py_backend` where the later is deprecated and about to be removed. See NOAA-GFDL/NDSL#312 and NOAA-GFDL/NDSL#314 for context. Co-authored-by: Roman Cattaneo <[email protected]>
We decided to remove `NullComm` from NDSL in favor of `MPIComm` and `LocalComm`, see NOAA-GFDL/NDSL#318 for context. pace exposes a `NullCommConfig` and is heavily relying on `NullComm` in tests. We thus suggest to move `NullComm` to `pace/comm.py`. Co-authored-by: Roman Cattaneo <[email protected]>
This PR updates pyFV3 and pySHiELD submodules in pace to include the latest changes. These changes are necessary to fully remove deprecated code in NDSL (which is planned for the upcoming 2026.01.00 release). We are also temporarily skipping `mypy` checking for `translate_fvdynamics.py`. There are type issue in the translate test system which we will address in the next release of NDSL. Once we update pyFV3 to NDSL version 2026.01.00, we should be able to remove that exception again. For context see NOAA-GFDL/NDSL#360. Co-authored-by: Roman Cattaneo <[email protected]>
Co-authored-by: Roman Cattaneo <>
Co-authored-by: Roman Cattaneo <[email protected]>
…GFDL#177) * refactor: get rid of deprecation warnings from NDSL 2026.01.00 * fixup: keep line beaks as-is --------- Co-authored-by: Roman Cattaneo <[email protected]>
With NDSL supporting more than one version of python, we'd like to have the possibility to run pyFV3 translate tests with different versions of python. This PR adds an optional input variable to the `workflow_call` which allows calling workflows to specify the python version. The PR also removes usage of the `component_trigger` input variable. The input is redundant with `component_name`, which is a non-empty string whenever `component_trigger` is true. I keep the input around until we update all calling workflows to not set it anymore. Co-authored-by: Roman Cattaneo <>
…DL#180) * [TMP] Move NDSL to `NDSL_Backend` and update CI external trigger * [TMP] Move submodules to `update/2026.02.00` worthy branches * Move to `Backend` object * Lint * X/Y/Z_DIM -> I/J/K_DIM * update pyFV3 and pySHiELD to latest version of 2026.02.00 branch * update pyFV3 (Backend in GEOSDycoreWrapper) * ci: NDSL 2026.02.00 and switch to python 3.12 * update pyFV3 and pySHiELD to latest develop This propagates the breaking changes from NDSL 2026.02.00 * removed debug print of config --------- Co-authored-by: Roman Cattaneo <[email protected]>
* tests: minor cleanup in tests - add `__init__.py` files where missing - `List` -> `list` and `Tuple` -> `tuple` - NDSL has no `Namelist` anymore - cleanup unused import and code in `test_init_from_geos.py` * remove stale reference to ndsl.Namelist
* Changing submodule names to reflect new repository names, and updating to latest versions * Updating linting tools * Re-adding back in flake8 checks * removing comments * Adding in type hinting for hooks variable in from_dict methods * Updating submodules * Removing optional None type hints * Reverting back to full typing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Include a summary of the change and which issue is fixed. Please also include
relevant motivation and context. List any dependencies that are required for
this change.
Fixes # (issue)
If this is a hotfix to a released version, please specify it.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Please also note
any relevant details for your test configuration (e.g. compiler, OS). Include
enough information so someone can reproduce your tests.
Checklist: