Skip to content

Add parallel I/O output system#713

Open
SeanBryan51 wants to merge 8 commits intomainfrom
add-parallelio-output-module
Open

Add parallel I/O output system#713
SeanBryan51 wants to merge 8 commits intomainfrom
add-parallelio-output-module

Conversation

@SeanBryan51
Copy link
Copy Markdown
Collaborator

@SeanBryan51 SeanBryan51 commented Apr 7, 2026

This change brings in a new output system based on the parallel I/O infrastructure introduced in #706, and is a direct replacement of the previous output module used for offline CABLE. The main motivation behind this is to add MPI support to the serial offline driver, and eventually, to replace the legacy MPI implementation (#358). This also makes progress towards the proposed output redesign (#715) by introducing the underlying data structures needed for its implementation.

The new output system brings in the following enhancements:

  • Optional parallel I/O support via PIO
  • A new "aggregator" class for calculating time aggregations throughout a simulation
  • New data structures for describing output variables which allows for easily adding new diagnostics and/or restarts variables

This change should be brought in after #712.

Type of change

Please delete options that are not relevant.

  • Enhancement
  • New or updated documentation

Checklist

  • The new content is accessible and located in the appropriate section
  • I have checked that links are valid and point to the intended content
  • I have checked my code/text and corrected any misspellings

Testing

  • Are the changes bitwise-compatible with the main branch? If working on an optional feature, are the results bitwise-compatible when this feature is off? If yes, copy benchcab output showing successful completion of the bitwise compatibility tests or equivalent results below this line.

CABLE benchcab runs tested using ifort 2021.10.0.

2026-04-28 06:43:59,566 - INFO - benchcab.benchcab.py:380 - Running comparison tasks...
2026-04-28 06:43:59,593 - INFO - benchcab.benchcab.py:381 - tasks: 168 (models: 2, sites: 42, science configurations: 4)
2026-04-28 06:46:47,366 - INFO - benchcab.benchcab.py:391 - 0 failed, 168 passed

Please add a reviewer when ready for review.


📚 Documentation preview 📚: https://cable--713.org.readthedocs.build/en/713/

@SeanBryan51 SeanBryan51 changed the title Add parallelio output module Add new parallel I/O output system Apr 7, 2026
@SeanBryan51 SeanBryan51 changed the title Add new parallel I/O output system Add parallel I/O output system Apr 7, 2026
@SeanBryan51 SeanBryan51 force-pushed the add-parallelio-output-module branch from 4c009fd to b226a95 Compare April 8, 2026 15:29
@SeanBryan51 SeanBryan51 force-pushed the add-parallelio-output-module branch 8 times, most recently from beefeab to 0d53f58 Compare April 13, 2026 19:05
@SeanBryan51 SeanBryan51 force-pushed the add-parallelio-output-module branch from 458d67d to df516f5 Compare April 27, 2026 19:31
@SeanBryan51 SeanBryan51 marked this pull request as ready for review April 27, 2026 19:49
@SeanBryan51
Copy link
Copy Markdown
Collaborator Author

Hi @abhaasgoyal @Whyborn, thank you for your time in reviewing these PRs! This one is now ready for review, and no rush of course, it's the heaviest one so far!

As an introduction, it might be useful to read these developer documentation pages:

Benchcab is running currently - I had to make a small change to achieve bitwise compatibility, I will update the status soon.

Thank you again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant