Skip to content

Creates separate output directories for workflow run in pipeline associated with runner name#160

Draft
singhd789 wants to merge 105 commits intomainfrom
update-ppp-pipeline
Draft

Creates separate output directories for workflow run in pipeline associated with runner name#160
singhd789 wants to merge 105 commits intomainfrom
update-ppp-pipeline

Conversation

@singhd789
Copy link
Copy Markdown
Contributor

@singhd789 singhd789 commented Sep 25, 2025

  • creates output dir based off runner number (to associate with runner if there are multiple running)
  • (to-do: ensures there's no race condition for 2 runners accessing same output directory)
  • could potentially solve permission denied errors we see every once in a while with the ppp-setup folder in the cloud?

Ciheim brought in this branch in his fork, created 2 different runners, and started them around the same time:

BUT - in those pipelines, the workflow took 3 HOURS AND 33 MINUTES, no bueno
Currently testing running 2 at the same time but accessing different history file locations

Here is successful output from a pipeline run:
https://github.com/NOAA-GFDL/fre-workflows/actions/runs/20470921836/job/58825830669

@singhd789 singhd789 changed the title Creates output directory for workflow run in pipeline Creates separate output directories for workflow run in pipeline associated with runner name Nov 18, 2025
@singhd789 singhd789 marked this pull request as draft November 18, 2025 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants