A design of 4 cycled ecflow definition to be used for NCO SPA tag up meeting by lgannoaa · Pull Request #4652 · NOAA-EMC/global-workflow

lgannoaa · 2026-03-12T14:02:24Z

This draft PR is for NCO SPA to checkout in our weekly meeting.

…meeting.

RussellManser-NCO

Hi @lgannoaa, thanks for providing this PR for review. I have commented on specific jobs and families that could be modified. I have some more general comments and questions as well.

I am still considering how prep and init families should be organized. Here are some of my related observations:

It appears that multiple jobs and families depend on prep and init jobs
Some jobs write to e.g., gfs.$PDY/$cyc/obs and others to directories specific to a step and component. For example, gfs/init/marine/jgfs_marine_bmat_init writes to gfs.$PDY/$cyc/bmatrix/{ocean,ice}
Ultimately, I am not sure if it makes sense to reorganize these families to align with the COM structure

Related to the above, I started discussing the ecFlow design with my colleagues on the SPA team. Many of them have more extensive historical context and expertise that may help me provide better guidance. I will pass along feedback as I receive it.

Finally, I only reviewed the 00 cycle. I expect that 06, 12, and 18 are extremely similar, so my comments apply to those cycles as well. If my assumption is wrong, please let me know.

RussellManser-NCO · 2026-03-12T14:29:37Z

ecf/defs/gfs_prod.def

+            task jgfs_atmos_prep_fsm
+              edit RJN 'prep'
+              trigger :TIME >= 0300 and :TIME < 0330
+              event 1 release_gfs_atmos_prep


Does this task perform any work? From what I can see it triggers jgf_atmos_prep between 0300Z and 0330Z. If that is the only work this job is doing, I recommend the following:

Include the time triger in jgfs_atmos_prep

Remove jgfs_atmos_prep_fsm

@lgannoaa We discussed that this job will remain while EMC is running GFS v17. When NCO starts running GFS v17, we may revisit this job to see if an ecFlow trigger is more appropriate.

RussellManser-NCO · 2026-03-12T14:31:52Z

ecf/defs/gfs_prod.def

+            task jgfs_marine_prep_fsm
+              edit RJN 'prep'
+              trigger :TIME >= 0300 and :TIME < 0330
+              event 1 release_gfs_marine_prepoceanobs


Similar to the previous comment. If the purpose of jfs_marine_prep_fsm is only to trigger jgfs_marine_prepoceanobs, you can

Move the time trigger to gfs_marine_prepoceanobs

Remove jgfs_marine_prep_fsm

@lgannoaa We discussed that this job will remain while EMC is running GFS v17. When NCO starts running GFS v17, we may revisit this job to see if an ecFlow trigger is more appropriate.

RussellManser-NCO · 2026-03-12T14:36:49Z

ecf/defs/gfs_prod.def

+          task jgfs_fcst
+            trigger ../analysis/atmos==complete and ../init/marine==complete and ../analysis/marine==complete


From discussion on the ecFlow review slides that I shared, I am inferring that jgfs_fcst requires one or more files from prep/wave/jgfs_wave_init. If that is correct, please add a trigger for prep/wave/jgfs_wave_init. CC @JessicaMeixner-NOAA

Yes, we need files from jgfs_wave_init for the forecasts.

Thank you @JessicaMeixner-NOAA. I think adding a trigger for prep/wave/jgfs_wave_init would be beneficial then, @lgannoaa.

RussellManser-NCO · 2026-03-12T14:57:30Z

ecf/defs/gfs_prod.def

+        family post
+          family wave


From logs at /lfs/h2/emc/gfstemp/emc.global/comroot/retrov17_01_realtime/logs/, it looks like jobs in this family write to gfs.$PDY/$cyc/products/wave/. Would it be appropriate to move the gfs/post/wave family to gfs/products/wave?

This change also seems more consistent with families such as gfs/products/ocean and gfs/products/ice.

Confirming wave_postsbs writes to gfs/products/wave

Thank you @JessicaMeixner-NOAA. After discussion with @lgannoaa today, we agreed that this change isn't necessary for an EMC ecFlow parallel run.

RussellManser-NCO · 2026-03-12T16:25:53Z

ecf/defs/gfs_prod.def

+        family gempak
+          family atmos
+            edit WGF 'atmos'
+            family gempak


Could this family gfs/gempak/atmos be moved to gfs/products/atmos/gempak? From logs files at /lfs/h2/emc/gfstemp/emc.global/comroot/retrov17_01_realtime the jobs seem to be writing to gfs.$PDY/$cyc/atmos/products/gempak/.

Similar to comments above, this change is not necessary at this time.

RussellManser-NCO · 2026-03-12T16:38:52Z

ecf/defs/gfs_prod.def

+            task jgdas_marine_prep_fsm
+              edit RJN 'prep'
+              trigger :TIME >= 0300 and :TIME < 0330
+              event 1 release_gdas_marine_prepoceanobs


Does jgdas_marine_prep_fsm perform any work besides setting an event to trigger jgdas_marine_prepoceanobs? If not, you can

Move the time trigger to jgdas_marine_prepoceanobs

Remove jgdas_marine_prep_fsm

We will keep *_fsm jobs for now. We will re-evaluate if a trigger is more appropriate when NCO starts running GFS v17.

RussellManser-NCO · 2026-03-12T19:25:44Z

ecf/defs/gfs_prod.def

+            event 207 release_gdas_wave_postsbs_f007
+            event 208 release_gdas_wave_postsbs_f008
+            event 209 release_gdas_wave_postsbs_f009
+          task jgdas_fcst


Are output files produced by gdas/init/wave/jgdas_wave_init required for jgdas_forecast? If so, please add a trigger here on gdas/init/wave/jgdas_wave_init. CC @JessicaMeixner-NOAA

Yes, output from gdas/init/wave/jgdas_wave_init is needed for jgdas_forecast

Thank you @JessicaMeixner-NOAA. I think adding a trigger for gdas/init/wave/jgdas_wave_init would be beneficial then, @lgannoaa.

RussellManser-NCO · 2026-03-12T19:31:00Z

ecf/defs/gfs_prod.def

+        family post
+          family wave


From logs at /lfs/h2/emc/gfstemp/emc.global/comroot/retrov17_01_realtime/logs/ it looks like these jobs write to gdas.$PDY/$cyc/products/wave/[gridded,station]. Does it make sense to move gdas/post/wave to gdas/product/wave?

wave_post_sbs - > gdas.$PDY/$cyc/products/wave/[gridded]

there are wave post point jobs that write to gdas.$PDY/$cyc/products/wave/[station]

After discussion with @lgannoaa today, we agreed that this change is not necessary for an EMC ecFlow parallel run.

RussellManser-NCO · 2026-03-12T19:36:52Z

ecf/defs/gfs_prod.def

+        family gempak
+          family atmos


Could this family gdas/gempak/atmos/gempak be moved to gdas/products/atmos/gempak? From logs files at /lfs/h2/emc/gfstemp/emc.global/comroot/retrov17_01_realtime/logs the jobs seem to be writing to gdas.$PDY/$cyc/atmos/products/gempak/.

After discussion with @lgannoaa today, we agreed that this change is not necessary for an EMC ecFlow parallel run.

RussellManser-NCO · 2026-03-12T20:21:35Z

ecf/defs/gfs_prod.def

+                trigger ../../../forecast/jgfs_fcst_fsm:release_gfs_atmos_goesupp_f384
+            endfamily		//goesupp
+          endfamily		//atmos
+          family wave


Perhaps gfs/gempak/wave could be moved to gfs/products/wave/gempak?

After discussion with @lgannoaa today, we agreed that this change is not necessary for an EMC ecFlow parallel run.

RussellManser-NCO · 2026-03-13T17:12:02Z

Thank you for the discussion today, @lgannoaa. This ecFlow definition is in good shape for an EMC ecFlow parallel run. We agreed that there will be future iterations after NCO inspects EMC ecFlow runs. I think the triggers for wave prep/init jobs on forecast jobs are a good idea to add now, but if your workflow is not broken then we can add them later.

A design of 4 cycled ecflow definition to be used for NCO SPA tag up …

8a5f73c

…meeting.

RussellManser-NCO reviewed Mar 12, 2026

View reviewed changes

		task jgfs_fcst
		trigger ../analysis/atmos==complete and ../init/marine==complete and ../analysis/marine==complete

Conversation

lgannoaa commented Mar 12, 2026

Uh oh!

RussellManser-NCO left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RussellManser-NCO commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

RussellManser-NCO left a comment •

edited

Loading

RussellManser-NCO commented Mar 13, 2026 •

edited

Loading