Zero-diff changes to sfc updates, to speed up v17

[ClaraDraper-NOAA]

Changes to speed up the sfcanl and esfc jobs.

1. Merges in UFS_UTILS hash with updates to reduce file-copying and speed up the sfcanl jobs.
   This update didn't have much impact on the timing, but it reduces the number of files, and removes copious (10,000s lines) of comments from global_cycle log files - worth keeping for these changes.

2. Updates to scripting with associated file name changes for the above hash update, and removal of a redundant option for global_cycle.

3. Split the sfcanl job into two separate jobs:
   sfcanl_gcycle - to do the call to global_cycle (dependent on *_anal)
   sfcanl_regrid - to regrid the soil increments from the gsi_enkf (depdendent on the gdas_enkf)

Splitting these also allows dependencies for the two jobs to be separated out. This is important as the gcycle part of the job takes much longer, but can be launched earlier as the analysis usually finishes before the enkf. From experiments on WCOSS2, we estimate that the forecast job can launch 3-4 minutes earlier with this change.

4. Corrected a dependency in the gdas_eupd job. This was dependent on the analcalc job, however it only needs the anal to have finished - I'm guessing that this wasn't changed when we moved the Gaussian surface analysis into the post-processing jobs.

Changing this dependency means that the esfc job can start about 20 minutes earlier, removing it from the critical path. It has been taking 15-20 minutes. If I've traced the dependencies properly (big if, I didn't follow the marine analysis through), this will mean that the ensemble forecast can launch about 10 minutes earlier (once the ensemble re-centering is done).

Resolves issues 4655 and [4533] and (#4533)

The changes do not result in any changes to the output.

Type of change

• [ x] Code improvement

Change characteristics

• Is this change expected to change outputs (e.g. value changes to existing outputs, new files stored in COM, files removed from COM, filename changes, additions/subtractions to archives)? NO

• Is this a breaking change (a change in existing functionality)? NO

• Does this change require a documentation update? NO

• Does this change require an update to any of the following submodules? YES

  • UFS-utils PR

Above needs to be merged first.

How has this been tested?

Testing underway

Checklist

• Any dependent changes have been merged and published
• [ x] My code follows the style guidelines of this project
• [ x] I have performed a self-review of my own code
• [ x] I have commented my code, particularly in hard-to-understand areas
• [ x] I have documented my code, including function, input, and output descriptions
• [x ] My changes generate no new warnings
• New and existing tests pass with my changes
• [ x] This change is covered by an existing CI test or a new one has been added
• [ x] Any new scripts have been added to the .github/CODEOWNERS file with owners
• [ x] I have made corresponding changes to the system documentation if necessary

[ClaraDraper-NOAA]

Draft, while I do testing and we get the UFS_UTILS branch merged.

@tsga Here is the branch to use to test the speed-up for the deterministic analysis on WCOSS.

[tsga]

this should be feature/speedup_sfcanl?

[ClaraDraper-NOAA]

Yes! Thanks. I'll push an update.

[ClaraDraper-NOAA]

This PR doesn't improve on the timing much. I'm going to leave it in draft, until we can supplement it with additional changes.

[github-actions]

[shfmt] _{reported by reviewdog 🐶}

Suggested change

	gcycle_dates=("${PDY}${cyc}") # Always update surface restarts at middle of window
	gcycle_dates=("${PDY}${cyc}") # Always update surface restarts at middle of window

[github-actions]

[shfmt] _{reported by reviewdog 🐶}

Suggested change

	# Collect the dates in the window to update surface restarts
	# Collect the dates in the window to update surface restarts

[ClaraDraper-NOAA]

@DavidHuber-NOAA @CatherineThomas-NOAA

I'd like to split up the esfc job before finalizing this PR (won't save any time, but is more consistent). Then I'd like to run the full suite of tests, in case I missed a specific combination of settings. I've run out a selection of tests for 24 hours, and the output is zero diff.

Would you mind having a look through it to see if there are any deal-breakers in there for merging into v17?