Skip to content

Turn off anlstat job for GSI based experiments by default#4710

Open
Copilot wants to merge 4 commits intodevelopfrom
copilot/turn-off-anlstat-job
Open

Turn off anlstat job for GSI based experiments by default#4710
Copilot wants to merge 4 commits intodevelopfrom
copilot/turn-off-anlstat-job

Conversation

Copy link
Contributor

Copilot AI commented Mar 26, 2026

Description

The anlstat job was unconditionally running for GSI-based experiments, causing issues on some platforms. This PR turns it off by default for GSI experiments and adds an explicit opt-in flag, while preserving the existing behavior that enables DO_ANLSTAT for JEDI-based experiments.

Changes:

  • dev/parm/config/gfs/config.base.j2: Reverts DO_ANLSTAT default back to "NO" (as it was in e8d51af); adds new DO_GSI_ANLSTAT="NO" flag; restores the conditional logic from e8d51af that turns DO_ANLSTAT back to "YES" for JEDI-based experiments:
    • When DO_JEDIATMVAR="YES" (JEDI atmospheric variational DA)
    • When any of DO_AERO="YES", DO_JEDIOCNVAR="YES", or DO_JEDISNOWDA="YES"
  • dev/scripts/exglobal_analysis_stats.py: Replaces the unconditional else: append('atmos_gsi') with elif config.DO_GSI_ANLSTAT: — GSI analysis stats now only run when explicitly opted in
  • docs/source/configure.rst: Added documentation for DO_ANLSTAT and DO_GSI_ANLSTAT options
# Before: atmos_gsi always added when JEDI not active
if config.DO_JEDIATMVAR:
    config.STAT_ANALYSES.append('atmos')
else:
    config.STAT_ANALYSES.append('atmos_gsi')

# After: atmos_gsi only added when DO_GSI_ANLSTAT=YES
if config.DO_JEDIATMVAR:
    config.STAT_ANALYSES.append('atmos')
elif config.DO_GSI_ANLSTAT:
    config.STAT_ANALYSES.append('atmos_gsi')

To re-enable GSI analysis statistics, set DO_GSI_ANLSTAT="YES" in the experiment configuration. JEDI-based experiments are unaffected — DO_ANLSTAT is still automatically enabled for them via the conditional logic in config.base.j2.

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this change expected to change outputs (e.g. value changes to existing outputs, new files stored in COM, files removed from COM, filename changes, additions/subtractions to archives)? NO
    • GFS
    • GEFS
    • SFS
    • GCAFS
  • Is this a breaking change (a change in existing functionality)? YES — GSI experiments that relied on anlstat running by default will no longer run it unless DO_GSI_ANLSTAT="YES" is set
  • Does this change require a documentation update? YES — documentation for DO_ANLSTAT and DO_GSI_ANLSTAT has been added to docs/source/configure.rst
  • Does this change require an update to any of the following submodules? NO
    • EMC verif-global
    • GDAS
    • GFS-utils
    • GSI
    • GSI-monitor
    • GSI-utils
    • UFS-utils
    • UFS-weather-model
    • wxflow

How has this been tested?

Code review only; no runtime tests conducted.

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • Any new scripts have been added to the .github/CODEOWNERS file with owners
  • I have made corresponding changes to the system documentation if necessary

⌨️ Start Copilot coding agent tasks without leaving your editor — available in VS Code, Visual Studio, JetBrains IDEs and Eclipse.

Co-authored-by: CoryMartin-NOAA <6354668+CoryMartin-NOAA@users.noreply.github.com>
Agent-Logs-Url: https://github.com/NOAA-EMC/global-workflow/sessions/e807e5c3-fd50-4155-8fb7-66f5c153d3f9
Copilot AI changed the title [WIP] Turn off anlstat job for GSI based experiments Turn off anlstat job for GSI based experiments by default Mar 26, 2026
Copilot AI requested a review from CoryMartin-NOAA March 26, 2026 12:34
@CoryMartin-NOAA
Copy link
Contributor

@copilot this isn't quite right, please look at the entire config.base file in that hash, there was logic towards the end that turned it back to YES depending on other settings

…riments

Co-authored-by: CoryMartin-NOAA <6354668+CoryMartin-NOAA@users.noreply.github.com>
Agent-Logs-Url: https://github.com/NOAA-EMC/global-workflow/sessions/cefb67f9-77b4-4927-9b2b-8729585fd41b
Copy link
Contributor Author

Copilot AI commented Mar 26, 2026

@copilot this isn't quite right, please look at the entire config.base file in that hash, there was logic towards the end that turned it back to YES depending on other settings

Fixed in aa11aae. Restored the conditional block from e8d51af that sets DO_ANLSTAT="YES" for JEDI-based experiments — when DO_JEDIATMVAR="YES", or when any of DO_AERO, DO_JEDIOCNVAR, or DO_JEDISNOWDA are YES. GSI-only experiments remain off by default (controllable via DO_GSI_ANLSTAT).

@DavidHuber-NOAA
Copy link
Contributor

@CoryMartin-NOAA @DavidNew-NOAA I think this would also resolve #4450 and could supersede #4465. Could we close #4465 in favor of this approach?

@CoryMartin-NOAA
Copy link
Contributor

Probably. @DavidNew-NOAA thoughts? I somehow missed that this was open

@DavidNew-NOAA
Copy link
Contributor

@CoryMartin-NOAA @DavidHuber-NOAA Yes, agree

DavidNew-NOAA
DavidNew-NOAA previously approved these changes Mar 26, 2026
@DavidHuber-NOAA
Copy link
Contributor

DavidHuber-NOAA commented Mar 26, 2026

Launching CI on Hercules.

@DavidHuber-NOAA The build on Hercules Slurm timed out building gdas ~T.McG

| gfs_ww3prepost     | SUCCEEDED  |
| gdas               | DEAD       |
| gsi_enkf           | SUCCEEDED  |
[2026-03-26T12:35:19.015] error: *** JOB 8391847 ON hercules-02-03 CANCELLED AT 2026-03-26T12:35:19 DUE TO TIME LIMIT **

@DavidHuber-NOAA
Copy link
Contributor

@copilot please add documentation to docs/source/configure.rst to illustrate what the DO_ANLSTAT and DO_GSI_ANLSTAT options do.

@DavidHuber-NOAA
Copy link
Contributor

I didn't know I could hijack someone else's copilot session 😮

Copilot AI dismissed stale reviews from DavidHuber-NOAA and DavidNew-NOAA via 12a8990 March 26, 2026 13:39
Copy link
Contributor Author

Copilot AI commented Mar 26, 2026

@copilot please add documentation to docs/source/configure.rst to illustrate what the DO_ANLSTAT and DO_GSI_ANLSTAT options do.

Added in 12a8990. The two new entries appear after DO_VMINMON in the monitoring section of the table:

  • DO_ANLSTAT — defaults NO, adjusts XML; notes it's automatically set to YES for JEDI-based experiments
  • DO_GSI_ANLSTAT — defaults NO, does not adjust XML; explains when it's relevant (GSI + DO_ANLSTAT=YES)

Copilot AI requested a review from DavidHuber-NOAA March 26, 2026 13:41
@emcbot emcbot added CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed and removed CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules labels Mar 26, 2026
@TravisElless-NOAA
Copy link
Contributor

Hercules CI failure here appears to be due to the build job timing out. I'll rekick it and see if it gets going again

@emcbot emcbot added the CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules label Mar 26, 2026
@TravisElless-NOAA TravisElless-NOAA removed the CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed label Mar 26, 2026
@emcbot emcbot added CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed and removed CI-Hercules-Building **Bot use only** CI testing is cloning/building on Hercules labels Mar 26, 2026
@TerrenceMcGuinness-NOAA
Copy link
Collaborator

@TravisElless-NOAA Yes gdas timed out its build on Hercules. BTW when restarting a build job in GitLab it will start over again because it has to re-clone unlike all the other jobs that pick up where they left off.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI-Hercules-Failed **Bot use only** CI testing on Hercules for this PR has failed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants