Skip to content

Update snocvr_snomad IODA files in the ensemble for v17#4597

Merged
DavidHuber-NOAA merged 2 commits intoNOAA-EMC:dev/gfs.v17from
CatherineThomas-NOAA:v17_snomad
Mar 3, 2026
Merged

Update snocvr_snomad IODA files in the ensemble for v17#4597
DavidHuber-NOAA merged 2 commits intoNOAA-EMC:dev/gfs.v17from
CatherineThomas-NOAA:v17_snomad

Conversation

@CatherineThomas-NOAA
Copy link
Contributor

@CatherineThomas-NOAA CatherineThomas-NOAA commented Feb 26, 2026 •
edited
Loading

Description

The snocvr_snomad IODA files are not generated correctly during the snow ensemble analysis. The fix has already been included in the g-w develop branch as part of #4401. Only a small part of that PR addresses this particular bug. The deterministic snow DA is correct.

Resolves #4566

Type of change

  • Bug fix (fixes something broken)

Change characteristics

  • Is this change expected to change outputs (e.g. value changes to existing outputs, new files stored in COM, files removed from COM, filename changes, additions/subtractions to archives)? YES
    • GFS
    • GEFS
    • SFS
    • GCAFS
  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? NO

How has this been tested?

  • C384/C192 S2S + snow DA for several cycles on Cactus (verified by @jiaruidong2017)
  • g-w CI tests (./generate_workflow.sh -G) run on Cactus and Gaea with commit 37168f3. All tests passed except 3DVar extended due to disk quota issues.
  • Tests were rerun with latest dev/gfs.v17 merges. All -G tests passed.

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • Any new scripts have been added to the .github/CODEOWNERS file with owners
  • I have made corresponding changes to the system documentation if necessary

@CatherineThomas-NOAA
Copy link
Contributor Author

CatherineThomas-NOAA commented Feb 27, 2026

Gaea C6 CI Tests

/gpfs/f6/gfs-cpu/world-shared/Catherine.Thomas/tmp/RUNTESTS_snomad2/EXPDIR/C48_ATM_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Feb 27 2026 01:55:08    Feb 27 2026 02:45:13
 
/gpfs/f6/gfs-cpu/world-shared/Catherine.Thomas/tmp/RUNTESTS_snomad2/EXPDIR/C48mx500_3DVarAOWCDA_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241800        Done    Feb 27 2026 01:55:08    Feb 27 2026 02:10:13
202103250000        Done    Feb 27 2026 01:55:08    Feb 27 2026 03:25:11
 
/gpfs/f6/gfs-cpu/world-shared/Catherine.Thomas/tmp/RUNTESTS_snomad2/EXPDIR/C48mx500_hybAOWCDA_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241800        Done    Feb 27 2026 01:55:08    Feb 27 2026 02:10:13
202103250000        Done    Feb 27 2026 01:55:08    Feb 27 2026 03:05:08
 
/gpfs/f6/gfs-cpu/world-shared/Catherine.Thomas/tmp/RUNTESTS_snomad2/EXPDIR/C48_S2SW_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Feb 27 2026 01:55:08    Feb 27 2026 02:55:09
 
/gpfs/f6/gfs-cpu/world-shared/Catherine.Thomas/tmp/RUNTESTS_snomad2/EXPDIR/C96_atm3DVar_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Feb 27 2026 01:55:09    Feb 27 2026 02:10:13
202112210000        Done    Feb 27 2026 01:55:09    Feb 27 2026 03:45:12
202112210600        Done    Feb 27 2026 01:55:09    Feb 27 2026 03:45:12
 
/gpfs/f6/gfs-cpu/world-shared/Catherine.Thomas/tmp/RUNTESTS_snomad2/EXPDIR/C96C48_hybatmDA_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Feb 27 2026 01:55:09    Feb 27 2026 02:10:13
202112210000        Done    Feb 27 2026 01:55:09    Feb 27 2026 03:40:07
202112210600        Done    Feb 27 2026 01:55:09    Feb 27 2026 04:00:08
 
/gpfs/f6/gfs-cpu/world-shared/Catherine.Thomas/tmp/RUNTESTS_snomad2/EXPDIR/C96C48_hybatmsnowDA_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201200        Done    Feb 27 2026 01:55:08    Feb 27 2026 02:10:12
202112201800        Done    Feb 27 2026 01:55:08    Feb 27 2026 03:40:07
202112210000        Done    Feb 27 2026 01:55:08    Feb 27 2026 03:45:12
 
/gpfs/f6/gfs-cpu/world-shared/Catherine.Thomas/tmp/RUNTESTS_snomad2/EXPDIR/C96C48_hybatmsoilDA_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202205150600        Done    Feb 27 2026 01:55:08    Feb 27 2026 02:15:09
202205151200        Done    Feb 27 2026 01:55:08    Feb 27 2026 03:45:12
202205151800        Done    Feb 27 2026 01:55:08    Feb 27 2026 04:05:08
 
/gpfs/f6/gfs-cpu/world-shared/Catherine.Thomas/tmp/RUNTESTS_snomad2/EXPDIR/C96C48mx500_S2SW_cyc_gfs_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201200        Done    Feb 27 2026 01:55:08    Feb 27 2026 02:10:12
202112201800        Done    Feb 27 2026 01:55:08    Feb 27 2026 04:20:10
202112210000        Done    Feb 27 2026 01:55:08    Feb 27 2026 04:45:12

@CatherineThomas-NOAA
Copy link
Contributor Author

CatherineThomas-NOAA commented Feb 27, 2026

WCOSS2 CI Tests

/lfs/h2/emc/ptmp/catherine.thomas/RUNTESTS_snomad2/EXPDIR/C48_ATM_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Feb 27 2026 01:55:21    Feb 27 2026 02:55:27
 
/lfs/h2/emc/ptmp/catherine.thomas/RUNTESTS_snomad2/EXPDIR/C48mx500_3DVarAOWCDA_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241800        Done    Feb 27 2026 01:55:41    Feb 27 2026 02:15:38
202103250000        Done    Feb 27 2026 01:55:41    Feb 27 2026 03:40:29
 
/lfs/h2/emc/ptmp/catherine.thomas/RUNTESTS_snomad2/EXPDIR/C48mx500_hybAOWCDA_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103241800        Done    Feb 27 2026 01:55:12    Feb 27 2026 02:15:25
202103250000        Done    Feb 27 2026 01:55:12    Feb 27 2026 03:15:26
 
/lfs/h2/emc/ptmp/catherine.thomas/RUNTESTS_snomad2/EXPDIR/C48_S2SW_extended_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202103231200        Done    Feb 27 2026 01:55:23    Feb 27 2026 04:20:21
202103231800        Done    Feb 27 2026 01:55:23    Feb 27 2026 04:10:17
  
/lfs/h2/emc/ptmp/catherine.thomas/RUNTESTS_snomad2/EXPDIR/C96_atm3DVar_extended_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Feb 27 2026 01:55:47    Feb 27 2026 02:15:25
202112210000        Done    Feb 27 2026 01:55:47    Feb 27 2026 06:40:27
202112210600        Done    Feb 27 2026 01:55:47    Feb 27 2026 07:30:22
202112211200        Done    Feb 27 2026 02:20:40    Feb 27 2026 08:55:25
202112211800        Done    Feb 27 2026 06:45:19    Feb 27 2026 11:20:28
  
/lfs/h2/emc/ptmp/catherine.thomas/RUNTESTS_snomad2/EXPDIR/C96C48_hybatmDA_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201800        Done    Feb 27 2026 01:55:18    Feb 27 2026 02:15:25
202112210000        Done    Feb 27 2026 01:55:18    Feb 27 2026 04:10:18
202112210600        Done    Feb 27 2026 01:55:18    Feb 27 2026 04:00:12
 
/lfs/h2/emc/ptmp/catherine.thomas/RUNTESTS_snomad2/EXPDIR/C96C48_hybatmsnowDA_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201200        Done    Feb 27 2026 01:55:15    Feb 27 2026 02:15:37
202112201800        Done    Feb 27 2026 01:55:15    Feb 27 2026 04:05:09
202112210000        Done    Feb 27 2026 01:55:15    Feb 27 2026 04:00:32
 
/lfs/h2/emc/ptmp/catherine.thomas/RUNTESTS_snomad2/EXPDIR/C96C48_hybatmsoilDA_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202205150600        Done    Feb 27 2026 01:55:36    Feb 27 2026 02:15:38
202205151200        Done    Feb 27 2026 01:55:36    Feb 27 2026 04:10:15
202205151800        Done    Feb 27 2026 01:55:36    Feb 27 2026 04:10:15
 
/lfs/h2/emc/ptmp/catherine.thomas/RUNTESTS_snomad2/EXPDIR/C96C48mx500_S2SW_cyc_gfs_snomad2
   CYCLE         STATE           ACTIVATED              DEACTIVATED     
202112201200        Done    Feb 27 2026 01:55:46    Feb 27 2026 02:15:12
202112201800        Done    Feb 27 2026 01:55:46    Feb 27 2026 04:45:37
202112210000        Done    Feb 27 2026 01:55:46    Feb 27 2026 05:10:28
202112211800        Done    Feb 27 2026 02:20:21    Feb 27 2026 05:20:16

@CatherineThomas-NOAA CatherineThomas-NOAA marked this pull request as ready for review February 27, 2026 14:03
@CatherineThomas-NOAA
Copy link
Contributor Author

CatherineThomas-NOAA commented Feb 27, 2026

All -G tests passed on Gaea and WCOSS2. Ready for review.

DavidNew-NOAA
DavidNew-NOAA approved these changes Feb 27, 2026
View reviewed changes
Copy link
Contributor

@DavidNew-NOAA DavidNew-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good for now, though snow_prepobs_path is defined twice, once here and once in the jcb-base.yaml.j2 for snow in GDASApp here https://github.com/NOAA-EMC/GDASApp/blob/6b356ac669c66697dc847b4e8580b33b2f9934fd/parm/snow/jcb-base.yaml.j2#L87 We'll need to make this consistent eventually.

@CatherineThomas-NOAA
Copy link
Contributor Author

CatherineThomas-NOAA commented Feb 27, 2026

@DavidNew-NOAA - This makes me a little confused then. While the link you provided isn't the same hash as we're using in v17 (we have a production branch for GDASApp), I see the same line in the equivalent file in that branch (here). My understanding is that the esnowanl job wasn't getting this setting, so there must be something I'm missing. Can you or @jiaruidong2017 @CoryMartin-NOAA clarify?

@jiaruidong2017
Copy link
Contributor

jiaruidong2017 commented Feb 27, 2026

The snow_prepobs_path directory is required in the snow_anl.initialize() step to temporarily store the preprocessed GHCN and snocvr_snomad observations.

The snow_prepobs_path defined in jcb-base.yaml.j2 cannot be directly accessed within snow_analysis.py, because jcb-base.yaml.j2 is a Jinja2 template that is rendered into a YAML file at runtime with workflow variables substituted. As a result, its values are not inherently available inside the Python script unless they are explicitly passed through the task configuration or environment.

Therefore, the path needs to be explicitly defined or passed to the initialize() step in the Python script to ensure it is available at runtime. @CoryMartin-NOAA @DavidNew-NOAA Please let me know if my understanding is incorrect.

@DavidNew-NOAA
Copy link
Contributor

DavidNew-NOAA commented Feb 27, 2026 •
edited
Loading

@CatherineThomas-NOAA So the simplest fix is to set:

snow_prepobs_path: "{{ snow_prepobs_path }}"

in jcb-base.yaml.j2 since we're already adding snow_prepobs_path to task_config which is used to render jcb-base.yaml.j2.

Now, snow_prepobs_path is indeed accessible from jcb-base.yaml.j2 via the Jedi class. You would invoke it in this Python script as:

self.jedi_dict['snowanlvar'].jcb_config.snow_prepobs_path

That's probably overkill though. I had assumed that yaml.j2 files like https://github.com/NOAA-EMC/GDASApp/blob/6b356ac669c66697dc847b4e8580b33b2f9934fd/parm/snow/prep/prep_ghcn.yaml.j2 which need snow_prepobs_path will be retired eventually as our obs infrastructure matures? Am I wrong about his? @jiaruidong2017 @CoryMartin-NOAA

@jiaruidong2017
Copy link
Contributor

jiaruidong2017 commented Mar 2, 2026

Thanks @DavidNew-NOAA. Both proposed methods sound good to me. I prefer the second one, which keeps the definition of snow_prepobs_path in jcb-base.yaml.j2 and invokes it in the Python script as:

self.jedi_dict['snowanlvar'].jcb_config.snow_prepobs_path

This approach avoids redefining snow_prepobs_path and keeps its configuration centralized in jcb-base.yaml.j2.

@CatherineThomas-NOAA If you’d like to use this approach, I can send you an example code that you can use in gfs.v17.

@CatherineThomas-NOAA
Copy link
Contributor Author

CatherineThomas-NOAA commented Mar 2, 2026 •
edited
Loading

@jiaruidong2017 @DavidNew-NOAA - Thanks for both of your explanations. It sounds like we will want the suggested change to the python script in the previous comment, but since this PR's current change has been tested and confirmed, I think we should move ahead with this PR as is, test the other one in parallel, and open a separate PR when ready. Since it should be non-answer changing and sounds like its only in the script (no recompile or submodule updates), it can be included fairly easily at a later time. Does that sound reasonable to everyone?

@DavidNew-NOAA
Copy link
Contributor

DavidNew-NOAA commented Mar 2, 2026

@CatherineThomas-NOAA Good plan. This isn't a urgent priority

@jiaruidong2017
Copy link
Contributor

jiaruidong2017 commented Mar 2, 2026

@CatherineThomas-NOAA @DavidNew-NOAA Sounds good. I will first add the second method to my OCNRES PR on the global-workflow develop branch.

@CatherineThomas-NOAA
Copy link
Contributor Author

CatherineThomas-NOAA commented Mar 2, 2026

Thanks @DavidNew-NOAA and @jiaruidong2017

@DavidHuber-NOAA @JessicaMeixner-NOAA @RuiyuSun - This PR is fine to move forward as is.

@DavidHuber-NOAA DavidHuber-NOAA merged commit d0e6646 into NOAA-EMC:dev/gfs.v17 Mar 3, 2026
7 checks passed
@CatherineThomas-NOAA CatherineThomas-NOAA deleted the v17_snomad branch March 3, 2026 13:35
@jiaruidong2017 jiaruidong2017 mentioned this pull request Mar 3, 2026
Closed
6 tasks
@JessicaMeixner-NOAA JessicaMeixner-NOAA mentioned this pull request Mar 3, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JessicaMeixner-NOAA JessicaMeixner-NOAA JessicaMeixner-NOAA approved these changes

@jiaruidong2017 jiaruidong2017 jiaruidong2017 approved these changes

@DavidHuber-NOAA DavidHuber-NOAA DavidHuber-NOAA approved these changes

@DavidNew-NOAA DavidNew-NOAA DavidNew-NOAA approved these changes

@RuiyuSun RuiyuSun Awaiting requested review from RuiyuSun

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@CatherineThomas-NOAA @jiaruidong2017 @DavidNew-NOAA @JessicaMeixner-NOAA @DavidHuber-NOAA