Linkbcs Standalone Script by sshakoor1 · Pull Request #820 · GEOS-ESM/GEOSgcm_App

sshakoor1 · 2026-03-06T18:36:43Z

The purpose of this PR is create a new linkbcs that is no longer a template file detemplated by gcm_setup, but rather a standalone python script ran via a config yaml. This new script should essentially be zero-diff from the old one.

To run, use $INSTALL_DIR/bin/linkbcs.py --config <USER_CONFIG>.yaml --timestamp <YYYY>-<MM>-<DD>T<HH>:<MM>:<SS> from inside the directory you want your symlinks to be created.

Here is a skeleton of what the config yaml should look like, along with valid options for each key:

platform: #nccs or nas
experiment_type: #GCM or ADAS
agcm_grid: #c12, c24, c48, c90, etc.
land_version: #icarus, icarus-NLv3, v12, v14
ocean_model: #data, mom6, MIT
ogcm_grid: #reynolds, merra-2, ostia, cubed_sphere_ostia, cubed_sphere_ostia_r21c
seaice_model: #cice4 or cice6 (this entry is only required for coupled ocean)
precip_correction: #True or False 
catchcn: #True or False
pchem_species: #ops, cmip, s2s, merra-2
install_dir: /path/to/your/install/dir

SOME NOTES:

these entries are case sensitive at the moment.
The install_dir should point to the directory your build of GEOS is installed in. For most people this will simply be .../<user_dir>/GEOSgcm/install/
Right now only the year matters in the timestamp, but you still need to put entries for the time/month/day (they could all be zero).

catalog.yaml

mathomp4 · 2026-03-16T17:14:28Z

linkbcs.py

+    except ValueError:
+        raise argparse.ArgumentTypeError(
+            f"Invalid datetime format: '{datetime_string}'. "
+            f"Expected ISO format (YYYY-MM-DDTHH:MM:SS), e.g., '2025-04-04T00:00:00'"


If you can, can you make a small python script that would convert our "pack time":

YYYYMMDD hhmmss

that we have in cap_restart (a la 20000414 210000) and would emit:

YYYY-MM-DDThh:mm:ss

so that could then be passed to linkbcs.py. So something like:

set ISO_TIME = `convert_time_string.py cap_restart` linkbcs.py config.yaml $ISO_TIME

rtodling

Just a general comment: this branching naming seems very unconventional. I'd expect something more like feature/linkbcs_... or feature/username/linkbcs...

rtodling

another general comment:

I would prefer to see the file catalog.yaml renamed bcs_catalog.yaml (or something like it) - I name that associates the catalog to its function.

sshakoor1 · 2026-03-17T19:47:18Z

Just a general comment: this branching naming seems very unconventional. I'd expect something more like feature/linkbcs_... or feature/username/linkbcs...

Unfortunately I cannot rename this branch without deleting it and making a new one. Please refer to #823 for the renamed branch.

rtodling · 2026-03-17T21:00:51Z

I am getting an error when tring this:

linkbcs.py --config linkbcs.yaml --timestamp 2025-11-28T21:00:00
Traceback (most recent call last):
File "/gpfsm/dnb10/projects/p61/rtodling/4OPS/g540x/g542x/1173_GEOSadas/install-SLES15/bin/linkbcs.py", line 408, in
main()
File "/gpfsm/dnb10/projects/p61/rtodling/4OPS/g540x/g542x/1173_GEOSadas/install-SLES15/bin/linkbcs.py", line 403, in main
symlink_creator = SymlinkCreator(catalog_manager, args.timestamp.year)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/gpfsm/dnb10/projects/p61/rtodling/4OPS/g540x/g542x/1173_GEOSadas/install-SLES15/bin/linkbcs.py", line 110, in init
self.bcs_dir = self.boundary_dir / "bcs_shared/fvInput/ExtData/esm/tiles" / catalog.get_value('lsm_bcs', 'land_version')
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TypeError: unsupported operand type(s) for /: 'PosixPath' and 'NoneType'
else

sshakoor1 · 2026-03-17T21:04:57Z

I am getting an error when tring this:

linkbcs.py --config linkbcs.yaml --timestamp 2025-11-28T21:00:00 Traceback (most recent call last): File "/gpfsm/dnb10/projects/p61/rtodling/4OPS/g540x/g542x/1173_GEOSadas/install-SLES15/bin/linkbcs.py", line 408, in main() File "/gpfsm/dnb10/projects/p61/rtodling/4OPS/g540x/g542x/1173_GEOSadas/install-SLES15/bin/linkbcs.py", line 403, in main symlink_creator = SymlinkCreator(catalog_manager, args.timestamp.year) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/gpfsm/dnb10/projects/p61/rtodling/4OPS/g540x/g542x/1173_GEOSadas/install-SLES15/bin/linkbcs.py", line 110, in init self.bcs_dir = self.boundary_dir / "bcs_shared/fvInput/ExtData/esm/tiles" / catalog.get_value('lsm_bcs', 'land_version') ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ TypeError: unsupported operand type(s) for /: 'PosixPath' and 'NoneType' else

@rtodling Can I see inside your linkbcs.yaml file?

rtodling · 2026-03-17T21:12:56Z

Just for the record, since this change needs to also get (not only in develop) but in GEOS-FP-5_44 ... I copied the code to:

feature/linkbcs_py_5_44

you don't have to worry about maintaining both the code in both locations. I can bring your changes into the branch above.

rtodling · 2026-03-17T21:29:31Z

I am getting an error when tring this:
linkbcs.py --config linkbcs.yaml --timestamp 2025-11-28T21:00:00 Traceback (most recent call last): File "/gpfsm/dnb10/projects/p61/rtodling/4OPS/g540x/g542x/1173_GEOSadas/install-SLES15/bin/linkbcs.py", line 408, in main() File "/gpfsm/dnb10/projects/p61/rtodling/4OPS/g540x/g542x/1173_GEOSadas/install-SLES15/bin/linkbcs.py", line 403, in main symlink_creator = SymlinkCreator(catalog_manager, args.timestamp.year) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/gpfsm/dnb10/projects/p61/rtodling/4OPS/g540x/g542x/1173_GEOSadas/install-SLES15/bin/linkbcs.py", line 110, in init self.bcs_dir = self.boundary_dir / "bcs_shared/fvInput/ExtData/esm/tiles" / catalog.get_value('lsm_bcs', 'land_version') ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ TypeError: unsupported operand type(s) for /: 'PosixPath' and 'NoneType' else

@rtodling Can I see inside your linkbcs.yaml file?

Sure:

platform: nccs
experiment_type: ADAS
agcm_grid: c360
land_version: Icarus-NLv3
ocean_model: data
ogcm_grid: cubed_sphere_ostia
seaice_model: null
precip_correction: False
catchcn: False
pchem_species: ops # or cmip, s2s, merra-2
install_dir: /gpfsm/dnb10/projects/p61/rtodling/4OPS/g540x/g542x/1173_GEOSadas/install-SLES15

rtodling · 2026-03-17T21:31:42Z

I am thinking that maybe python / the yaml-interpreter doesn't like the comment after the line for pchem ... I removed that and am trying again.

sshakoor1 · 2026-03-17T21:35:41Z

just eye-balling your config I can see on the line land_version: Icarus-NLv3, the I in icarus needs to be lower-case. Unfortunately the script is case sensitive at the moment.

rtodling · 2026-03-18T00:44:39Z

Btw: this doesn't look right to me:

bcs_shared/fvInput/ExtData/

in the error above.

In the ADAS, fvInput is the highest directory level for bcs ... there is no such things as bcs_shared that I know of.

rtodling · 2026-03-18T17:08:05Z

just eye-balling your config I can see on the line land_version: Icarus-NLv3, the I in icarus needs to be lower-case. Unfortunately the script is case sensitive at the moment.

That's inconsistent w/ how we've been referring to the land type for a really long time!

mathomp4 · 2026-03-18T18:40:11Z

Btw: this doesn't look right to me:

bcs_shared/fvInput/ExtData/

in the error above.

In the ADAS, fvInput is the highest directory level for bcs ... there is no such things as bcs_shared that I know of.

@rtodling

That bcs_shared location is where @biljanaorescanin has been creating the new boundary conditions we use now in v11 and v12. The full path is (in current linkbcs):

/discover/nobackup/projects/gmao/bcs_shared/fvInput/ExtData/esm/tiles

to make things simpler for this work, @mssolomo created a symlink so that:

/discover/nobackup/projects/gmao/gmao_ops/bcs_shared/fvInput/ExtData/esm/tiles

exists.

But, yes, there is an fvInput in there.

rtodling · 2026-03-18T18:49:12Z

Getting better ... indeed it seem the case of the land file might have been the issue associated w/ the crash of the program.

So, the test case is running but I can see the following files:

/bin/ln -sf /discover/nobackup/ltakacs/bcs/Icarus-NLv3/Icarus-NLv3_Reynolds/Shared/*bin .
 /bin/ln -sf /discover/nobackup/ltakacs/bcs/Icarus-NLv3/Icarus-NLv3_Reynolds/Shared/*c2l*.nc4 .

are not handed by your program - the files above correspond to the files needed by history when doing interpolation from the native grid to the grids in history. Without these being linked in the work dir, the model will run considerably slower ....

mathomp4 · 2026-03-18T19:10:14Z

Getting better ... indeed it seem the case of the land file might have been the issue associated w/ the crash of the program.

So, the test case is running but I can see the following files:
/bin/ln -sf /discover/nobackup/ltakacs/bcs/Icarus-NLv3/Icarus-NLv3_Reynolds/Shared/*bin .
 /bin/ln -sf /discover/nobackup/ltakacs/bcs/Icarus-NLv3/Icarus-NLv3_Reynolds/Shared/*c2l*.nc4 .
are not handed by your program - the files above correspond to the files needed by history when doing interpolation from the native grid to the grids in history. Without these being linked in the work dir, the model will run considerably slower ....

@rtodling These are no longer needed in GEOSgcm v11 (maybe even v10). When we moved to ESMF regridding, these files became redundant.

Indeed in a current v11 experiment, the scratch directory has only one .bin file now:

> ls /discover/nobackup/mathomp4/Experiments/stock-v11-2026Mar03-1day-c24-L72/scratch/*c2l*
"/discover/nobackup/mathomp4/Experiments/stock-v11-2026Mar03-1day-c24-L72/scratch/*c2l*": No such file or directory (os error 2)

> ls /discover/nobackup/mathomp4/Experiments/stock-v11-2026Mar03-1day-c24-L72/scratch/*.bin
 /discover/nobackup/mathomp4/Experiments/stock-v11-2026Mar03-1day-c24-L72/scratch/tile.bin

I'll ping @bena-nasa and @atrayano to confirm.

bena-nasa · 2026-03-18T19:20:50Z

Yes the c2l files are only needed ESMF regridding days as Matt said, these should be needed in any model version in the last several years at least.

biljanaorescanin · 2026-03-18T23:55:39Z

@mathomp4 I just want to chime and offer additional simpler option to simplify code by avoiding user owned directory Ricardo uses now. Or just additional information.

Path that Ricardo says he needs he doesn't have to point to this one:
/bin/ln -sf /discover/nobackup/ltakacs/bcs/Icarus-NLv3/Icarus-NLv3_Reynolds/Shared/*bin .
/bin/ln -sf /discover/nobackup/ltakacs/bcs/Icarus-NLv3/Icarus-NLv3_Reynolds/Shared/*c2l*.nc4 .

All of those files zero-diff are split to two locations in bcs_shared :
Here:
/discover/nobackup/projects/gmao/bcs_shared/fvInput/ExtData/esm/tiles/NL3/Shared
and PCHEM here:
/discover/nobackup/projects/gmao/bcs_shared/fvInput/ExtData/esm/tiles/NL3/PCHEM

Someone would have to compare directories to confirm again noting was added but at the time that was synced to Larry's area. But if that is not used anymore it doesn't matter then ignore my comment.

rtodling · 2026-03-19T00:07:15Z

@mathomp4 I just want to chime and offer additional simpler option to simplify code by avoiding user owned directory Ricardo uses now. Or just additional information.

Path that Ricardo says he needs he doesn't have to point to this one: /bin/ln -sf /discover/nobackup/ltakacs/bcs/Icarus-NLv3/Icarus-NLv3_Reynolds/Shared/*bin . /bin/ln -sf /discover/nobackup/ltakacs/bcs/Icarus-NLv3/Icarus-NLv3_Reynolds/Shared/*c2l*.nc4 .

All of those files zero-diff are split to two locations in bcs_shared : Here: /discover/nobackup/projects/gmao/bcs_shared/fvInput/ExtData/esm/tiles/NL3/Shared and PCHEM here: /discover/nobackup/projects/gmao/bcs_shared/fvInput/ExtData/esm/tiles/NL3/PCHEM

Someone would have to compare directories to confirm again noting was added but at the time that was synced to Larry's area. But if that is not used anymore it doesn't matter then ignore my comment.

I remember after we had our more recent meeting on this on Teams you asked me for a dump fvwork directory so you could compare what I use and what I would be getting if I am to replace the link script - did anyone have a chance to look at that?

What I am getting is very different from what I need - sure enough the model crash ... and here is one example of how it crashes:

Character Resource Parameter: COMPONENT_TO_RECORD:[Empty formatted value], (default value)
Character Resource Parameter: TILING_FILE:tile.bin

The reason being that the tile.bin file is empty!

mathomp4 · 2026-03-19T13:17:32Z

Character Resource Parameter: COMPONENT_TO_RECORD:[Empty formatted value], (default value) Character Resource Parameter: TILING_FILE:tile.bin

The reason being that the tile.bin file is empty!

@rtodling Can you point us to your experiment/fvwork directory? So we can see what sort of symlinks are being made?

rtodling · 2026-03-19T13:39:19Z

Character Resource Parameter: COMPONENT_TO_RECORD:[Empty formatted value], (default value) Character Resource Parameter: TILING_FILE:tile.bin
The reason being that the tile.bin file is empty!

@rtodling Can you point us to your experiment/fvwork directory? So we can see what sort of symlinks are being made?

/gpfsm/dnb10/projects/p139/rtodling/545/fvwork.67332

rtodling · 2026-03-19T13:40:29Z

It's also unclear to me how your programs distinguishes from v11 and v12 bcs? I would expect to see an entry in the input yaml file referring to that, no?

sshakoor1 · 2026-03-19T15:21:15Z

Character Resource Parameter: COMPONENT_TO_RECORD:[Empty formatted value], (default value) Character Resource Parameter: TILING_FILE:tile.bin
The reason being that the tile.bin file is empty!

@rtodling Can you point us to your experiment/fvwork directory? So we can see what sort of symlinks are being made?

/gpfsm/dnb10/projects/p139/rtodling/545/fvwork.67332

@rtodling Can you point us to a directory with symlinks made by this script? All the symlinks here are from your pearl script.

rtodling · 2026-03-19T15:42:27Z

Sorry - i thought I was at another PR and pressed the update branch button - I hope I didn't mess anyone up.

mathomp4 · 2026-03-19T15:49:11Z

Character Resource Parameter: COMPONENT_TO_RECORD:[Empty formatted value], (default value) Character Resource Parameter: TILING_FILE:tile.bin

The reason being that the tile.bin file is empty!

Ahh. Okay. Now I remember. This is a difference between the GCM and the ADAS. In the gcm we do inside gcm_run.j:

setenv YEAR $yearc
./linkbcs

if (! -e tile.bin) then
$GEOSBIN/binarytile.x tile.data tile.bin
endif

So, we run binarytile.x outside the linkbcs process. But in gen_lnbcs.pl that is done in your lnbcs:

  \$FVROOT/bin/binarytile.x tile.data tile.bin

I suppose this is a philosophical question for @tclune: Should our new linkbcs.py call other executables? It's probably not hard for @sshakoor1 to do some sort of subprocess call to binarytile.x. I'm just not sure what the "right" thing is.

mathomp4 · 2026-03-19T15:51:11Z

It's also unclear to me how your programs distinguishes from v11 and v12 bcs? I would expect to see an entry in the input yaml file referring to that, no?

I might need to ping @biljanaorescanin. But I think the main BCs difference between v11 and v12 is the land BCs. That is by default we use NL3 Land in v11 but in v12 we move to v12 Land (and maybe soon v14).

Everything else is sort of just grids and all, but it's the default land that has changed.

biljanaorescanin · 2026-03-19T16:30:31Z

It's also unclear to me how your programs distinguishes from v11 and v12 bcs? I would expect to see an entry in the input yaml file referring to that, no?

I might need to ping @biljanaorescanin. But I think the main BCs difference between v11 and v12 is the land BCs. That is by default we use NL3 Land in v11 but in v12 we move to v12 Land (and maybe soon v14).

Everything else is sort of just grids and all, but it's the default land that has changed.

The goal going forward is to transition to v12 BCs in GCM v12 (and potentially v13/v14 once testing confirms improvements). GCM v11 as Matt said and current FP still use old land from 10+ years ago (NL3 or Icarus-NLv3)

What actually changes as far BCs sets in more details if interested:

NL3 → v11
This is a major upgrade in land representation (literally different land), including:
Updated vegetation canopy height (lidar-based instead of lookup tables)
Improved snow albedo (MODIS-based climatology)
Integration of PEATMAP and more modern land datasets
General modernization of land inputs and parameterizations

v11 → v12
This is a much smaller, targeted update:
Switch from HWSD → HWSD_b (bug-fixed soil dataset)
Includes the Argentina peatland fix (~38S, 60W)
Minor technical updates (e.g., outlet version, tile format, elevation fix)

The idea is to move toward v12 as the new baseline, and then:
Evaluate v13/v14 as they mature.
Adopt improvements (e.g., coupled grid fixes, peat/land refinements)

I can give more details on all that is in q but I think that would be a long list for a message here.

tclune · 2026-03-19T16:47:59Z

I suppose this is a philosophical question for @tclune: Should our new linkbcs.py call other executables? It's probably not hard for @sshakoor1 to do some sort of subprocess call to binarytile.x. I'm just not sure what the "right" thing is.

I'm having trouble getting the full context into my "buffer" at the moment. Will discuss with @mathomp4 tomorrow.

linkbcs.py beta release

6b4c845

sshakoor1 self-assigned this Mar 6, 2026

modded cmakelists and fixed land symlink names

f0daa9d

sshakoor1 added enhancement New feature or request 0 diff trivial The changes in this pull request are trivially zero-diff (documentation, build failure, &c.) labels Mar 6, 2026

sshakoor1 requested review from mathomp4, rtodling and tclune March 6, 2026 18:56

Shayon Shakoorzadeh added 2 commits March 12, 2026 20:05

added ADAS support

e8fc61a

added shebang run option

878a7d1

mathomp4 reviewed Mar 16, 2026

View reviewed changes

catalog.yaml Outdated Show resolved Hide resolved

mathomp4 reviewed Mar 16, 2026

View reviewed changes

catalog.yaml Show resolved Hide resolved

mathomp4 reviewed Mar 16, 2026

View reviewed changes

Shayon Shakoorzadeh added 3 commits March 16, 2026 14:06

created a helper script for packtime to iso format

d93656e

added gmao_desktop to catalog

30c20f0

updated cmakelists.txt

c7c4e12

rtodling reviewed Mar 17, 2026

View reviewed changes

sshakoor1 mentioned this pull request Mar 17, 2026

Linkbcs Standalone Script #823

Draft

Merge branch 'develop' into linkbcs_standalone_script

151f0a8

Conversation

sshakoor1 commented Mar 6, 2026 • edited by tclune Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mathomp4 Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

rtodling left a comment

Choose a reason for hiding this comment

Uh oh!

rtodling left a comment

Choose a reason for hiding this comment

Uh oh!

sshakoor1 commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rtodling commented Mar 17, 2026

Uh oh!

sshakoor1 commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rtodling commented Mar 17, 2026

Uh oh!

rtodling commented Mar 17, 2026

Uh oh!

rtodling commented Mar 17, 2026

Uh oh!

sshakoor1 commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rtodling commented Mar 18, 2026

Uh oh!

rtodling commented Mar 18, 2026

Uh oh!

mathomp4 commented Mar 18, 2026

Uh oh!

rtodling commented Mar 18, 2026

Uh oh!

mathomp4 commented Mar 18, 2026

Uh oh!

bena-nasa commented Mar 18, 2026

Uh oh!

biljanaorescanin commented Mar 18, 2026

Uh oh!

rtodling commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mathomp4 commented Mar 19, 2026

Uh oh!

rtodling commented Mar 19, 2026

Uh oh!

rtodling commented Mar 19, 2026

Uh oh!

sshakoor1 commented Mar 19, 2026

Uh oh!

rtodling commented Mar 19, 2026

Uh oh!

mathomp4 commented Mar 19, 2026

Uh oh!

mathomp4 commented Mar 19, 2026

Uh oh!

biljanaorescanin commented Mar 19, 2026

Uh oh!

tclune commented Mar 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

sshakoor1 commented Mar 6, 2026 •

edited by tclune

Loading

sshakoor1 commented Mar 17, 2026 •

edited

Loading

sshakoor1 commented Mar 17, 2026 •

edited

Loading

sshakoor1 commented Mar 17, 2026 •

edited

Loading

rtodling commented Mar 19, 2026 •

edited

Loading