Pull UKMO components from Github by Whyborn · Pull Request #385 · ACCESS-NRI/access-spack-packages

Whyborn · 2026-02-17T03:35:50Z

With the recent migration of the UKMO model components to Github, we can pull the auxiliary components (GCOM, Casim, Socrates and UKCA) from ACCESS-NRI forks of the codes, instead of the MOSRS mirror.

Tested against the previous release configuration (the one used in the release-n96e configuration) in this PR. Shows bitwise identical results for the first few months in the NetCDF output.

Should we also change the svn location to a git location while we're here?

penguian · 2026-02-17T23:39:56Z

Since we will now have the capability to do this for AM3, what is stopping us for doing it for the UM in general (e.g. rAM3, CM3)? Would the general capability need to be documented in a separate issue?

See also #377 which is still using MOSRS.

penguian · 2026-02-17T23:43:43Z

-        "jules_rev",
-        "shumlib_rev",
-        "socrates_rev")
-


Does removing the "_rev" variants force all builds to use GitHub only? Will MOSRS-based builds still succeed?

I'm genuinely unsure. The logic in the spack package is too hard to follow. The existing AM3 spack.yaml did not specify revisions or variants for these components- did that mean it took the revision from the specified UM revision?

I will do some testing today.

CI is failing with

Run . /opt/spack/share/spack/setup-env.sh ==> Error: cannot load package 'um' from the 'access.nri' repository: name '_rev_variants' is not defined ==> Created environment default in: /opt/environments/default ==> Activate with: spack env activate default ==> Activated default environment in /opt/environments/default ==> Error: cannot load package 'um' from the 'access.nri' repository: name '_rev_variants' is not defined Error: Process completed with exit code 1.

It is probably best to revert your removal of the _rev variants so that the remainder of the CI check can proceed.

Also, it is may be better to migrate from MOSRS to GitHub gradually. I will need to think of how to do this.

For the UM Spack build, the main configuration elements are the model/*/rose-app.conf files. The Python code for the UM Spack package recipe starts from the appropriate rose-app.conf file corresponding to the version, modifies this config based on the variants provided by the Spack spec, exports the result to corresponding environment variables, and then runs fcm make using fcm-make.cfg. The fcm make run depends only on the environment variables and the contents of fcm-make.cfg. In particular, the config_root_path variable determines where

include = $config_root_path/fcm-make/$platform_config_dir/um-$config_type-$optimisation_level.cfg$config_revision

points to. If (e.g.) config_root_path is left as fcm:um.xm_tr, then fcm make will use the UM trunk from the MOSRS mirror. See #362

At this point, I would recommend waiting until I have made the changes needed for #362 to be merged.

penguian

Is the intent to change all UM builds to use GitHub only, or just the AM3 builds?

penguian · 2026-02-17T23:46:44Z

If the intent is to change all UM builds, what testing has been done with rAM3?

Whyborn · 2026-02-18T01:54:09Z

Is the intent to change all UM builds to use GitHub only, or just the AM3 builds?

I don't see why we wouldn't. I've confirmed bitwise equivalent results for AM3, so we should be able to get the same for other components. What are the advantages to maintaining support for MOSRS components?

If the intent is to change all UM builds, what testing has been done with rAM3?

No testing has been done yet, but we don't need to update its spack-packages version yet.

Whyborn · 2026-02-18T01:56:57Z

@penguian Do you know the history as to why GCOM got its own spack package, but the other UKMO components did not e.g. socrates, ukca? Should we be making spack packages for these components?

penguian · 2026-02-18T03:58:36Z

@Whyborn GCOM is compiled separately as a library. The other "components" are cherry-picked for source code extraction by FCM and are compiled into a monolithic UM executable. Short of re-architecting the other components for standalone build, or refactoring the FCM make so that it works differently from the way it does now, the simplest thing to do was leave the FCM make as-is, and use its features while respecting its limitations.

penguian · 2026-02-18T04:04:57Z

Is the intent to change all UM builds to use GitHub only, or just the AM3 builds?

I don't see why we wouldn't. I've confirmed bitwise equivalent results for AM3, so we should be able to get the same for other components. What are the advantages to maintaining support for MOSRS components?

At this point, there is no advantage in maintaining MOSRS support. The main "gotcha" that I see coming is that the UKMO builds from UM 14.X GitHub differently from the way that we build from UM 13.X, so when we get to UM 14.X we will probably need to revisit the Spack builds. Hopefully by that stage we can do away with FCM -- but only if FAB works better with GitHub.

It is probably wise to test building with rAM3 now to make sure that we don't have to make changes for rAM3 to work that will affect AM3 as well.

Whyborn · 2026-02-18T23:04:59Z

@Whyborn GCOM is compiled separately as a library. The other "components" are cherry-picked for source code extraction by FCM and are compiled into a monolithic UM executable. Short of re-architecting the other components for standalone build, or refactoring the FCM make so that it works differently from the way it does now, the simplest thing to do was leave the FCM make as-is, and use its features while respecting its limitations.

Ah ok, so it's the only one that's a true library.

The main "gotcha" that I see coming is that the UKMO builds from UM 14.X GitHub differently from the way that we build from UM 13.X, so when we get to UM 14.X we will probably need to revisit the Spack builds. Hopefully by that stage we can do away with FCM -- but only if FAB works better with GitHub.

Are we ever planning to upgrade to UM14? Do you think it'd be possible and/or productive to just CMake-ify the UM stack?

penguian · 2026-02-19T00:47:56Z

As for UM 14 or LFRic, ask @heidinett or Chermell Engel. They might have a better handle on what the Atmosphere WG intends to do with ACCESS-rAM3 in future, vs what ACCESS-NRI is doing to track the progress of Momentum Partnership regional workflows and RAL4. (For example, see https://code.metoffice.gov.uk/trac/rmed/query?group=status&milestone=RAL4 ).

penguian

Wait for pull request #362.

penguian · 2026-02-19T00:58:53Z

-        "jules_rev",
-        "shumlib_rev",
-        "socrates_rev")
-


I will do some testing today.

penguian · 2026-02-19T01:01:04Z

    homepage = "https://code.metoffice.gov.uk/trac/gcom"
-    svn = "file:///g/data/ki32/mosrs/gcom/main/trunk"
-
+    # svn = "file:///g/data/ki32/mosrs/gcom/main/trunk"


If you want to remove a line, just remove it, otherwise explain in the comment why the line is commented out.

penguian · 2026-02-19T01:01:48Z

-
+    # svn = "file:///g/data/ki32/mosrs/gcom/main/trunk"
+    git = "https://github.com/ACCESS-NRI/gcom"
    maintainers("scottwales", "paulleopardi")


Should you add yourself to maintainers?

penguian · 2026-02-19T01:21:50Z

-        "jules_rev",
-        "shumlib_rev",
-        "socrates_rev")
-


CI is failing with

Run . /opt/spack/share/spack/setup-env.sh ==> Error: cannot load package 'um' from the 'access.nri' repository: name '_rev_variants' is not defined ==> Created environment default in: /opt/environments/default ==> Activate with: spack env activate default ==> Activated default environment in /opt/environments/default ==> Error: cannot load package 'um' from the 'access.nri' repository: name '_rev_variants' is not defined Error: Process completed with exit code 1.

It is probably best to revert your removal of the _rev variants so that the remainder of the CI check can proceed.

penguian · 2026-02-19T01:29:04Z

-        "jules_rev",
-        "shumlib_rev",
-        "socrates_rev")
-


Also, it is may be better to migrate from MOSRS to GitHub gradually. I will need to think of how to do this.

penguian · 2026-02-19T01:36:47Z

-        "jules_rev",
-        "shumlib_rev",
-        "socrates_rev")
-


For the UM Spack build, the main configuration elements are the model/*/rose-app.conf files. The Python code for the UM Spack package recipe starts from the appropriate rose-app.conf file corresponding to the version, modifies this config based on the variants provided by the Spack spec, exports the result to corresponding environment variables, and then runs fcm make using fcm-make.cfg. The fcm make run depends only on the environment variables and the contents of fcm-make.cfg. In particular, the config_root_path variable determines where

include = $config_root_path/fcm-make/$platform_config_dir/um-$config_type-$optimisation_level.cfg$config_revision

points to. If (e.g.) config_root_path is left as fcm:um.xm_tr, then fcm make will use the UM trunk from the MOSRS mirror. See #362

penguian · 2026-02-19T01:38:06Z

-        "jules_rev",
-        "shumlib_rev",
-        "socrates_rev")
-


At this point, I would recommend waiting until I have made the changes needed for #362 to be merged.

penguian · 2026-03-12T23:38:11Z

See #400

Whyborn · 2026-03-17T04:46:11Z

Superseded by #400

Whyborn added 13 commits February 10, 2026 16:07

Change UKCA source

7eabff3

Fix missing bracket

26e2087

Missed a _ref on a name

743214a

Add shumlib, socrates, casim to github refs

381aaf3

Set GCOM source to Github

ba20703

Fix incorrect syntax

c0f985e

Change UKCA source

a68a97e

Fix missing bracket

34a9f2d

Missed a _ref on a name

0e6d9e9

Add shumlib, socrates, casim to github refs

25cb004

Set GCOM source to Github

381e0ef

Fix incorrect syntax

67bbaaf

Rebase onto api-v2

374891d

Whyborn requested review from harshula and penguian February 17, 2026 03:35

Remove _rev_variants from um package

89c8977

penguian reviewed Feb 17, 2026

View reviewed changes

ci: Remove um and gcom from excluded packages list

2302f86

penguian requested changes Feb 19, 2026

View reviewed changes

Whyborn and others added 2 commits February 23, 2026 14:52

Remove all references to _rev_variants

43e0d97

Merge branch 'api-v2' into 381-pull-UKMO-components-from-github

5f4de99

This was referenced Mar 10, 2026

Migrate um and um_createbc to Spack 1.1 and GitHub #397

Closed

Refactor the UM Spack package to use a new build system, add a Spack package for um-createbc, and migrate to GitHub #400

Merged

Whyborn closed this Mar 17, 2026

Whyborn deleted the 381-pull-UKMO-components-from-github branch March 17, 2026 04:47

Conversation

Whyborn commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

penguian commented Feb 17, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

penguian left a comment

Choose a reason for hiding this comment

Uh oh!

penguian commented Feb 17, 2026

Uh oh!

Whyborn commented Feb 18, 2026

Uh oh!

Whyborn commented Feb 18, 2026

Uh oh!

penguian commented Feb 18, 2026

Uh oh!

penguian commented Feb 18, 2026

Uh oh!

Whyborn commented Feb 18, 2026

Uh oh!

penguian commented Feb 19, 2026

Uh oh!

penguian left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

penguian commented Mar 12, 2026

Uh oh!

Whyborn commented Mar 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Whyborn commented Feb 17, 2026 •

edited

Loading