Test replacing -xCORE-AVX2 with -mavx2 by dougiesquire · Pull Request #192 · ACCESS-NRI/ACCESS-ESM1.6

dougiesquire · 2026-02-25T03:52:55Z

This PR is to test replacing -xCORE*/-xavx/-axCORE-* with -mavx2 across the ACCESS-ESM1.6 stack

🚀 The latest prerelease access-esm1p6/pr192-11 at d863eb5 is here: #192 (comment) 🚀

…RE-AVX512)

github-actions · 2026-02-25T04:04:20Z

🚀 Attempted to deploy access-esm1p6 Prerelease pr192-1 with commit 822f79e

🖥️ `Gadi` Deployment ❌

github-actions · 2026-02-25T05:58:33Z

🚀 Attempted to deploy access-esm1p6 Prerelease pr192-2 with commit d433092

🖥️ `Gadi` Deployment ❌

github-actions · 2026-02-25T10:12:36Z

🚀 Attempted to deploy access-esm1p6 Prerelease pr192-3 with commit 613ef1e

🖥️ `Gadi` Deployment ✔️

Usage Instructions

access-esm1.6, defined in ./spack.yaml, will be deployed to Gadi as:

2026.02.000 as a Release (when merged).
pr192-3 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr192-3

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr192-3 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version 97f928e45d1d7a9f930a5475b8cdb9d3261db472
access-nri/spack-config version 2026.02.000

If the above was not what was expected, commit changes to config/versions.json in this PR.

github-actions · 2026-02-25T10:36:25Z

🚀 Attempted to deploy access-esm1p6 Prerelease pr192-4 with commit 1df1f9c

🖥️ `Gadi` Deployment ✔️

Usage Instructions

access-esm1.6, defined in ./spack.yaml, will be deployed to Gadi as:

2026.02.000 as a Release (when merged).
pr192-4 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr192-4

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr192-4 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version 97f928e45d1d7a9f930a5475b8cdb9d3261db472
access-nri/spack-config version 2026.02.000

If the above was not what was expected, commit changes to config/versions.json in this PR.

github-actions · 2026-02-25T10:53:25Z

🚀 Attempted to deploy access-esm1p6 Prerelease pr192-5 with commit 5c77cfe

🖥️ `Gadi` Deployment ✔️

Usage Instructions

access-esm1.6, defined in ./spack.yaml, will be deployed to Gadi as:

2026.02.000 as a Release (when merged).
pr192-5 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr192-5

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr192-5 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version 9193e827018c93e64ca16987d7d732ac8961ce92
access-nri/spack-config version 2026.02.000

If the above was not what was expected, commit changes to config/versions.json in this PR.

dougiesquire · 2026-02-25T21:35:48Z

@manodeep, there's three prereleases here that hopefully will be useful for testing performance:

access-esm1p6/pr192-3: Uses -mavx2 for everything except gcom4 (uses -xavx) and um7 (uses -xCORE-AVX512)
access-esm1p6/pr192-4: Uses -mavx2 for everything except um7 (uses -xCORE-AVX512)
access-esm1p6/pr192-5: Uses -mavx2 for everything

Would you be able to run your usual tests?

manodeep · 2026-02-26T19:36:23Z

Piggy-backing off this PR to test the performance impact of the other extreme of adding sapphirerapids target

github-actions · 2026-02-26T19:41:27Z

🚀 Attempted to deploy access-esm1p6 Prerelease pr192-6 with commit 6c25ef9

🖥️ `Gadi` Deployment ❌

manodeep · 2026-02-26T19:48:32Z

The performance numbers for the previous builds are

yrs/Wday	WTime (imbal.)	SUs	CPUs	yrs	yrs/core/Wday	Atm Wt(GO2A, PO2A, Wt-O2A)	Ocn Wt (recvs, Wt - recvs)	ATM, OCN layouts	Directory
25.3	57.0 (12.4%)	994.4	520	1.0	4.9e-02	55.8 ( 2.2, 0.3, 53)	56.6 ( 4.6, 52)	16x16, 16x15	20260226-PI-config-base (output000)
24.9	57.8 ( 9.8%)	1014.5	520	1.0	4.8e-02	56.5 ( 2.5, 0.3, 54)	57.4 ( 2.9, 55)	16x16, 16x15	20260226-PI-config-base-pr192-4 (output000)
24.7	58.2 (10.9%)	1021.0	520	1.0	4.8e-02	57.0 ( 3.3, 0.3, 53)	57.9 ( 2.7, 55)	16x16, 16x15	20260226-PI-config-base-pr192-3 (output002)
24.7	58.4 (10.2%)	1021.7	520	1.0	4.7e-02	57.2 ( 2.6, 0.3, 54)	58.1 ( 3.0, 55)	16x16, 16x15	20260226-PI-config-base-pr192-5 (output002)

Printing summary table of top 4 results (all times are in mins, sorted by 'Simulated Years per wallclock-day'):

yrs/Wallday	WTime (mins)	SUs	CPUs	yrs/core/Wday	Directory
25.3	57.0	994	520	4.9e-02	20260226-PI-config-base (output000)
24.9	57.8	1014	520	4.8e-02	20260226-PI-config-base-pr192-4 (output000)
24.7	58.2	1021	520	4.8e-02	20260226-PI-config-base-pr192-3 (output002)
24.7	58.4	1022	520	4.7e-02	20260226-PI-config-base-pr192-5 (output002)

edit All the runs were bitwise identical.

[~/perf-esm1p6/test-mavx2-vs-avx512 @gadi-login-04]$ ~/codes/utils/check_for_repro_multi_dirs.sh *
Checking that all arguments are directories ...done
Checking that all arguments are unique directories ...done
Comparing Atmosphere in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-3): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-3): Status -  PASSED
Comparing Atmosphere in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-4): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-4): Status -  PASSED
Comparing Atmosphere in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-5): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-5): Status -  PASSED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-3,20260226-PI-config-base-pr192-4): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-3,20260226-PI-config-base-pr192-4): Status -  PASSED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-3,20260226-PI-config-base-pr192-5): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-3,20260226-PI-config-base-pr192-5): Status -  PASSED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-4,20260226-PI-config-base-pr192-5): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-4,20260226-PI-config-base-pr192-5): Status -  PASSED

manodeep · 2026-02-26T19:53:25Z

!redeploy

github-actions · 2026-02-26T19:58:49Z

🚀 Attempted to deploy access-esm1p6 Prerelease pr192-7 with commit 6c25ef9

🖥️ `Gadi` Deployment ❌

github-actions · 2026-02-27T00:13:58Z

🚀 Attempted to deploy access-esm1p6 Prerelease pr192-8 with commit 576438f

🖥️ `Gadi` Deployment ❌

manodeep · 2026-02-27T00:25:24Z

Sadly the compile job got killed (exceeding memory/time on the login node)

manodeep · 2026-02-27T00:25:33Z

!redeploy

github-actions · 2026-02-27T01:27:32Z

🚀 Attempted to deploy access-esm1p6 Prerelease pr192-9 with commit 576438f

🖥️ `Gadi` Deployment ✔️

Usage Instructions

access-esm1.6, defined in ./spack.yaml, will be deployed to Gadi as:

2026.02.000 as a Release (when merged).
pr192-9 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr192-9

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr192-9 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version 9193e827018c93e64ca16987d7d732ac8961ce92
access-nri/spack-config version 2026.02.000

If the above was not what was expected, commit changes to config/versions.json in this PR.

github-actions · 2026-02-27T02:32:51Z

🚀 Attempted to deploy access-esm1p6 Prerelease pr192-10 with commit a0ad398

🖥️ `Gadi` Deployment ✔️

Usage Instructions

access-esm1.6, defined in ./spack.yaml, will be deployed to Gadi as:

2026.02.000 as a Release (when merged).
pr192-10 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr192-10

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr192-10 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version 9193e827018c93e64ca16987d7d732ac8961ce92
access-nri/spack-config version 2026.02.000

If the above was not what was expected, commit changes to config/versions.json in this PR.

github-actions · 2026-02-27T03:07:11Z

🚀 Attempted to deploy access-esm1p6 Prerelease pr192-11 with commit d863eb5

🖥️ `Gadi` Deployment ✔️

Usage Instructions

access-esm1.6, defined in ./spack.yaml, will be deployed to Gadi as:

2026.02.000 as a Release (when merged).
pr192-11 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr192-11

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr192-11 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version 9193e827018c93e64ca16987d7d732ac8961ce92
access-nri/spack-config version 2026.02.000

If the above was not what was expected, commit changes to config/versions.json in this PR.

manodeep · 2026-02-27T06:49:58Z

Printing runtime and imbalance results (all times are in mins, sorted by 'Simulated Years per wallclock-day'):

yrs/Wday	WTime (imbal.)	SUs	CPUs	yrs	yrs/core/Wday	Atm Wt(GO2A, PO2A, Wt-O2A)	Ocn Wt (recvs, Wt - recvs)	ATM, OCN layouts	Directory
25.4	56.7 (15.3%)	1007.8	520	1.0	4.9e-02	55.5 ( 5.7, 0.3, 50)	56.4 ( 2.7, 54)	16x16, 16x15	20260226-PI-config-base-pr192-10 (output001)
25.3	57.0 (12.4%)	994.4	520	1.0	4.9e-02	55.8 ( 2.2, 0.3, 53)	56.6 ( 4.6, 52)	16x16, 16x15	20260226-PI-config-base (output000)
25.2	57.1 (15.6%)	1008.4	520	1.0	4.9e-02	55.8 ( 5.8, 0.3, 50)	56.7 ( 2.8, 54)	16x16, 16x15	20260226-PI-config-base-pr192-11 (output002)
24.9	57.8 ( 9.8%)	1014.5	520	1.0	4.8e-02	56.5 ( 2.5, 0.3, 54)	57.4 ( 2.9, 55)	16x16, 16x15	20260226-PI-config-base-pr192-4 (output000)
24.8	58.0 ( 9.9%)	1029.3	520	1.0	4.8e-02	56.8 ( 2.5, 0.3, 54)	57.6 ( 2.9, 55)	16x16, 16x15	20260226-PI-config-base-pr192-9 (output002)
24.7	58.2 (10.9%)	1021.0	520	1.0	4.8e-02	57.0 ( 3.3, 0.3, 53)	57.9 ( 2.7, 55)	16x16, 16x15	20260226-PI-config-base-pr192-3 (output002)
24.7	58.4 (10.2%)	1021.7	520	1.0	4.7e-02	57.2 ( 2.6, 0.3, 54)	58.1 ( 3.0, 55)	16x16, 16x15	20260226-PI-config-base-pr192-5 (output002)

Printing summary table of top 7 results (all times are in mins, sorted by 'Simulated Years per wallclock-day'):

yrs/Wallday	WTime (mins)	SUs	CPUs	yrs/core/Wday	Directory
25.4	56.7	1008	520	4.9e-02	20260226-PI-config-base-pr192-10 (output001)
25.3	57.0	994	520	4.9e-02	20260226-PI-config-base (output000)
25.2	57.1	1008	520	4.9e-02	20260226-PI-config-base-pr192-11 (output002)
24.9	57.8	1014	520	4.8e-02	20260226-PI-config-base-pr192-4 (output000)
24.8	58.0	1029	520	4.8e-02	20260226-PI-config-base-pr192-9 (output002)
24.7	58.2	1021	520	4.8e-02	20260226-PI-config-base-pr192-3 (output002)
24.7	58.4	1022	520	4.7e-02	20260226-PI-config-base-pr192-5 (output002)

Deploy 10 and 11 are not bitwise identical with the rest, but bitwise identical with each other. All the other builds are bitwise identical (as noted in a previous comment)

[~/perf-esm1p6/test-mavx2-vs-avx512 @gadi-login-06]$ ~/codes/utils/check_for_repro_multi_dirs.sh *
Checking that all arguments are directories ...done
Checking that all arguments are unique directories ...done
Comparing Atmosphere in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-10): Status -  FAILED
Comparing Ocean in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-10): Status -  FAILED
Comparing Atmosphere in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-11): Status -  FAILED
Comparing Ocean in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-11): Status -  FAILED
Comparing Atmosphere in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-3): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-3): Status -  PASSED
Comparing Atmosphere in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-4): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-4): Status -  PASSED
Comparing Atmosphere in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-5): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-5): Status -  PASSED
Comparing Atmosphere in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-9): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base,20260226-PI-config-base-pr192-9): Status -  PASSED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-10,20260226-PI-config-base-pr192-11): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-10,20260226-PI-config-base-pr192-11): Status -  PASSED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-10,20260226-PI-config-base-pr192-3): Status -  FAILED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-10,20260226-PI-config-base-pr192-3): Status -  FAILED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-10,20260226-PI-config-base-pr192-4): Status -  FAILED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-10,20260226-PI-config-base-pr192-4): Status -  FAILED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-10,20260226-PI-config-base-pr192-5): Status -  FAILED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-10,20260226-PI-config-base-pr192-5): Status -  FAILED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-10,20260226-PI-config-base-pr192-9): Status -  FAILED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-10,20260226-PI-config-base-pr192-9): Status -  FAILED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-11,20260226-PI-config-base-pr192-3): Status -  FAILED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-11,20260226-PI-config-base-pr192-3): Status -  FAILED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-11,20260226-PI-config-base-pr192-4): Status -  FAILED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-11,20260226-PI-config-base-pr192-4): Status -  FAILED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-11,20260226-PI-config-base-pr192-5): Status -  FAILED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-11,20260226-PI-config-base-pr192-5): Status -  FAILED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-11,20260226-PI-config-base-pr192-9): Status -  FAILED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-11,20260226-PI-config-base-pr192-9): Status -  FAILED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-3,20260226-PI-config-base-pr192-4): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-3,20260226-PI-config-base-pr192-4): Status -  PASSED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-3,20260226-PI-config-base-pr192-5): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-3,20260226-PI-config-base-pr192-5): Status -  PASSED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-3,20260226-PI-config-base-pr192-9): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-3,20260226-PI-config-base-pr192-9): Status -  PASSED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-4,20260226-PI-config-base-pr192-5): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-4,20260226-PI-config-base-pr192-5): Status -  PASSED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-4,20260226-PI-config-base-pr192-9): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-4,20260226-PI-config-base-pr192-9): Status -  PASSED
Comparing Atmosphere in dirs = (20260226-PI-config-base-pr192-5,20260226-PI-config-base-pr192-9): Status -  PASSED
Comparing Ocean in dirs = (20260226-PI-config-base-pr192-5,20260226-PI-config-base-pr192-9): Status -  PASSED

Performance slow-downs or improvements are in the few % level - nothing major as the 12% drop seen in OM2. However, UM7 seems to be waiting longer for MOM5 - so adding march=sapphirerapids improves UM performance more than it does for MOM5 (and potentially a slightly tweaked partitioning could result in a better load-balance)

manodeep · 2026-03-03T11:33:02Z

One thing was bothering me that even with -O2 for gcom4 (deployment 11, also includes -march=sapphirerapids -mtune=sapphirerapids for all built components), the PI-config performance did not improve much. However, since it is a coupled run, even if UM was performing much better, it would still be constrained to run at the speed of MOM. To test the UM performance more cleanly, I ran the AMIP base config, and then the 11 deployment, and there is a 5% improvement in performance in the 2-node config. Here is the performance summary:

yrs/Wallday	WTime (mins)	SUs	CPUs	yrs/core/Wday	Directory
25.2	57.2	134	208	1.2e-01	20260303-dev-amip-pr192-11 (output001)
23.9	60.2	148	208	1.2e-01	20260303-dev-amip (output001)

Results are bitwise identical - this is in contrast to PI-config, where the results were different. Not sure if that is expected, or why that would be the case.

[~/perf-esm1p6/test-amip-with-gcom4-O2 @gadi-login-01]$ ~/codes/utils/check_for_repro_multi_dirs.sh *
Checking that all arguments are directories ...done
Checking that all arguments are unique directories ...done
Comparing Atmosphere in dirs = (20260303-dev-amip,20260303-dev-amip-pr192-11): Status -  PASSED

In case the new build scales better, I am now running two tests with 400 cores - one with the base config (which I am reasonably confident does not scale well to 4-nodes) and another for the deployment 11. Let's see what the numbers say.

manodeep · 2026-03-03T21:19:56Z

Updated summary table:

Printing summary table of top 6 results (all times are in mins, sorted by 'Simulated Years per wallclock-day'):

yrs/Wallday	WTime (mins)	SUs	CPUs	yrs/core/Wday	Directory
28.5	50.6	178	312	9.1e-02	20260303-dev-amip-pr192-11-3node-306cores (output001)
27.3	52.8	250	416	6.6e-02	20260303-dev-amip-pr192-11-4nodes-400cores (output000)
25.7	56.0	273	416	6.2e-02	20260303-dev-amip-4nodes-400cores (output001)
25.2	57.2	134	208	1.2e-01	20260303-dev-amip-pr192-11 (output001)
23.9	60.2	148	208	1.2e-01	20260303-dev-amip (output001)
15.9	90.5	101	100	1.6e-01	20260303-dev-amip-pr192-11-1node-100cores (output000)

From these runs, there are no bitwise identical pairs, except for the 2-node configs that I ran last night.

[~/perf-esm1p6/test-amip-with-gcom4-O2 @gadi-login-04]$ ~/codes/utils/check_for_repro_multi_dirs.sh *
Checking that all arguments are directories ...done
Checking that all arguments are unique directories ...done
Comparing Atmosphere in dirs = (20260303-dev-amip,20260303-dev-amip-4nodes-400cores): Status -  FAILED
Comparing Atmosphere in dirs = (20260303-dev-amip,20260303-dev-amip-pr192-11): Status -  PASSED
Comparing Atmosphere in dirs = (20260303-dev-amip,20260303-dev-amip-pr192-11-1node-100cores): Status -  FAILED
Comparing Atmosphere in dirs = (20260303-dev-amip,20260303-dev-amip-pr192-11-3node-306cores): Status -  FAILED
Comparing Atmosphere in dirs = (20260303-dev-amip,20260303-dev-amip-pr192-11-4nodes-400cores): Status -  FAILED
Comparing Atmosphere in dirs = (20260303-dev-amip-4nodes-400cores,20260303-dev-amip-pr192-11): Status -  FAILED
Comparing Atmosphere in dirs = (20260303-dev-amip-4nodes-400cores,20260303-dev-amip-pr192-11-1node-100cores): Status -  FAILED
Comparing Atmosphere in dirs = (20260303-dev-amip-4nodes-400cores,20260303-dev-amip-pr192-11-3node-306cores): Status -  FAILED
Comparing Atmosphere in dirs = (20260303-dev-amip-4nodes-400cores,20260303-dev-amip-pr192-11-4nodes-400cores): Status -  FAILED
Comparing Atmosphere in dirs = (20260303-dev-amip-pr192-11,20260303-dev-amip-pr192-11-1node-100cores): Status -  FAILED
Comparing Atmosphere in dirs = (20260303-dev-amip-pr192-11,20260303-dev-amip-pr192-11-3node-306cores): Status -  FAILED
Comparing Atmosphere in dirs = (20260303-dev-amip-pr192-11,20260303-dev-amip-pr192-11-4nodes-400cores): Status -  FAILED
Comparing Atmosphere in dirs = (20260303-dev-amip-pr192-11-1node-100cores,20260303-dev-amip-pr192-11-3node-306cores): Status -  FAILED
Comparing Atmosphere in dirs = (20260303-dev-amip-pr192-11-1node-100cores,20260303-dev-amip-pr192-11-4nodes-400cores): Status -  FAILED
Comparing Atmosphere in dirs = (20260303-dev-amip-pr192-11-3node-306cores,20260303-dev-amip-pr192-11-4nodes-400cores): Status -  FAILED

Use mavx2 for everything except gcom4 (uses -xavx) and um7 (uses -xCO…

822f79e

…RE-AVX512)

dougiesquire had a problem deploying to Gadi Prerelease February 25, 2026 03:53 — with GitHub Actions Failure

Update to spack-packages branch based on api-v1

d433092

dougiesquire had a problem deploying to Gadi Prerelease February 25, 2026 05:13 — with GitHub Actions Failure

Correct mom5 hash after rebase

613ef1e

dougiesquire temporarily deployed to Gadi Prerelease February 25, 2026 09:56 — with GitHub Actions Inactive

Update gcom4 to use -mavx2 instead of -xavx

1df1f9c

dougiesquire temporarily deployed to Gadi Prerelease February 25, 2026 10:18 — with GitHub Actions Inactive

Update um7 to use -mavx2 instead of -xCORE-AVX512

5c77cfe

dougiesquire temporarily deployed to Gadi Prerelease February 25, 2026 10:37 — with GitHub Actions Inactive

Set target to sapphirerapids

6c25ef9

manodeep had a problem deploying to Gadi Prerelease February 26, 2026 19:35 — with GitHub Actions Failure

Targets needs to be under concretizer

576438f

manodeep had a problem deploying to Gadi Prerelease February 26, 2026 23:17 — with GitHub Actions Failure

Added explicit march and mtune to all ACCESS packages

a0ad398

manodeep temporarily deployed to Gadi Prerelease February 27, 2026 01:57 — with GitHub Actions Inactive

Added O2 to gcom4 (does not have any by default)

d863eb5

manodeep temporarily deployed to Gadi Prerelease February 27, 2026 02:33 — with GitHub Actions Inactive

Conversation

dougiesquire commented Feb 25, 2026 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 25, 2026

🖥️ Gadi Deployment ❌

Uh oh!

github-actions bot commented Feb 25, 2026

🖥️ Gadi Deployment ❌

Uh oh!

github-actions bot commented Feb 25, 2026

🖥️ Gadi Deployment ✔️

Uh oh!

github-actions bot commented Feb 25, 2026

🖥️ Gadi Deployment ✔️

Uh oh!

github-actions bot commented Feb 25, 2026

🖥️ Gadi Deployment ✔️

Uh oh!

dougiesquire commented Feb 25, 2026

Uh oh!

manodeep commented Feb 26, 2026

Uh oh!

github-actions bot commented Feb 26, 2026

🖥️ Gadi Deployment ❌

Uh oh!

manodeep commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

manodeep commented Feb 26, 2026

Uh oh!

github-actions bot commented Feb 26, 2026

🖥️ Gadi Deployment ❌

Uh oh!

github-actions bot commented Feb 27, 2026

🖥️ Gadi Deployment ❌

Uh oh!

manodeep commented Feb 27, 2026

Uh oh!

manodeep commented Feb 27, 2026

Uh oh!

github-actions bot commented Feb 27, 2026

🖥️ Gadi Deployment ✔️

Uh oh!

github-actions bot commented Feb 27, 2026

🖥️ Gadi Deployment ✔️

Uh oh!

github-actions bot commented Feb 27, 2026

🖥️ Gadi Deployment ✔️

Uh oh!

manodeep commented Feb 27, 2026

Uh oh!

manodeep commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

manodeep commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dougiesquire commented Feb 25, 2026 •

edited by github-actions bot

Loading

🖥️ `Gadi` Deployment ❌

🖥️ `Gadi` Deployment ❌

🖥️ `Gadi` Deployment ✔️

🖥️ `Gadi` Deployment ✔️

🖥️ `Gadi` Deployment ✔️

🖥️ `Gadi` Deployment ❌

manodeep commented Feb 26, 2026 •

edited

Loading

🖥️ `Gadi` Deployment ❌

🖥️ `Gadi` Deployment ❌

🖥️ `Gadi` Deployment ✔️

🖥️ `Gadi` Deployment ✔️

🖥️ `Gadi` Deployment ✔️

manodeep commented Mar 3, 2026 •

edited

Loading

manodeep commented Mar 3, 2026 •

edited

Loading