Skip to content

Conversation

@dougiesquire
Copy link
Collaborator

This PR is to test a prerelease build using the oasis3-mct master branch and demonstrate that this does not change answers.

pr123-1 uses oasis3-mct built from the master branch and is otherwise identical to 2025.07.000
@dougiesquire
Copy link
Collaborator Author

!test repro

@github-actions
Copy link

✅ The Bitwise Reproducibility Check Succeeded ✅

When comparing:

  • preindustrial+concentrations-oasis3-master (checksums created using commit 5d216e9), against
  • dev-preindustrial+concentrations (checksums in commit 9fa7b7f)
Further information

The experiment can be found on Gadi at /scratch/tm70/repro-ci/experiments/access-esm1.6-configs/5d216e901d34ba23ebf9a787be7b1433be3bcf12, and the test results at https://github.com/ACCESS-NRI/access-esm1.6-configs/runs/48484765395.

The checksums generated by this !test command are found in the testing/checksum directory of https://github.com/ACCESS-NRI/access-esm1.6-configs/actions/runs/17097195840/artifacts/3808177447.

The checksums compared against are found here https://github.com/ACCESS-NRI/access-esm1.6-configs/tree/9fa7b7f4545bc6764bb175dcdf4b1b835f0fe8cf/testing/checksum

Test summary:
test_repro_historical
test_repro_determinism

pr123-2 uses the oasis3-mct access-om2 spack version
@dougiesquire
Copy link
Collaborator Author

!test repro

@github-actions
Copy link

❌ The Bitwise Reproducibility Check Failed ❌

When comparing:

  • preindustrial+concentrations-oasis3-master (checksums created using commit 0fde150), against
  • dev-preindustrial+concentrations (checksums in commit 9fa7b7f)
Further information

The experiment can be found on Gadi at /scratch/tm70/repro-ci/experiments/access-esm1.6-configs/0fde15019f4fd08670695f185862d03360655587, and the test results at https://github.com/ACCESS-NRI/access-esm1.6-configs/runs/48961712100.

The checksums generated by this !test command are found in the testing/checksum directory of https://github.com/ACCESS-NRI/access-esm1.6-configs/actions/runs/17253731746/artifacts/3859004845.

The checksums compared against are found here https://github.com/ACCESS-NRI/access-esm1.6-configs/tree/9fa7b7f4545bc6764bb175dcdf4b1b835f0fe8cf/testing/checksum

Test summary:
test_repro_determinism
test_repro_historical

@dougiesquire
Copy link
Collaborator Author

test_repro_determinism failed with:

CICE: ERROR failed to open input_ice.nml. Error code:  29  - file not found, un
 it 11, file /scratch/tm70/repro-ci/experiments/access-esm1.6-configs/0fde15019f
 4fd08670695f185862d03360655587/lab/work/exp_1d_runtime_repeat/ocean/input_ice.n
 ml

This is a known issue that occurs randomly. Will rerun.

@dougiesquire
Copy link
Collaborator Author

!test repro

@github-actions
Copy link

✅ The Bitwise Reproducibility Check Succeeded ✅

When comparing:

  • preindustrial+concentrations-oasis3-master (checksums created using commit 0fde150), against
  • dev-preindustrial+concentrations (checksums in commit 9fa7b7f)
Further information

The experiment can be found on Gadi at /scratch/tm70/repro-ci/experiments/access-esm1.6-configs/0fde15019f4fd08670695f185862d03360655587, and the test results at https://github.com/ACCESS-NRI/access-esm1.6-configs/runs/48962303635.

The checksums generated by this !test command are found in the testing/checksum directory of https://github.com/ACCESS-NRI/access-esm1.6-configs/actions/runs/17253951757/artifacts/3859073925.

The checksums compared against are found here https://github.com/ACCESS-NRI/access-esm1.6-configs/tree/9fa7b7f4545bc6764bb175dcdf4b1b835f0fe8cf/testing/checksum

Test summary:
test_repro_historical
test_repro_determinism

@dougiesquire
Copy link
Collaborator Author

I've done some basic testing of the performance impact of this change. I've run two experiments:

  • Exp 1 uses the current dev-preindustrial+concentrations config as is (uses access-esm1p6/2025.07.000).
  • Exp 2 uses the same configuration, but modified to use access-esm1p6/pr123-2. This prerelease uses oasis3-mct built from the master branch using the access-om2 version.

The experiments were run at the same time. Times below are the PAYU_WALLTIME from job.yaml for 2 x 1 year:

Exp 1 Exp 2
year 1 1hr 15mins 1hr 15mins
year 2 1hr 15mins 1hr 16mins

So moving to using the master branch of oasis3-mct and switching from -xCORE-AVX512 to -axCORE-AVX2:

  1. does not change answers - see above
  2. has no obvious impact on performance

@manodeep, @harshula, @blimlim, @penguian, I think we agreed that with the above confirmed we are happy to use -axCORE-AVX2 in ACCESS-ESM1.6 as in ACCESS-OM2. I'll follow up with the change to the oasis3-mct SPR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant