-
Notifications
You must be signed in to change notification settings - Fork 1
Build and deployment documentation #280
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
|
@atteggiani The config docs for ESM1.6 fail to deploy with the preview. Does it need all the PR previews to work in order to deploy? I'm happy to go and fix PR 252, just making sure that should solve the problem before investing time. |
Yes it needs all PR previews to work, unfortunately. Then you would need to re-trigger this PR. You cannot simply retrigger the build manually, so you'll have to add a new commit or amend (no edits) and force push the latest one. |
|
Thanks @ccarouge, fwiw, you might find some of this material helpful: |
Sure thing, @anton-seaice might be a helpful person to canvas too. |
Added information on required repositories and building methods for ACCESS-ESM1.6.
| If you need to modify the source code for ACCESS-ESM1.6, you will need to use the Spack infrastructure to create your own build. There are two ways to do so (on GitHub or locally) both with benefits and limitations. | ||
| In order to build your own version of ACCESS-ESM1.6, you will need read access to the following repositories: | ||
|
|
||
| - https://github.com/ACCESS-NRI/cice5 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| - https://github.com/ACCESS-NRI/cice5 | |
| - [github.com/ACCESS-NRI/cice5](https://github.com/ACCESS-NRI/cice5) |
And so on for the other links...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. But I have completely redesigned the build page.
aidanheerdegen
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A couple of comments, but otherwise seems good, thanks.
…higher up in mkdocs.yml Got tired of having to scroll down or search for the nav entry when it's the bit we modify the most.
|
@anton-seaice Could you review this, please? I will be away so if you can review for 2nd Feb that would be great. |
…ent-documentation
anton-seaice
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry a bit of a chaotic review, I can have another look as needed
|
|
||
| ## Software structure | ||
|
|
||
| ACCESS-ESM1.6 being a Earth System model, it is constituted of several model components, each residing in its own repository on GitHub. In addition, some model components need other utilities at compilation time that are stored in separate repositories. This results in a complex software structure involving a large number of repositories: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| ACCESS-ESM1.6 being a Earth System model, it is constituted of several model components, each residing in its own repository on GitHub. In addition, some model components need other utilities at compilation time that are stored in separate repositories. This results in a complex software structure involving a large number of repositories: | |
| ACCESS-ESM1.6 being a Earth System model, it is constituted of several model components, each residing in its own repository on GitHub. In addition, some model components need other utilities at compilation time that are stored in separate repositories. This results in a complex software structure involving a number of repositories, including: |
|
|
||
| ## Building ACCESS-ESM1.6 using the deployment infrastructure | ||
|
|
||
| This is the recommended method to build ad-hoc versions of ACCESS-ESM1.6 because it does not require users to have access to all the repositories, it handles the build automatically and it provides traceability and shareability of the deployment. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It needs an introduction sentence
e.g. It is possible to:
- semi-automatically build ACCESS-ESM1.6 using github actions to trigger a build
- and make that available to build on gadi.
- this is good because it requires much less setup than using spack directly.
| # Build infrastructure | ||
|
|
||
| Most users of ACCESS-ESM1.6 will not need to build the model. Released versions of the model are deployed by ACCESS-NRI to Gadi and available to users. Check the [Deployment page](/documentation/docs/pages/infrastructure/Deployment.md) to learn about available deployments. If you require your own build of the model, typically because you need to modify one or several model components, you will find the required information on this Builing page. | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to define release and deploy, we use them both, slightly interchangeably
| - Deployment: pages/infrastructure/Deployment.md | ||
| - Building: pages/infrastructure/Building.md | ||
| - Git practices: pages/infrastructure/Git-practices.md | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why did this section get moved ?
|
|
||
| ## Local build of ACCESS-ESM1.6 | ||
|
|
||
| The Spack infrastructure allows you to create your own local build on _Gadi_ for ACCESS-ESM1.6. Once setup, this might be faster than using the deployment infrastructure, especially when you want a lot of small incremental builds during development. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| The Spack infrastructure allows you to create your own local build on _Gadi_ for ACCESS-ESM1.6. Once setup, this might be faster than using the deployment infrastructure, especially when you want a lot of small incremental builds during development. | |
| The Spack infrastructure allows you to create your own local build on _Gadi_ for ACCESS-ESM1.6. Once setup, this is faster than using the deployment infrastructure, especially when writing new code. |
|
|
||
| There are development releases (identified with a `dev_` prefix to the date tag) and full releases (identified with only a date tag). We recommend new users start with the newest full release (i.e., with the most recent date tag). | ||
|
|
||
| These builds are optimised for the Sapphire-Rapid hardware from Intel (i.e., the *normalsr* queue), and may not run on Cascade-Lake (i.e., the *normal* queue) or older hardware. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm pretty confident it would run fine on normal. I can't see any HW specific optimisation going on
| @@ -0,0 +1,29 @@ | |||
| # Deployment | |||
|
|
|||
| All released versions of ACCESS-ESM1.6 come with a pre-deployed build on _Gadi_. Deployed builds are stored under the project `vk83` (hence requiring [membership to this project](https://my.nci.org.au/mancini/project/vk83/join) for access). You can see all deployed builds via: | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mostly the words a "release version" would refer to a configuration, not a model exe
I think the point here is - release configurations use a certain deployment of the model code. Mostly users wouldn't mess with that.
Up to date configs will use the latest deployment, the older ones are kept for folks who are running experiments based on older configurations
| @@ -0,0 +1,29 @@ | |||
| # Deployment | |||
|
|
|||
| All released versions of ACCESS-ESM1.6 come with a pre-deployed build on _Gadi_. Deployed builds are stored under the project `vk83` (hence requiring [membership to this project](https://my.nci.org.au/mancini/project/vk83/join) for access). You can see all deployed builds via: | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
under the project `vk83` (hence requiring [membership to this project](https://my.nci.org.au/mancini/project/vk83/join) for access).
this project membership is surely self evident by this point, its where payu is available from
| module avail access-esm1p6 | ||
| ``` | ||
|
|
||
| There are development releases (identified with a `dev_` prefix to the date tag) and full releases (identified with only a date tag). We recommend new users start with the newest full release (i.e., with the most recent date tag). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This isn't really a page for new users. What we actually recommend is that new users use a released configuration
| @@ -0,0 +1,44 @@ | |||
| # Build infrastructure | |||
|
|
|||
| Most users of ACCESS-ESM1.6 will not need to build the model. Released versions of the model are deployed by ACCESS-NRI to Gadi and available to users. Check the [Deployment page](/documentation/docs/pages/infrastructure/Deployment.md) to learn about available deployments. If you require your own build of the model, typically because you need to modify one or several model components, you will find the required information on this Builing page. | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The link to Deployment page doesn't work , this might be correct
| Most users of ACCESS-ESM1.6 will not need to build the model. Released versions of the model are deployed by ACCESS-NRI to Gadi and available to users. Check the [Deployment page](/documentation/docs/pages/infrastructure/Deployment.md) to learn about available deployments. If you require your own build of the model, typically because you need to modify one or several model components, you will find the required information on this Builing page. | |
| Most users of ACCESS-ESM1.6 will not need to build the model. Released versions of the model are deployed by ACCESS-NRI to Gadi and available to users. Check the [Deployment page](infrastructure/Deployment.md) to learn about available deployments. If you require your own build of the model, typically because you need to modify one or several model components, you will find the required information on this Builing page. |
1. Summary:
Added documentation about the building and deployment process for ACCESS-ESM1.6
2. Issues Addressed:
3. Dependencies (e.g. on payu, or model)
This change requires changes to (note pull request(s) where relevant):
4. Ad-hoc Testing
What ad-hoc testing was done? How are you convinced this change is correct (plots are good)?
Documentation only. No testing done. Relying on build preview in PR to correct formatting.
5. CI Testing
!test reprohas been run6. Reproducibility
Is this reproducible with the previous commit? (If not, why not?)
!test repro commithas been run.8. Manifests
Have you changed the executable, the input files and/or the restart files?
If yes, have you updated the manifests?
To update the manifests, run payu setup (in a cloned copy of your feature branch) with reproducibility tests turned off:
Then commit the newly created manifest files (under manifests/) only to the branch for this PR.
7. Documentation
Is the documentation updated?
9. Merge Strategy