Skip to content

Update container#4929

Merged
jasonb5 merged 39 commits intomasterfrom
fix-containers
Mar 12, 2026
Merged

Update container#4929
jasonb5 merged 39 commits intomasterfrom
fix-containers

Conversation

@jasonb5
Copy link
Collaborator

@jasonb5 jasonb5 commented Feb 9, 2026

Description

  • Migrates from conda to spack
  • Fix mixed perl environment (conda/system)
  • Moves to UV to manage python
  • Simplifies usage and testing workflow

Checklist

  • My code follows the style guidelines of this project (black formatting)
  • I have performed a self-review of my own code
  • My changes generate no new warnings
  • I have added tests that exercise my feature/fix and existing tests continue to pass
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding additions and changes to the documentation

Copy link
Contributor

@jedwards4b jedwards4b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a variable user_home which it seems like should replace the hardcoded
/home/cime throughout - but then you are also handling a case of root running so I'm a little lost as to the intent.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the project’s container image and CI entrypoint behavior to improve dependency handling (notably Perl), move inputdata download into the entrypoint logic, and adjust GitHub Actions to explicitly source the container entrypoint.

Changes:

  • Switch Docker base image reference to quay.io/condaforge/miniforge3 and adjust OS-level package installs (Perl + gosu).
  • Refactor docker/entrypoint.sh to handle HOME/.cime setup in CI/root scenarios and add helper functions (e.g., inputdata download, cprnc build).
  • Update CI workflow steps to manually source /entrypoint.sh and remove old .cime copy logic (plus a temporary debug listing).

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File Description
docker/entrypoint.sh Adds CI/root HOME handling, .cime relocation, and new helper functions; changes environment setup behavior.
docker/cime.yaml Adds editor dependency and enables HDF5 OpenMPI build in the conda env spec.
docker/Dockerfile Updates base image source, installs Perl/gosu via apt, and runs inputdata download via sourced entrypoint during build.
.github/workflows/testing.yml Sources /entrypoint.sh in jobs and removes legacy .cime copy; includes a temporary debug ls.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

COPY .cime .cime
COPY entrypoint.sh /entrypoint.sh

RUN bash -c "SKIP_ENTRYPOINT=true . /entrypoint.sh && download_input_data"
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Docker build step calls download_input_data, but the image no longer installs wget (or curl) via apt and it’s not guaranteed to be available in the base conda env. This can make the build fail when wget is missing. Install wget/curl in the image layer before this RUN step, or invoke the downloader via conda run -n <env> / an explicit path to the env’s wget.

Copilot uses AI. Check for mistakes.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added wget/curl/subversion to dockerfile.

Comment on lines +114 to +115
# Clobber PATH so system perl is used instead of conda perl
echo "export PATH=/usr/bin:\$PATH"
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generated .bashrc prepends /usr/bin to PATH after conda activation. This can cause system tools (e.g., python/cmake/mpirun) to be picked over the conda environment versions, breaking builds/tests. If the intent is only to force system Perl, use a more targeted approach (e.g., set PERL=/usr/bin/perl, hash -p /usr/bin/perl perl, or adjust only perl resolution) rather than reordering the entire PATH.

Suggested change
# Clobber PATH so system perl is used instead of conda perl
echo "export PATH=/usr/bin:\$PATH"
# Ensure system perl is used instead of conda perl without changing PATH order
echo "hash -p /usr/bin/perl perl"

Copilot uses AI. Check for mistakes.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Used both hash and PERL variable.

Comment on lines +4 to +6
export USER=`id -nu`
export LOGNAME="${USER}"
export USER_ID="${USER_ID:-1000}"
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use $(...) instead of legacy backticks for command substitution (easier to read/nest, less error-prone).

Copilot uses AI. Check for mistakes.
Comment on lines 239 to 241
# TODO remove just for testing
ls -la "${HOME}"

Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the temporary debug ls -la "${HOME}" step (and the TODO) before merging; it adds noise to CI logs and can inadvertently expose environment details in logs.

Suggested change
# TODO remove just for testing
ls -la "${HOME}"

Copilot uses AI. Check for mistakes.
@github-actions
Copy link
Contributor

github-actions bot commented Mar 4, 2026

You can preview documentation at https://esmci.github.io/cime/branch/fix-containers/html/index.html

- hdf5@1.12.3+cxx+fortran+hl+mpi+shared %gcc@13.3.0
- netcdf-c@4.9.2+mpi %gcc@13.3.0
- netcdf-fortran@4.6.1 %gcc@13.3.0
- parallel-netcdf@1.12.3 %gcc@13.3.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jayeshkrishna can you comment on these versions?

@jasonb5
Copy link
Collaborator Author

jasonb5 commented Mar 11, 2026

@jedwards4b @billsacks I don't think the CESM failure is caused by this PR, probably related to this change. If it is I'll just merge this PR. Thanks!

@billsacks
Copy link
Member

@jasonb5 - Thanks - I agree that the CESM failure is due to a CICE change - we just saw that in our recent testing as well without this CIME change.

@jasonb5 jasonb5 merged commit ffbf6c5 into master Mar 12, 2026
11 checks passed
@jasonb5 jasonb5 deleted the fix-containers branch March 12, 2026 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants