Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't install modules when version_naming is set #685

Open
muffato opened this issue Jan 14, 2025 · 3 comments
Open

Can't install modules when version_naming is set #685

muffato opened this issue Jan 14, 2025 · 3 comments

Comments

@muffato
Copy link
Contributor

muffato commented Jan 14, 2025

Describe the bug

Sorry I couldn't chime in during #679, I got sick. I'm now trying to test the feature but can't make it work.

The problem I'm facing is that <module_base>/<container_path>/<version> is meant to be a directory under which shpc creates the bin directory with all the wrapper scripts.
With version_naming set to true, shpc now also assumes with path to be a file to write the content of the TCL module, hence a clash. cc @Amjadhpc

To Reproduce

With the default settings (version_naming set to false):

$ rm -r modules
$ shpc config get version_naming
False
$ shpc install python:3.9.2-alpine
Module python:3.9.2-alpine was created.

With the option turned on

$ shpc config set version_naming True
Updated version_naming to be True
$ shpc config get version_naming
True
$ rm -r modules
$ shpc install python:3.9.2-alpine
Traceback (most recent call last):
  File "/software/treeoflife/conda/users/envs/team328/mm49/shpc/bin/shpc", line 8, in <module>
    sys.exit(run_shpc())
             ^^^^^^^^^^
  File "/nfs/users/nfs_m/mm49/workspace/tol-it/central_software/singularity-hpc/shpc/client/__init__.py", line 556, in run_shpc
    main(args=args, parser=parser, extra=extra, subparser=helper)
  File "/nfs/users/nfs_m/mm49/workspace/tol-it/central_software/singularity-hpc/shpc/client/install.py", line 27, in main
    cli.install(
  File "/nfs/users/nfs_m/mm49/workspace/tol-it/central_software/singularity-hpc/shpc/main/modules/base.py", line 485, in install
    self.container.install(module_path, template, module, kwargs.get("features"))
  File "/nfs/users/nfs_m/mm49/workspace/tol-it/central_software/singularity-hpc/shpc/main/container/singularity.py", line 233, in install
    utils.write_file(module_path, out)
  File "/nfs/users/nfs_m/mm49/workspace/tol-it/central_software/singularity-hpc/shpc/utils/fileio.py", line 176, in write_file
    with open(filename, mode) as filey:
         ^^^^^^^^^^^^^^^^^^^^
IsADirectoryError: [Errno 21] Is a directory: '/nfs/users/nfs_m/mm49/workspace/tol-it/central_software/singularity-hpc/modules/python/3.9.2-alpine'

Expected behavior

I believe shpc install python:3.9.2-alpine should succeed out of the box, especially as I didn't see any indication the new option could only be used in certain conditions.

Also, I wonder how it works in the test case: https://github.com/singularityhub/singularity-hpc/pull/679/files#diff-9ef112d1d82a448d99f2cf48d7718defdde5618f6657ab2a99e6f6f5d7941729 . Am I doing anything different ?

Version of Singularity and Singularity Registry HPC Client

$ git show
commit 0848c8747042240f36f99cfa886a13724443578f (HEAD -> main, tag: 0.1.31, origin/main, origin/HEAD)
Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Date:   Sat Jan 11 13:59:13 2025 -0700

    Automated deployment to update contributors 2025-01-11 (#684)
    
    Co-authored-by: github-actions <[email protected]>

$ git diff
diff --git a/shpc/settings.yml b/shpc/settings.yml
index ed80a5715..4f68cbf56 100644
--- a/shpc/settings.yml
+++ b/shpc/settings.yml
@@ -5,7 +5,7 @@
 # set a default module system
 # Currently lmod and tcl are supported. To request an additional system,
 # please open an issue https://github.com/singularityhub/singularity-hpc
-module_sys: lmod
+module_sys: tcl
 
 # config editor
 config_editor: vim
@vsoch
Copy link
Member

vsoch commented Jan 14, 2025

I'll revert until this is addressed: #686

@Amjadhpc
Copy link
Contributor

@muffato , correct me if i am wrong, you are just deleting the modules file with
rm -r modules ?
Can you please use shpc uninstall instead of rm ?

shpc uninstall python:3.9.2-alpine
python:3.9.2-alpine? (yes/no)? yes
$container_base/python:3.9.2-alpine and all subdirectories have been removed.
$module_base/python:3.9.2-alpine and all subdirectories have been removed.
$wrapper_base/python:3.9.2-alpine and all subdirectories have been removed.
singularity-hpc]# shpc config set version_naming True
Updated version_naming to be True
singularity-hpc]# shpc install python:3.9.2-alpine
singularity-hpc]#Module python:3.9.2-alpine was created.

@muffato
Copy link
Contributor Author

muffato commented Jan 14, 2025

Same error when doing shpc uninstall because all the files involved were in modules/ and shpc uninstall merely does a rm -r under the hood. But that made me understand the difference and the problem better !

In default settings, module_base == wrapper_base == $root_dir/modules. That's essentially what's causing my bug.
If I set wrapper_base to a different location, then the wrapper script directory doesn't clash anymore with the module directory. In other words, version_naming requires wrapper_base to be set to a different path from module_base.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants