Known issues #283

jaimergp · 2023-09-19T17:13:09Z

The following items are known and either awaiting a bugfix elsewhere or, if no fix available, a workaround is suggested.

`libarchive`: "library not loaded" or "cannot open shared object file"

Observed errors

On Linux:

ImportError: libarchive.so.13: cannot open shared object file: No such file or directory

On macOS:

ImportError: dlopen(~/miniconda3/lib/python3.11/site-packages/libmambapy/bindings.cpython-311-darwin.so, 2): Library not loaded: @rpath/libarchive.13.dylib Referenced from: ~/miniconda3/lib/libmamba.2.0.0.dylib Reason: image not found

Explanation

This is a known incompatibility between the libarchive packages published on defaults and conda-forge. The root cause is a problem in the CMake build configuration upstream (libarchive/libarchive#1857).

Solution

This is a packaging problem that will be eventually solved, but that won't happen until libarchive 3.8 is published (or later). For now, the recommendation is to make sure that libarchive and libmamba come from the same channel.

References

Cannot use `defaults::package` specs

Observed errors

RuntimeError: The package "pkgs/main::zlib" is not available for the specified platform

InvalidMatchSpec: Invalid spec 'pkgs/main::python': This is a bug in libmamba 1.5.1 when using 'defaults::<spec>' or 'pkgs/main::<spec>'. Consider using '-c defaults' instead.

Explanation

This is a known issue reported in libmamba: mamba-org/mamba#2431

Solution

Fixed in libmamba 1.5.3. Please update to the latest version.

Cannot Ctrl-C out of `conda`

Observed errors

Users are unable to use Ctrl+C to exit conda commands when libmamba is in use.

Explanation

The libmamba C++ library has some custom signal handlers that take over the signal controllers in Python's thread. Instantiating the libmambapy.Context object is enough to install those handlers in the process. This prevents Python from handling SIGINT correctly.

Solution

Fixed in libmamba 1.5.3. A small change is needed in conda-libmamba-solver, and will be made available soon.

In the meantime, users can use a different signal to kill the process. For example, Ctrl+Z to pause it and then issue SIGKILL / 9 via top or htop.

The text was updated successfully, but these errors were encountered:

spyoungtech · 2023-10-10T00:39:18Z

For now, the recommendation is to make sure that libarchive and libmamba come from the same channel.

To expand on this, libarchive seems to come with conda by default (when installed via miniconda). So, even if you're working with a clean conda install and using conda-forge as your only configured channel, you'll get this error until you reinstall libarchive.

I suspect miniforge users won't have this issue.

See: conda/conda-libmamba-solver#283 Signed-off-by: Patrick Avery <[email protected]>

gosunny2050 · 2024-01-18T05:29:45Z

how to solve it

noahlibby17 · 2024-02-16T19:19:08Z

Any update on how to solve this issue?

spyoungtech · 2024-02-17T01:05:22Z

@gosunny2050 @noahlibby17

As noted above:

For now, the recommendation is to make sure that libarchive and libmamba come from the same channel.

I'll try to elaborate on this to the best of my ability...

If you run conda list, it will show your installed packages, the version/build, and the channel source. So, if you find the entries for libarchive and libmamba in the output of conda list: BOTH of them should show the same channel (for example, conda-forge). If you see they are installed from different channels, you must uninstall/reinstall one or both of the packages so that they are installed from the same channel.

In my case, I was running into this issue even with a fresh conda base environment (via miniconda) because, by default, libarchive is present and installed from the defaults channel. So, when I installed libmamba from conda-forge, this issue occurred due to the mismatching channels.

To fix this, when I create my environment, I make sure to remove the defaults channel then run conda update --all which will cause all packages to be from conda-forge (assuming you have no other channels configured):

# make sure conda-forge is the only channel
conda config --remove channels defaults
conda config --add channels conda-forge

# update all packages to make sure they're from conda-forge, not defaults
conda update --all

After updating with the new channel configuration, when you run conda list again, you should see they are using the same channel:

$ conda list
# packages in environment at /opt/miniconda:
#
# Name                    Version                   Build  Channel
[...]
libarchive                3.7.2                h2aa1ff5_1    conda-forge
[...]
libmamba                  1.5.6                had39da4_0    conda-forge

gregorywaynepower · 2024-04-22T03:30:30Z

I was able to resolve this issue by running conda install libarchive within my base environment and giving the defaults channel priority in my .condarc file. Where it replaced the condaforge version with the pkgs/main version.

newptcai · 2024-09-02T09:37:38Z

I got this problem after updating conda 24.7.1. @spyoungtech gave a solution which fixed my problem.

yarikoptic · 2024-10-31T20:22:36Z

FWIW I get

❯ /tmp/mini1/bin/conda config --show channels
channels:
  - nodefaults
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/r
❯ /tmp/mini1/bin/conda config --add channels conda-forge
❯ /tmp/mini1/bin/conda config --show channels
channels:
  - conda-forge
  - nodefaults
  - https://repo.anaconda.com/pkgs/main
  - https://repo.anaconda.com/pkgs/r
❯ /tmp/mini1/bin/conda config --remove channels defaults

CondaKeyError: 'channels': value 'defaults' not present in config

with current miniconda, and even more fun

❯ /tmp/mini1/bin/conda config --remove channels https://repo.anaconda.com/pkgs/main

CondaKeyError: 'channels': value 'https://repo.anaconda.com/pkgs/main' not present in config

so remains unclear how to tame miniconda to behave.

jaimergp · 2024-11-04T08:19:57Z

If you use conda config --show-sources you can see which source is providing that config and maybe edit that manually, as a workaround.

first issue here: conda/conda-libmamba-solver#283 -> adding libarchive and libmamba from *same* channel (only saw the problem with Python 3.12)

peterbraden · 2024-11-26T17:19:33Z

In order to resolve I needed to use:

conda install --solver=classic conda-forge::conda-libmamba-solver conda-forge::libmamba conda-forge::libmambapy conda-forge::libarchive

For some reason, there was no other way to purge the version of libarchive from the default channel.

first issue here: conda/conda-libmamba-solver#283 -> adding libarchive and libmamba from *same* channel (only saw the problem with Python 3.12)

…#4126) Summary: Explanation of changes: - - Python upgraded to 3.12 because otherwise we fail with the below error, and I thought we might want to keep telemetry? Let me know otherwise ``` LibMambaUnsatisfiableError: Encountered problems while solving: - package conda-anaconda-telemetry-0.1.1-py312h06a4308_0 requires python >=3.12,<3.13.0a0, but none of the providers can be installed Could not solve for environment specs The following packages are incompatible ├─ conda-anaconda-telemetry is installable and it requires │ └─ python >=3.12,<3.13.0a0 , which can be installed; └─ python 3.11** is not installable because it conflicts with any installable versions previously reported. ``` - Solver changes: conda/conda-libmamba-solver#283 (comment) We need to manually install libmamba from conda-forge instead of defaults, then use it. If it and libarchive come from different channels, build fails with: ``` Error while loading conda entry point: conda-libmamba-solver (libarchive.so.20: cannot open shared object file: No such file or directory) CondaValueError: You have chosen a non-default solver backend (libmamba) but it was not recognized. Choose one of: classic ``` - Why not use the classic solver, why do we need the above change to keep using libmamba? classic is not able to resolve dependencies for us. Others suggest just to use libmamba: conda/conda-build#5516 (comment). Classic fails with ``` DependencyNeedsBuildingError: Unsatisfiable dependencies for platform linux-64: {'__glibc', '__archspec', '__unix', '__conda', '__linux'} ``` - cpuinfo.py copied in because Python 3.12 removes distutils, and setuptools does not have feature parity. People have complained that they can't use FAISS with Python > 3.11 before and this is the dependency that I see. facebookresearch#3936 - cuda-toolkit upgraded to 12.5.1 because of this error on 12.4.1: ``` nvlink fatal : Input file '/home/runner/miniconda3/bin/../lib/libcudadevrt.a:cuda_device_runtime.o' newer than toolkit (126 vs 124) ``` - Differential Revision: D68043874

…#4126) Summary: Good resource on overriding channels to make sure we aren't using `defaults`:https://stackoverflow.com/questions/67695893/how-do-i-completely-purge-and-disable-the-default-channel-in-anaconda-and-switch Explanation of changes: - - Python upgraded to 3.12 because otherwise we fail with the below error, and I thought we might want to keep telemetry? Let me know otherwise ``` LibMambaUnsatisfiableError: Encountered problems while solving: - package conda-anaconda-telemetry-0.1.1-py312h06a4308_0 requires python >=3.12,<3.13.0a0, but none of the providers can be installed Could not solve for environment specs The following packages are incompatible ├─ conda-anaconda-telemetry is installable and it requires │ └─ python >=3.12,<3.13.0a0 , which can be installed; └─ python 3.11** is not installable because it conflicts with any installable versions previously reported. ``` - Solver changes: conda/conda-libmamba-solver#283 (comment) We need to manually install libmamba from conda-forge instead of defaults, then use it. If it and libarchive come from different channels, build fails with: ``` Error while loading conda entry point: conda-libmamba-solver (libarchive.so.20: cannot open shared object file: No such file or directory) CondaValueError: You have chosen a non-default solver backend (libmamba) but it was not recognized. Choose one of: classic ``` or the very cryptic: (it refers to glibc version...) ``` Your installed version is: 2.35 ``` - Why not use the classic solver, why do we need the above change to keep using libmamba? classic is not able to resolve dependencies for us. Others suggest just to use libmamba: conda/conda-build#5516 (comment). Classic fails with ``` DependencyNeedsBuildingError: Unsatisfiable dependencies for platform linux-64: {'__glibc', '__archspec', '__unix', '__conda', '__linux'} ``` - cpuinfo.py copied in because Python 3.12 removes distutils, and setuptools does not have feature parity. People have complained that they can't use FAISS with Python > 3.11 before and this is the dependency that I see. facebookresearch#3936 - cuda-toolkit upgraded to 12.5.1 because of this error on 12.4.1: ``` nvlink fatal : Input file '/home/runner/miniconda3/bin/../lib/libcudadevrt.a:cuda_device_runtime.o' newer than toolkit (126 vs 124) ``` - Differential Revision: D68043874

jaimergp pinned this issue Sep 19, 2023

conda-bot added this to 🧭 Planning Sep 19, 2023

github-project-automation bot moved this to 🆕 New in 🧭 Planning Sep 19, 2023

This was referenced Sep 19, 2023

Could not load conda plugin conda-libmamba-solver #244

Closed

Unable to install package due to ImportError: libarchive.so.13 conda/conda#13148

Closed

tobiasdiez mentioned this issue Oct 2, 2023

Add needed pip packages to src/environment.yml, minimize conda environments sagemath/sage#35593

Open

5 tasks

pxLi mentioned this issue Oct 6, 2023

[BUG] failed to build integration images due to mamba solver incompatible issue NVIDIA/spark-rapids#9393

Closed

h-vetinari mentioned this issue Oct 9, 2023

error while loading shared libraries: libarchive.so.13: cannot open shared object file: No such file or directory conda-forge/cmake-feedstock#14

Closed

psavery added a commit to HEXRD/hexrd that referenced this issue Oct 18, 2023

Try to install libarchive for libmamba on MacOSX

1e6021e

See: conda/conda-libmamba-solver#283 Signed-off-by: Patrick Avery <[email protected]>

jezdez added source::contributor created by a frequent contributor severity::4 low; functionality is inconvenient documentation::explanation related to higher-level clarification (understanding-oriented) labels Nov 2, 2023

loganbvh mentioned this issue Jan 26, 2024

errors during tdgl testing loganbvh/py-tdgl#69

Closed

jaimergp mentioned this issue Feb 1, 2024

Error while loading conda entry point: conda-libmamba-solver (libarchive.so.10: cannot open shared object file: No such file or directory) #424

Closed

2 tasks

jylee3247 mentioned this issue Feb 21, 2024

conda 24.1.1 libmamba solver not recognized ContinuumIO/anaconda-issues#13353

Closed

8 tasks

marthacryan mentioned this issue Aug 26, 2024

Fix CI failure plotly/plotly.py#4728

Merged

This was referenced Sep 17, 2024

Miniconda installer started to succumb to conflict with mamba when working with conda-forge conda/constructor#862

Closed

git-annex -m conda installation fails in CI datalad/datalad-installer#206

Open

yarikoptic mentioned this issue Oct 31, 2024

config: refuses to --remove channel (URL) listed by --show conda/conda#14360

Open

2 tasks

oyamad mentioned this issue Nov 10, 2024

CI: Install libmamba from conda-forge QuantEcon/QuantEcon.py#743

Merged

emiliom mentioned this issue Nov 14, 2024

R image: update python, pangeo-notebook and rstudio-server oceanhackweek/jupyter-image#97

Merged

mistafunk added a commit to Esri/pyprt that referenced this issue Nov 24, 2024

conda: workarounds for issues on linux

98106e0

first issue here: conda/conda-libmamba-solver#283 -> adding libarchive and libmamba from *same* channel (only saw the problem with Python 3.12)

mistafunk added a commit to Esri/pyprt that referenced this issue Dec 2, 2024

build env: workarounds for conda issues on linux

aea456f

first issue here: conda/conda-libmamba-solver#283 -> adding libarchive and libmamba from *same* channel (only saw the problem with Python 3.12)

mistafunk added a commit to Esri/pyprt that referenced this issue Dec 3, 2024

build env: workarounds for conda issues on linux

4a5fb9b

first issue here: conda/conda-libmamba-solver#283 -> adding libarchive and libmamba from *same* channel (only saw the problem with Python 3.12)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Known issues #283

Known issues #283

jaimergp commented Sep 19, 2023 •

edited

Loading

spyoungtech commented Oct 10, 2023

gosunny2050 commented Jan 18, 2024

noahlibby17 commented Feb 16, 2024

spyoungtech commented Feb 17, 2024 •

edited

Loading

gregorywaynepower commented Apr 22, 2024 •

edited

Loading

newptcai commented Sep 2, 2024

yarikoptic commented Oct 31, 2024

jaimergp commented Nov 4, 2024

peterbraden commented Nov 26, 2024

Known issues #283

Known issues #283

Comments

jaimergp commented Sep 19, 2023 • edited Loading

libarchive: "library not loaded" or "cannot open shared object file"

Observed errors

Explanation

Solution

References

Cannot use defaults::package specs

Observed errors

Explanation

Solution

Cannot Ctrl-C out of conda

Observed errors

Explanation

Solution

spyoungtech commented Oct 10, 2023

gosunny2050 commented Jan 18, 2024

noahlibby17 commented Feb 16, 2024

spyoungtech commented Feb 17, 2024 • edited Loading

gregorywaynepower commented Apr 22, 2024 • edited Loading

newptcai commented Sep 2, 2024

yarikoptic commented Oct 31, 2024

jaimergp commented Nov 4, 2024

peterbraden commented Nov 26, 2024

jaimergp commented Sep 19, 2023 •

edited

Loading

`libarchive`: "library not loaded" or "cannot open shared object file"

Cannot use `defaults::package` specs

Cannot Ctrl-C out of `conda`

spyoungtech commented Feb 17, 2024 •

edited

Loading

gregorywaynepower commented Apr 22, 2024 •

edited

Loading