Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Python] pyfury0.9.0 installed from the source folder does not work properly #2035

Open
2 tasks done
LouisLou2 opened this issue Feb 2, 2025 · 3 comments
Open
2 tasks done
Labels
bug Something isn't working

Comments

@LouisLou2
Copy link

LouisLou2 commented Feb 2, 2025

Search before asking

  • I had searched in the issues and found no similar issues.

Version

OS: Ubuntu 24.04.1 LTS

gcc

Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-linux-gnu/13/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Thread model: posix
gcc version 13.3.0 (Ubuntu 13.3.0-6ubuntu2~24.04) 

Anaconda Env:
name: pyfury_env2
version: Python 3.9.21
pip list:

(pyfury_env2) leo@leo-dev:~$ pip list
Package         Version     Editable project location
--------------- ----------- ------------------------------------
Bottleneck      1.4.2
cloudpickle     3.1.1
Cython          3.0.11
exceptiongroup  1.2.2
iniconfig       2.0.0
mkl_fft         1.3.11
mkl_random      1.2.8
mkl-service     2.4.0
mmh3            5.1.0
numexpr         2.10.1
numpy           1.26.4
packaging       24.2
pandas          2.2.3
pip             25.0
pluggy          1.5.0
pyarrow         14.0.0
pyfury          0.9.0       /home/leo/projects/fury-0.9.0/python
pytest          8.3.4
python-dateutil 2.9.0.post0
pytz            2024.1
setuptools      75.8.0
six             1.16.0
tomli           2.2.1
tzdata          2023.3
wheel           0.45.1

structure of /home/leo/projects/fury-0.9.0/python

(pyfury_env2) leo@leo-dev:~$ tree /home/leo/projects/fury-0.9.0/python
/home/leo/projects/fury-0.9.0/python
├── pyfury
│   ├── buffer.py
│   ├── codegen.py
│   ├── error.py
│   ├── format
│   │   ├── encoder.pxi
│   │   ├── encoder.py
│   │   ├── _format.pyx
│   │   ├── _format.so
│   │   ├── infer.py
│   │   ├── __init__.py
│   │   ├── row.pxi
│   │   ├── serializer.py
│   │   ├── tests
│   │   │   ├── __init__.py
│   │   │   ├── test_encoder.py
│   │   │   ├── test_infer.py
│   │   │   └── test_vectorized.py
│   │   └── vectorized.pxi
│   ├── _fury.py
│   ├── includes
│   │   ├── __init__.pxd
│   │   ├── libabsl.pxd
│   │   ├── libformat.pxd
│   │   └── libutil.pxd
│   ├── __init__.pxd
│   ├── __init__.py
│   ├── lib
│   │   ├── collection.py
│   │   ├── __init__.pxd
│   │   ├── __init__.py
│   │   ├── mmh3
│   │   │   ├── __init__.pxd
│   │   │   ├── __init__.py
│   │   │   ├── mmh3.pxd
│   │   │   ├── mmh3.pyx
│   │   │   ├── mmh3.so
│   │   │   └── __pycache__
│   │   │       ├── __init__.cpython-311.pyc
│   │   │       ├── __init__.cpython-38.pyc
│   │   │       └── __init__.cpython-39.pyc
│   │   ├── __pycache__
│   │   │   ├── __init__.cpython-311.pyc
│   │   │   ├── __init__.cpython-38.pyc
│   │   │   └── __init__.cpython-39.pyc
│   │   └── tests
│   │       ├── __init__.py
│   │       ├── test_collection.py
│   │       └── test_mmh3.py
│   ├── meta
│   │   ├── __init__.py
│   │   └── metastring.py
│   ├── __pycache__
│   │   ├── __init__.cpython-311.pyc
│   │   ├── __init__.cpython-38.pyc
│   │   └── __init__.cpython-39.pyc
│   ├── resolver.py
│   ├── _serialization.pyx
│   ├── _serialization.so
│   ├── _serializer.py
│   ├── serializer.py
│   ├── _struct.py
│   ├── tests
│   │   ├── benchmark.py
│   │   ├── core.py
│   │   ├── __init__.py
│   │   ├── profiling.py
│   │   ├── record.py
│   │   ├── test_buffer.py
│   │   ├── test_codegen.py
│   │   ├── test_cross_language.py
│   │   ├── test_metastring.py
│   │   ├── test_serializer.py
│   │   ├── test_struct.py
│   │   └── test_type.py
│   ├── type.py
│   ├── _util.pxd
│   ├── util.py
│   ├── _util.pyx
│   └── _util.so
├── pyfury.egg-info
│   ├── dependency_links.txt
│   ├── not-zip-safe
│   ├── PKG-INFO
│   ├── requires.txt
│   ├── SOURCES.txt
│   └── top_level.txt
├── README.md
└── setup.py

14 directories, 76 files

Component(s)

Python

Minimal reproduce step

cd /path/to/pyfury/python
pip install pyarrow==14.0.0 Cython wheel numpy==1.26.4 pytest
pip install -v -e .

No errors were encountered in the above steps

Trying to run the testing section gives a compilation error
pytest -v -s .

What did you expect to see?

I expect to see test cases that should execute correctly, or at least need to be runnable

What did you see instead?

(pyfury_env2) leo@leo-dev:~/projects/fury-0.9.0/python$ pytest -v -s .
=========================================================================== test session starts ===========================================================================
platform linux -- Python 3.9.21, pytest-8.3.4, pluggy-1.5.0 -- /home/leo/Toolset/lang/anaconda/envs/pyfury_env2/bin/python
cachedir: .pytest_cache
rootdir: /home/leo/projects/fury-0.9.0/python
collected 0 items / 12 errors                                                                                                                                             

================================================================================= ERRORS ==================================================================================
__________________________________________________________ ERROR collecting pyfury/format/tests/test_encoder.py ___________________________________________________________
ImportError while importing test module '/home/leo/projects/fury-0.9.0/python/pyfury/format/tests/test_encoder.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../Toolset/lang/anaconda/envs/pyfury_env2/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pyfury/__init__.py:18: in <module>
    from pyfury import lib  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/__init__.py:18: in <module>
    from pyfury.lib import mmh3  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/mmh3/__init__.py:18: in <module>
    from .mmh3 import *  # noqa: F401,F403 # pylint: disable=unused-import
E   ImportError: /home/leo/projects/fury-0.9.0/python/pyfury/lib/mmh3/mmh3.so: undefined symbol: PyGC_Disable
___________________________________________________________ ERROR collecting pyfury/format/tests/test_infer.py ____________________________________________________________
ImportError while importing test module '/home/leo/projects/fury-0.9.0/python/pyfury/format/tests/test_infer.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../Toolset/lang/anaconda/envs/pyfury_env2/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pyfury/__init__.py:18: in <module>
    from pyfury import lib  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/__init__.py:18: in <module>
    from pyfury.lib import mmh3  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/mmh3/__init__.py:18: in <module>
    from .mmh3 import *  # noqa: F401,F403 # pylint: disable=unused-import
E   ImportError: /home/leo/projects/fury-0.9.0/python/pyfury/lib/mmh3/mmh3.so: undefined symbol: PyGC_Disable
_________________________________________________________ ERROR collecting pyfury/format/tests/test_vectorized.py _________________________________________________________
ImportError while importing test module '/home/leo/projects/fury-0.9.0/python/pyfury/format/tests/test_vectorized.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../Toolset/lang/anaconda/envs/pyfury_env2/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pyfury/__init__.py:18: in <module>
    from pyfury import lib  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/__init__.py:18: in <module>
    from pyfury.lib import mmh3  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/mmh3/__init__.py:18: in <module>
    from .mmh3 import *  # noqa: F401,F403 # pylint: disable=unused-import
E   ImportError: /home/leo/projects/fury-0.9.0/python/pyfury/lib/mmh3/mmh3.so: undefined symbol: PyGC_Disable
__________________________________________________________ ERROR collecting pyfury/lib/tests/test_collection.py ___________________________________________________________
ImportError while importing test module '/home/leo/projects/fury-0.9.0/python/pyfury/lib/tests/test_collection.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../Toolset/lang/anaconda/envs/pyfury_env2/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pyfury/__init__.py:18: in <module>
    from pyfury import lib  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/__init__.py:18: in <module>
    from pyfury.lib import mmh3  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/mmh3/__init__.py:18: in <module>
    from .mmh3 import *  # noqa: F401,F403 # pylint: disable=unused-import
E   ImportError: /home/leo/projects/fury-0.9.0/python/pyfury/lib/mmh3/mmh3.so: undefined symbol: PyGC_Disable
_____________________________________________________________ ERROR collecting pyfury/lib/tests/test_mmh3.py ______________________________________________________________
ImportError while importing test module '/home/leo/projects/fury-0.9.0/python/pyfury/lib/tests/test_mmh3.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../Toolset/lang/anaconda/envs/pyfury_env2/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pyfury/__init__.py:18: in <module>
    from pyfury import lib  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/__init__.py:18: in <module>
    from pyfury.lib import mmh3  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/mmh3/__init__.py:18: in <module>
    from .mmh3 import *  # noqa: F401,F403 # pylint: disable=unused-import
E   ImportError: /home/leo/projects/fury-0.9.0/python/pyfury/lib/mmh3/mmh3.so: undefined symbol: PyGC_Disable
______________________________________________________________ ERROR collecting pyfury/tests/test_buffer.py _______________________________________________________________
ImportError while importing test module '/home/leo/projects/fury-0.9.0/python/pyfury/tests/test_buffer.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../Toolset/lang/anaconda/envs/pyfury_env2/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pyfury/__init__.py:18: in <module>
    from pyfury import lib  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/__init__.py:18: in <module>
    from pyfury.lib import mmh3  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/mmh3/__init__.py:18: in <module>
    from .mmh3 import *  # noqa: F401,F403 # pylint: disable=unused-import
E   ImportError: /home/leo/projects/fury-0.9.0/python/pyfury/lib/mmh3/mmh3.so: undefined symbol: PyGC_Disable
______________________________________________________________ ERROR collecting pyfury/tests/test_codegen.py ______________________________________________________________
ImportError while importing test module '/home/leo/projects/fury-0.9.0/python/pyfury/tests/test_codegen.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../Toolset/lang/anaconda/envs/pyfury_env2/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pyfury/__init__.py:18: in <module>
    from pyfury import lib  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/__init__.py:18: in <module>
    from pyfury.lib import mmh3  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/mmh3/__init__.py:18: in <module>
    from .mmh3 import *  # noqa: F401,F403 # pylint: disable=unused-import
E   ImportError: /home/leo/projects/fury-0.9.0/python/pyfury/lib/mmh3/mmh3.so: undefined symbol: PyGC_Disable
__________________________________________________________ ERROR collecting pyfury/tests/test_cross_language.py ___________________________________________________________
ImportError while importing test module '/home/leo/projects/fury-0.9.0/python/pyfury/tests/test_cross_language.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../Toolset/lang/anaconda/envs/pyfury_env2/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pyfury/__init__.py:18: in <module>
    from pyfury import lib  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/__init__.py:18: in <module>
    from pyfury.lib import mmh3  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/mmh3/__init__.py:18: in <module>
    from .mmh3 import *  # noqa: F401,F403 # pylint: disable=unused-import
E   ImportError: /home/leo/projects/fury-0.9.0/python/pyfury/lib/mmh3/mmh3.so: undefined symbol: PyGC_Disable
____________________________________________________________ ERROR collecting pyfury/tests/test_metastring.py _____________________________________________________________
ImportError while importing test module '/home/leo/projects/fury-0.9.0/python/pyfury/tests/test_metastring.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../Toolset/lang/anaconda/envs/pyfury_env2/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pyfury/__init__.py:18: in <module>
    from pyfury import lib  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/__init__.py:18: in <module>
    from pyfury.lib import mmh3  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/mmh3/__init__.py:18: in <module>
    from .mmh3 import *  # noqa: F401,F403 # pylint: disable=unused-import
E   ImportError: /home/leo/projects/fury-0.9.0/python/pyfury/lib/mmh3/mmh3.so: undefined symbol: PyGC_Disable
____________________________________________________________ ERROR collecting pyfury/tests/test_serializer.py _____________________________________________________________
ImportError while importing test module '/home/leo/projects/fury-0.9.0/python/pyfury/tests/test_serializer.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../Toolset/lang/anaconda/envs/pyfury_env2/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pyfury/__init__.py:18: in <module>
    from pyfury import lib  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/__init__.py:18: in <module>
    from pyfury.lib import mmh3  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/mmh3/__init__.py:18: in <module>
    from .mmh3 import *  # noqa: F401,F403 # pylint: disable=unused-import
E   ImportError: /home/leo/projects/fury-0.9.0/python/pyfury/lib/mmh3/mmh3.so: undefined symbol: PyGC_Disable
______________________________________________________________ ERROR collecting pyfury/tests/test_struct.py _______________________________________________________________
ImportError while importing test module '/home/leo/projects/fury-0.9.0/python/pyfury/tests/test_struct.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../Toolset/lang/anaconda/envs/pyfury_env2/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pyfury/__init__.py:18: in <module>
    from pyfury import lib  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/__init__.py:18: in <module>
    from pyfury.lib import mmh3  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/mmh3/__init__.py:18: in <module>
    from .mmh3 import *  # noqa: F401,F403 # pylint: disable=unused-import
E   ImportError: /home/leo/projects/fury-0.9.0/python/pyfury/lib/mmh3/mmh3.so: undefined symbol: PyGC_Disable
_______________________________________________________________ ERROR collecting pyfury/tests/test_type.py ________________________________________________________________
ImportError while importing test module '/home/leo/projects/fury-0.9.0/python/pyfury/tests/test_type.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../../Toolset/lang/anaconda/envs/pyfury_env2/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
pyfury/__init__.py:18: in <module>
    from pyfury import lib  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/__init__.py:18: in <module>
    from pyfury.lib import mmh3  # noqa: F401 # pylint: disable=unused-import
pyfury/lib/mmh3/__init__.py:18: in <module>
    from .mmh3 import *  # noqa: F401,F403 # pylint: disable=unused-import
E   ImportError: /home/leo/projects/fury-0.9.0/python/pyfury/lib/mmh3/mmh3.so: undefined symbol: PyGC_Disable
========================================================================= short test summary info =========================================================================
ERROR pyfury/format/tests/test_encoder.py
ERROR pyfury/format/tests/test_infer.py
ERROR pyfury/format/tests/test_vectorized.py
ERROR pyfury/lib/tests/test_collection.py
ERROR pyfury/lib/tests/test_mmh3.py
ERROR pyfury/tests/test_buffer.py
ERROR pyfury/tests/test_codegen.py
ERROR pyfury/tests/test_cross_language.py
ERROR pyfury/tests/test_metastring.py
ERROR pyfury/tests/test_serializer.py
ERROR pyfury/tests/test_struct.py
ERROR pyfury/tests/test_type.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 12 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================================================================== 12 errors in 0.08s ============================================================================
(pyfury_env2) leo@leo-dev:~/projects/fury-0.9.0/python$ 

Anything Else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!
@LouisLou2 LouisLou2 added the bug Something isn't working label Feb 2, 2025
@chaokunyang
Copy link
Collaborator

Hi @LouisLou2, are you using different Python for building and running putest? Could you check whether your pytest is installed in another pyenv?

@LouisLou2
Copy link
Author

Hi @LouisLou2, are you using different Python for building and running putest? Could you check whether your pytest is installed in another pyenv?

Yes, I'm sure I'm using the pyfury_env2 environment for this build. It is also true that testing is done in this environment.

(pyfury_env2) leo@leo-dev:~/projects/fury-0.9.0/python$ which pytest
/home/leo/Toolset/lang/anaconda/envs/pyfury_env2/bin/pytest
(pyfury_env2) leo@leo-dev:~/projects/fury-0.9.0/python$ which python
/home/leo/Toolset/lang/anaconda/envs/pyfury_env2/bin/python
(pyfury_env2) leo@leo-dev:~/projects/fury-0.9.0/python$ pytest -v -s .
============================= test session starts ==============================
platform linux -- Python 3.9.21, pytest-8.3.4, pluggy-1.5.0 -- /home/leo/Toolset/lang/anaconda/envs/pyfury_env2/bin/python
cachedir: .pytest_cache
rootdir: /home/leo/projects/fury-0.9.0/python
collected 0 items / 12 errors                                                  

==================================== ERRORS ====================================
_____________ ERROR collecting pyfury/format/tests/test_encoder.py _____________
...

@chaokunyang
Copy link
Collaborator

Looks strange, this error message we're encountering, undefined symbol: PyGC_Disable, typically indicates a compatibility issue between a Python extension module nd the version of Python being used. Most of times this can happen if the module was compiled against a different Python version than the one we are running

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants