Fix build with Python >= 3.12

[Neverlord]

The distutils package has been removed in Python 3.12. Hence, we need a different approach for finding the site-packages directory.

Using the site module for the job seems to be the simplest choice: https://docs.python.org/3/library/site.html#command-line-interface

[ckreibich]

Interesting — I was wondering why I didn't hit this (I'm on Python 3.12.1) and it turns out I have setuptools installed via a system package, which masks this.

I noticed that for me the two commands yield different paths, though:

$ python -m site --user-site
/home/christian/inst/lib/python3.12/site-packages

$ ipython
20:06:53 1> from distutils.sysconfig import get_python_lib; print(get_python_lib())
/usr/lib/python3.12/site-packages

So if the user runs a vanilla ./configure and expects system paths, an installation in the home is probably not the right thing. Should we use something based on site.getsitepackages() instead? For me site.getsitepackages()[-1] would give the same result, but that might just be my system.

[ckreibich]

I looked at this a bit more and the site.getsitepackages() results are technically part of the search path, but not all of the paths need to exist on the system. So I think we should use site.getsitepackages() and pick the first one that does, perhaps via

$ python -c 'import os.path,site; print([d for d in site.getsitepackages() if os.path.isdir(d)][0])'

For my system that would be /usr/local/lib/python3.12/site-packages, which seems reasonable.

[Neverlord]

I sat down with this again and looked not only on the Python docs, but also on the CMake docs. Should've looked there for a fix first.

Looks like someone already put quite a few thoughts into this: https://cmake.org/cmake/help/latest/module/FindPython.html#result-variables.

Since we're not installing "pure Python", it looks to me like we should install to Python_SITEARCH:

Third-party platform dependent installation directory.

For my python3 via Homebrew (macOS), this points to /opt/homebrew/lib/python3.12/site-packages. This would boil down to this change:

diff --git a/bindings/python/CMakeLists.txt b/bindings/python/CMakeLists.txt
index a3cad51c..a2002758 100644
--- a/bindings/python/CMakeLists.txt
+++ b/bindings/python/CMakeLists.txt
@@ -74,10 +74,7 @@ if ( NOT PY_MOD_INSTALL_DIR )
   elseif (BROKER_PYTHON_HOME)
     file(TO_CMAKE_PATH "${BROKER_PYTHON_HOME}/lib/python" PY_MOD_INSTALL_DIR)
   else ()
-    execute_process(COMMAND ${Python_EXECUTABLE} -m site --user-site
-      OUTPUT_VARIABLE python_site_packages
-      OUTPUT_STRIP_TRAILING_WHITESPACE)
-    file(TO_CMAKE_PATH ${python_site_packages} PY_MOD_INSTALL_DIR)
+    set(PY_MOD_INSTALL_DIR "${Python_SITEARCH}")
   endif ()
 endif ()
 message(STATUS "Python bindings will be built and installed to:")

@ckreibich I've amended the original commit. Let me know if you disagree with the CMake variable approach. 🙂