_cext.pyd depends on MSVCP140.dll, which is not distributed

[LordAro]

Version: 1.4.7

_cext.pyd still depends MSVCP140.dll, even though this was supposed to have been fixed by #179 (?)

$ python3.12 -m pip download kiwisolver
Collecting kiwisolver
  Using cached kiwisolver-1.4.7-cp312-cp312-win_amd64.whl.metadata (6.4 kB)
Using cached kiwisolver-1.4.7-cp312-cp312-win_amd64.whl (55 kB)
Saved c:\foo\kiwisolver-1.4.7-cp312-cp312-win_amd64.whl
Successfully downloaded kiwisolver

$ file kiwisolver-1.4.7-cp312-cp312-win_amd64.whl
kiwisolver-1.4.7-cp312-cp312-win_amd64.whl: Zip archive data, at least v2.0 to extract, compression method=deflate

$ unzip kiwisolver-1.4.7-cp312-cp312-win_amd64.whl
Archive:  kiwisolver-1.4.7-cp312-cp312-win_amd64.whl
  inflating: kiwisolver/__init__.py
  inflating: kiwisolver/_cext.cp312-win_amd64.pyd
  inflating: kiwisolver/_cext.pyi
  inflating: kiwisolver/exceptions.py
  inflating: kiwisolver/py.typed
  inflating: kiwisolver-1.4.7.dist-info/LICENSE
  inflating: kiwisolver-1.4.7.dist-info/METADATA
  inflating: kiwisolver-1.4.7.dist-info/WHEEL
  inflating: kiwisolver-1.4.7.dist-info/top_level.txt
  inflating: kiwisolver-1.4.7.dist-info/RECORD

$ objdump -p kiwisolver/_cext.cp312-win_amd64.pyd | grep DLL
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: MSVCP140.dll
        DLL Name: python312.dll
        DLL Name: KERNEL32.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll

Found by installing Python in Windows Sandbox and trying to import matplotlib. Matplotlib (3.9.2) does not depend on these external dependencies

Matplotlib deps

$ find matplotlib -name '*.pyd' -print -exec sh -c 'objdump -p {} | grep DLL' \;
matplotlib/backends/_backend_agg.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: python312.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: KERNEL32.dll
matplotlib/backends/_tkagg.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: COMCTL32.dll
        DLL Name: PSAPI.DLL
        DLL Name: python312.dll
        DLL Name: KERNEL32.dll
        DLL Name: VCRUNTIME140_1.dll
matplotlib/ft2font.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-locale-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: python312.dll
        DLL Name: api-ms-win-crt-utility-l1-1-0.dll
        DLL Name: api-ms-win-crt-convert-l1-1-0.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: KERNEL32.dll
matplotlib/_c_internal_utils.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: ole32.dll
        DLL Name: SHELL32.dll
        DLL Name: USER32.dll
        DLL Name: python312.dll
        DLL Name: KERNEL32.dll
        DLL Name: VCRUNTIME140_1.dll
matplotlib/_image.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: python312.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: KERNEL32.dll
matplotlib/_path.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: python312.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: KERNEL32.dll
matplotlib/_qhull.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: python312.dll
        DLL Name: api-ms-win-crt-utility-l1-1-0.dll
        DLL Name: api-ms-win-crt-time-l1-1-0.dll
        DLL Name: api-ms-win-crt-convert-l1-1-0.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: KERNEL32.dll
matplotlib/_tri.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: api-ms-win-crt-locale-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: python312.dll
        DLL Name: api-ms-win-crt-filesystem-l1-1-0.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: KERNEL32.dll
matplotlib/_ttconv.cp312-win_amd64.pyd
        DLL
 vma:            Hint    Time      Forward  DLL       First
        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: python312.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: KERNEL32.dll

Apologies if this counts as a duplicate of #168 or #181 (or the others), but the messaging in those issues seemed... muddy.

[MatthieuDartiailh]

Thanks for the separate ticket which makes things clearer on my end. I will do my best to investigate ASAP.

[Qix-]

Another alternative is statically linking the runtimes. Then none of the redist DLLs would be necessary to begin with, instead of needing to package them alongside the plugin.

[MatthieuDartiailh]

Python ships VCRUNTIME so there is no problem there but MSVCP is problematic.

@QuLogic I would welcome your insight since I mostly followed your suggestions here. One difference I noticed between kiwi setup and Matplotlib is that in kiwi I re-enabled FH4, do you think it may be related ?

[QuLogic]

I see you copied the setup from Matplotlib here:

kiwi/.github/workflows/release.yml

Lines 106 to 114 in 16735f9

	# On Windows, we explicitly request MSVC compilers (as GitHub Action runners have
	# MinGW on PATH that would be picked otherwise), switch to a static build for
	# runtimes, but use dynamic linking for `VCRUNTIME140.dll`, `VCRUNTIME140_1.dll`,
	# and the UCRT. This avoids requiring specific versions of `MSVCP140.dll`, while
	# keeping shared state with the rest of the Python process/extensions.
	CIBW_CONFIG_SETTINGS_WINDOWS: >-
	setup-args="--vsenv"
	setup-args="-Db_vscrt=mt"
	setup-args="-Dcpp_link_args=['ucrt.lib','vcruntime.lib','/nodefaultlib:libucrt.lib','/nodefaultlib:libvcruntime.lib']"

These are settings for Meson(-python) but I just noticed you are using setuptools to build. Those flags should make MSVCP statically linked with Meson, but they actually do nothing with setuptools (or perhaps something else entirely, as it didn't error?) Maybe there was a warning that could be seen if the build was in verbose mode?

From setuptools docs, it looks like you could set some environment variables to override build flags. The equivalent should be something like:

CFLAGS="/MT"
CPPFLAGS="/MT"
LDFLAGS="ucrt.lib vcruntime.lib /nodefaultlib:libucrt.lib /nodefaultlib:libvcruntime.lib"

[MatthieuDartiailh]

Thanks @QuLogic I missed this point. I considered switching to meson but never made the jump.

[MatthieuDartiailh]

I will have to try harder since my first attempt did not produce the expected result (https://github.com/nucleic/kiwi/actions/runs/11949548832).

[QuLogic]

Maybe try CIBW_ENVIRONMENT_WINDOWS

[MatthieuDartiailh]

Thanks for the suggestion but it does not seem to do the trick... (https://github.com/nucleic/kiwi/actions/runs/11999734728/job/33447988060)
The logs looks like the env vars are set but still something is not behaving properly. I will investigate directly patching setuptools in cppy. I will have the added benefit of applying to all the projects that uses it.

[MatthieuDartiailh]

Building locally after patching cppy, I obtain the following dependencies, are those fine;

        DLL Name: api-ms-win-crt-runtime-l1-1-0.dll
        DLL Name: api-ms-win-crt-heap-l1-1-0.dll
        DLL Name: api-ms-win-crt-string-l1-1-0.dll
        DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
        DLL Name: api-ms-win-crt-locale-l1-1-0.dll
        DLL Name: api-ms-win-crt-math-l1-1-0.dll
        DLL Name: VCRUNTIME140_1.dll
        DLL Name: VCRUNTIME140.dll
        DLL Name: python313.dll
        DLL Name: KERNEL32.dll

[MatthieuDartiailh]

Based on the first post of this issue I think so but I would be happy if somebody can confirm.

[LordAro]

Yup, vcruntime*.dll is distributed with cpython, kernel32.dll & api-*.dll are internal windows libraries, and python313 is... python :)

[QuLogic]

This is consistent with what we get: matplotlib/matplotlib#28687 (comment)

[MatthieuDartiailh]

Thanks

[MatthieuDartiailh]

Can somebody more knowledgeable than myself in cibuildwheel config help me figure out the reason this pipeline fails ?

https://github.com/nucleic/kiwi/actions/runs/12036490541/job/33557857394

The override of MD with MT while not ideal does not cause issues locally nor in the CI test build.

[QuLogic]

Ah, that's a 32-bit build; we don't have those any more in Matplotlib. Perhaps slightly different library names are needed there and @zooba might know.

[zooba]

Ah yes, for 32-bit builds I ended up adding an extra /alternatename:__except_handler4=__except_handler4_common link.exe argument. I'm not 100% sure that's going to be okay - none of my own code relies on C++ exceptions - but it seemed to be okay.

In context of my build backend that includes this as a (default) option: https://github.com/zooba/pymsbuild/blob/master/pymsbuild/targets/pyd.props#L21

[MatthieuDartiailh]

Thanks for the pointer. Given the name disabling FH4 might help. I will try to give that a try to see if I can keep 32bits windows this way.

[MatthieuDartiailh]

Sadly it does not work... I will just drop 32 bits build for now and re-investigate if somebody complains about their absence.

[zooba]

I will just drop 32 bits build for now and re-investigate if somebody complains about their absence.

Or document it as "for 32-bit builds you must also install the latest VC redist from https://aka.ms/vcredist" and move on with your life :)

[LordAro]

Looks like this has been fixed by #189 ? Any chance of a new release containing that in the next couple of weeks? I have deadlines :)

[MatthieuDartiailh]

I was considering waiting on the free-threaded build to be complete before cutting a new release. I will do my best to make a new release with or without free-threaded python support before Christmas.

[MatthieuDartiailh]

Since Setuptools new release is taking more time than I hoped I released 1.4.8 which fixes the linking issue and add support for free threaded Python except on Windows.