LUT 3D: with G'MIC 3.x or 4.x, a compressed LUT undoes -t N and can overflow per-thread buffers #22567

Is there an existing issue for this?

• I checked and did not find my issue in the already reported ones

Describe the bug

With a compressed LUT enabled, the X-Trans test edit made darktable-cli -t 4 abort with double free or corruption (out). A GUI run with the same edit segfaulted after switching to darkroom. The base edit without the LUT exported successfully.

Steps to reproduce

1. Build darktable commit f1adb4aca5 with OpenMP against G'MIC 4.0.5 built with OpenMP. Confirm that the LUT 3D plugin loads G'MIC 4.0.5 at runtime.

2. Create empty /tmp/22567/repro/config, /tmp/22567/repro/cache, and /tmp/22567/repro/xdg/gmic directories. Save the attached mire1-xtrans-lut3d.xmp.txt as /tmp/22567/repro/mire1-xtrans-lut3d.xmp. The XMP embeds the LUT; the .gmz archive is not needed for this export.

3. From the source checkout root, run the built darktable-cli:

   env -u OMP_NUM_THREADS -u OMP_THREAD_LIMIT \
     XDG_CACHE_HOME=/tmp/22567/repro/xdg \
     darktable-cli src/tests/integration/images/mire1-xtrans.raf \
     /tmp/22567/repro/mire1-xtrans-lut3d.xmp /tmp/22567/repro/out.png \
     --width 2048 --height 2048 --hq true \
     --apply-custom-presets false --core --disable-opencl \
     --library :memory: --configdir /tmp/22567/repro/config \
     --cachedir /tmp/22567/repro/cache -t 4

4. The export aborts with double free or corruption (out). This reproduced in five of five runs with fresh config and cache directories.

Expected behavior

darktable should finish the export while respecting the -t 4 thread count.

Logfile | Screenshot | Screencast

The CLI GDB trace ended with:

double free or corruption (out)

Thread 1 "darktable-cli" received signal SIGABRT, Aborted.
[...]
#7  malloc_printerr (str=str@entry=0x7ffff6ea6600 "double free or corruption (out)") at ./malloc/malloc.c:5341
[...]
#10 0x00007fffede710e6 in cleanup_pipe (self=<optimized out>, pipe=<optimized out>, piece=0x55555755a230) at /workspace/dt-master/src/iop/lut3d.c:1499

The GUI GDB trace caught:

Thread 525 "worker res 0" received signal SIGSEGV, Segmentation fault.
[...]
#0  0x00007fffd43ea271 in xtrans_markesteijn_interpolate._omp_fn.0 () at /workspace/dt-master/src/iop/demosaicing/xtrans.c:141
#1  0x00007ffff7e42b5f in ??? () at /usr/lib/x86_64-linux-gnu/libgomp.so.1

logs.zip

Commit

Not bisected. It depends on the G'MIC version: 2.9.4 does not change the count, 3.7.6 and 4.0.5 do. Versions in between were not checked.

Where did you obtain darktable from?

self compiled

darktable version

f1adb4a (current kofa/22436-threads-option-undone-by-graphicsmagick-overflows-perthread-buffers, based on master 8c64bf7)

What OS are you using?

Linux

What is the version of your OS?

Ubuntu 26.04.1 LTS

Describe your system

AMD Ryzen 5 5600X, 12 logical CPUs. G'MIC 4.0.5 was built with OpenMP and zlib enabled and GraphicsMagick disabled. darktable and G'MIC loaded the same libgomp.so.1. The GUI run used Xvfb.

Are you using OpenCL GPU in darktable?

No

If yes, what is the GPU card and driver?

No response

Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip

• The end-to-end CLI and GUI reproductions used G'MIC 4.0.5 on Ubuntu 26.04.1. G'MIC 3.x, other operating systems, other darktable versions, and JPEG inputs were not tested. The installed G'MIC 2.9.4 was used only for a standalone thread-count probe; it left the count unchanged.
• A fresh edit without history was not tested. Every export used a fresh config directory and XDG_CACHE_HOME; no Lua scripts ran.
• With the LUT, exports completed in three of three runs each with OMP_NUM_THREADS=12, without -t, and with -t 12. With OMP_NUM_THREADS=12 and -t 4, the GUI rendered the darkroom preview and stayed open for at least 20 seconds.

Thread-count evidence. GDB confirmed that G'MIC 4.0.5 called omp_set_num_threads(12) during LUT cache lookup and decompression (src/iop/lut3dgmic.cpp:48, :105; G'MIC src/gmic.cpp:3603) while OMP_NUM_THREADS was unset. darktable was started with -t 4, and X-Trans allocates per-thread buffers using darktable's thread count (src/iop/demosaicing/xtrans.c:66). This can leave an OpenMP region using more threads than the buffer allocation accounts for. The crash trace does not identify the first invalid write.

Relation to #22438. This is the G'MIC problem raised in #22438 (comment), not a GraphicsMagick problem. GraphicsMagick changes the count once, inside dt_init(), and #22438 restores it there. G'MIC changes it later, on the thread that loads the LUT, so #22438 cannot cover it.

Builds it can affect. cmake/modules/FindGMIC.cmake:35 accepts G'MIC from 2.7.0 up to, but not including, 10.0. The Windows nightly workflow installs the MSYS2 G'MIC package (.github/workflows/nightly.yml:245), currently 3.7.0. Its UCRT64 libgmic.dll imports omp_set_num_threads from libgomp-1.dll, the OpenMP runtime of GCC builds. The macOS build instructions use the MacPorts gmic-lib package (packaging/macosx/BUILD.txt:17), currently 4.0.5. Whether that package is built with OpenMP was not checked.

Workaround. Set OMP_NUM_THREADS to any value. G'MIC then does not change the thread count.

Possible fix. Restore the thread count when the functions in lut3dgmic.cpp return, or give darktable's parallel loops an explicit num_threads(dt_get_num_threads()), for example in DT_OMP_FOR (src/common/darktable.h:131). The second also covers other libraries that change the count.

Found by an AI agents that read the G'MIC and darktable sources while working on #22438.