Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructure RPATH and elf interpreter setting for Java, and add the LIBRARY_PATH to the additional RPATH #3571

Open
wants to merge 30 commits into
base: develop
Choose a base branch
from

Conversation

casparvl
Copy link
Contributor

This is the first step in fixing the issue observed here with Aladin-desktop that certain libraries aren't properly RPATH-ed, and as a result, the relevant X11 library cannot be found. The reason is that

  1. Java is a binary installation, so the compiler wrappers don't do any RPATH-ing. A previous PR already fixed that (OS) libraries from a custom sysroot would be added to the RPATH. However, that did not add anything from the regular dependencies to the RPATH. That omission is fixed in this PR.
  2. Java is currently at the system toolchain level, so the X11 libraries are not even among the dependencies (even though the Java libraries depend on them). This will be addressed in a follow-up PR to easybuild-easyconfigs.

Caspar van Leeuwen added 2 commits January 29, 2025 22:55
@casparvl
Copy link
Contributor Author

To get a baseline, these are the current headers and build logs for a library (libawt_xawt.so) and a binary (java) from EESSI, which uses both sysroot and rpath in it's builds:

$ readelf --dynamic /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so

Dynamic section at offset 0x5e278 contains 40 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libawt.so]
 0x0000000000000001 (NEEDED)             Shared library: [libXext.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libX11.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libXrender.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libXtst.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libXi.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libjava.so]
 0x0000000000000001 (NEEDED)             Shared library: [libjvm.so]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [libawt_xawt.so]
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64]
 0x000000000000000c (INIT)               0x10000
 0x000000000000000d (FINI)               0x4ce34
 0x0000000000000019 (INIT_ARRAY)         0x5f230
 0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x5f238
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x0000000000000004 (HASH)               0x64000
 0x000000006ffffef5 (GNU_HASH)           0x1dc8
 0x0000000000000005 (STRTAB)             0x65bc8
 0x0000000000000006 (SYMTAB)             0x33b0
 0x000000000000000a (STRSZ)              22356 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000003 (PLTGOT)             0x5f538
 0x0000000000000002 (PLTRELSZ)           7608 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0xdbd8
 0x0000000000000007 (RELA)               0xd698
 0x0000000000000008 (RELASZ)             1344 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x0000000000000018 (BIND_NOW)
 0x000000006ffffffb (FLAGS_1)            Flags: NOW ORIGIN
 0x000000006ffffffe (VERNEED)            0xd608
 0x000000006fffffff (VERNEEDNUM)         4
 0x000000006ffffff0 (VERSYM)             0xd018
 0x000000006ffffff9 (RELACOUNT)          32
 0x0000000000000000 (NULL)               0x0

and from the build logs

== 2024-05-29 04:43:42,834 java.py:178 DEBUG RPATH for /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so: $ORIGIN
== 2024-05-29 04:43:42,834 run.py:221 DEBUG run_cmd: running cmd patchelf --set-rpath '$ORIGIN:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec/gcc/x86_64-pc-linux-gnu/10:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib/gcc/x86_64-pc-linux-gnu/10' /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so (in /tmp/bot/easybuild/build/Java/17.0.6/system-system)
== 2024-05-29 04:43:42,834 run.py:247 INFO running cmd: patchelf --set-rpath '$ORIGIN:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec/gcc/x86_64-pc-linux-gnu/10:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib/gcc/x86_64-pc-linux-gnu/10' /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so
== 2024-05-29 04:43:42,842 run.py:689 DEBUG cmd "patchelf --set-rpath '$ORIGIN:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec/gcc/x86_64-pc-linux-gnu/10:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib/gcc/x86_64-pc-linux-gnu/10' /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so" exited with exit code 0 and output:

== 2024-05-29 04:43:42,842 run.py:725 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2024-05-29 04:43:42,842 run.py:221 DEBUG run_cmd: running cmd patchelf --print-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so (in /tmp/bot/easybuild/build/Java/17.0.6/system-system)
== 2024-05-29 04:43:42,842 run.py:247 INFO running cmd: patchelf --print-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so
== 2024-05-29 04:43:42,848 run.py:689 DEBUG cmd "patchelf --print-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so" exited with exit code 0 and output:
$ORIGIN:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec/gcc/x86_64-pc-linux-gnu/10:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib/gcc/x86_64-pc-linux-gnu/10

== 2024-05-29 04:43:42,848 run.py:725 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2024-05-29 04:43:42,848 java.py:186 DEBUG RPATH for /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib (prior to shrinking): $ORIGIN:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec/gcc/x86_64-pc-linux-gnu/10:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib/gcc/x86_64-pc-linux-gnu/10

== 2024-05-29 04:43:42,848 run.py:221 DEBUG run_cmd: running cmd patchelf --shrink-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so (in /tmp/bot/easybuild/build/Java/17.0.6/system-system)
== 2024-05-29 04:43:42,848 run.py:247 INFO running cmd: patchelf --shrink-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so
== 2024-05-29 04:43:42,941 run.py:689 DEBUG cmd "patchelf --shrink-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so" exited with exit code 0 and output:

== 2024-05-29 04:43:42,941 run.py:725 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2024-05-29 04:43:42,941 run.py:221 DEBUG run_cmd: running cmd patchelf --print-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so (in /tmp/bot/easybuild/build/Java/17.0.6/system-system)
== 2024-05-29 04:43:42,941 run.py:247 INFO running cmd: patchelf --print-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so
== 2024-05-29 04:43:42,947 run.py:689 DEBUG cmd "patchelf --print-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so" exited with exit code 0 and output:
$ORIGIN:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64

The same thing for a binary:

[casparl@tcn1 ~]$ readelf -l /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java | grep interpreter
      [Requesting program interpreter: /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/ld-linux-x86-64.so.2]
[casparl@tcn1 ~]$ readelf --dynamic /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java

Dynamic section at offset 0x2d20 contains 33 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libz.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libjli.so]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN:$ORIGIN/../lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64]
 0x000000000000000c (INIT)               0x1000
 0x000000000000000d (FINI)               0x141c
 0x0000000000000019 (INIT_ARRAY)         0x3d10
 0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x3d18
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x0000000000000004 (HASH)               0x5080
 0x000000006ffffef5 (GNU_HASH)           0x3c8
 0x0000000000000005 (STRTAB)             0x6000
 0x0000000000000006 (SYMTAB)             0x420
 0x000000000000000a (STRSZ)              952 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000015 (DEBUG)              0x0
 0x0000000000000003 (PLTGOT)             0x3f70
 0x0000000000000002 (PLTRELSZ)           264 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x8a0
 0x0000000000000007 (RELA)               0x7e0
 0x0000000000000008 (RELASZ)             192 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x0000000000000018 (BIND_NOW)
 0x000000006ffffffb (FLAGS_1)            Flags: NOW PIE
 0x000000006ffffffe (VERNEED)            0x7c0
 0x000000006fffffff (VERNEEDNUM)         1
 0x000000006ffffff0 (VERSYM)             0x796
 0x000000006ffffff9 (RELACOUNT)          4
 0x0000000000000000 (NULL)               0x0

And the build log:

== 2024-05-29 04:43:41,908 run.py:221 DEBUG run_cmd: running cmd file /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java (in /tmp/bot/easybuild/build/Java/17.0.6/system-system)
== 2024-05-29 04:43:41,908 run.py:247 INFO running cmd: file /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java
== 2024-05-29 04:43:41,915 run.py:689 DEBUG cmd "file /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java" exited with exit code 0 and output:
/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.18, not stripped

== 2024-05-29 04:43:41,915 run.py:725 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2024-05-29 04:43:41,915 run.py:221 DEBUG run_cmd: running cmd patchelf --print-interpreter /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java (in /tmp/bot/easybuild/build/Java/17.0.6/system-system)
== 2024-05-29 04:43:41,915 run.py:247 INFO running cmd: patchelf --print-interpreter /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java
== 2024-05-29 04:43:41,920 run.py:689 DEBUG cmd "patchelf --print-interpreter /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java" exited with exit code 0 and output:
/lib64/ld-linux-x86-64.so.2

== 2024-05-29 04:43:41,920 run.py:725 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2024-05-29 04:43:41,920 java.py:143 DEBUG ELF interpreter for /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java: /lib64/ld-linux-x86-64.so.2

== 2024-05-29 04:43:41,920 run.py:221 DEBUG run_cmd: running cmd patchelf --set-interpreter /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/ld-linux-x86-64.so.2 /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java (in /tmp/bot/easybuild/build/Java/17.0.6/system-system)
== 2024-05-29 04:43:41,920 run.py:247 INFO running cmd: patchelf --set-interpreter /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/ld-linux-x86-64.so.2 /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java
== 2024-05-29 04:43:41,926 run.py:689 DEBUG cmd "patchelf --set-interpreter /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/ld-linux-x86-64.so.2 /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java" exited with exit code 0 and output:

== 2024-05-29 04:43:41,926 run.py:725 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2024-05-29 04:43:41,927 run.py:221 DEBUG run_cmd: running cmd patchelf --print-interpreter /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java (in /tmp/bot/easybuild/build/Java/17.0.6/system-system)
== 2024-05-29 04:43:41,927 run.py:247 INFO running cmd: patchelf --print-interpreter /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java
== 2024-05-29 04:43:41,931 run.py:689 DEBUG cmd "patchelf --print-interpreter /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java" exited with exit code 0 and output:
/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/ld-linux-x86-64.so.2

== 2024-05-29 04:43:41,932 run.py:725 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2024-05-29 04:43:41,932 java.py:148 DEBUG ELF interpreter for /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java: /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/ld-linux-x86-64.so.2

== 2024-05-29 04:43:41,932 run.py:221 DEBUG run_cmd: running cmd patchelf --print-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java (in /tmp/bot/easybuild/build/Java/17.0.6/system-system)
== 2024-05-29 04:43:41,932 run.py:247 INFO running cmd: patchelf --print-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java
== 2024-05-29 04:43:41,937 run.py:689 DEBUG cmd "patchelf --print-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java" exited with exit code 0 and output:
$ORIGIN:$ORIGIN/../lib
== 2024-05-29 04:43:41,937 run.py:725 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2024-05-29 04:43:41,937 java.py:152 DEBUG RPATH for /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java: $ORIGIN:$ORIGIN/../lib
== 2024-05-29 04:43:41,937 run.py:221 DEBUG run_cmd: running cmd patchelf --set-rpath '$ORIGIN:$ORIGIN/../lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec/gcc/x86_64-pc-linux-gnu/10:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib/gcc/x86_64-pc-linux-gnu/10' /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java (in /tmp/bot/easybuild/build/Java/17.0.6/system-system)
== 2024-05-29 04:43:41,937 run.py:247 INFO running cmd: patchelf --set-rpath '$ORIGIN:$ORIGIN/../lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec/gcc/x86_64-pc-linux-gnu/10:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib/gcc/x86_64-pc-linux-gnu/10' /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java
== 2024-05-29 04:43:41,942 run.py:689 DEBUG cmd "patchelf --set-rpath '$ORIGIN:$ORIGIN/../lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/co
mpat/linux/x86_64/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64:/cvmfs/software.eessi.io/versions/2023
.06/compat/linux/x86_64/usr/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec/gcc/x86_64-pc-linux-gnu/10:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib/gcc/x86_64-pc-linux-gnu/10' /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java" exited with exit code 0 and output:

== 2024-05-29 04:43:41,942 run.py:725 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2024-05-29 04:43:41,943 run.py:221 DEBUG run_cmd: running cmd patchelf --print-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java (in /tmp/bot/ea
sybuild/build/Java/17.0.6/system-system)
== 2024-05-29 04:43:41,943 run.py:247 INFO running cmd: patchelf --print-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java
== 2024-05-29 04:43:41,948 run.py:689 DEBUG cmd "patchelf --print-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java" exited with exit code 0 and ou
tput:
$ORIGIN:$ORIGIN/../lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib:/cvmfs/software.eessi.io/versions/2023.
06/compat/linux/x86_64/usr/libexec/gcc/x86_64-pc-linux-gnu/10:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib/gcc/x86_64-pc-linux-gnu/10

== 2024-05-29 04:43:41,948 run.py:725 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2024-05-29 04:43:41,948 java.py:160 DEBUG RPATH for /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java (prior to shrinking): $ORIGIN:$ORIGIN/../lib:/cv
mfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/libexec:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr
/libexec/gcc/x86_64-pc-linux-gnu/10:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib/gcc/x86_64-pc-linux-gnu/10

== 2024-05-29 04:43:41,948 run.py:221 DEBUG run_cmd: running cmd patchelf --shrink-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java (in /tmp/bot/e
asybuild/build/Java/17.0.6/system-system)
== 2024-05-29 04:43:41,948 run.py:247 INFO running cmd: patchelf --shrink-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java
== 2024-05-29 04:43:41,954 run.py:689 DEBUG cmd "patchelf --shrink-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java" exited with exit code 0 and output:

== 2024-05-29 04:43:41,954 run.py:725 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2024-05-29 04:43:41,954 run.py:221 DEBUG run_cmd: running cmd patchelf --print-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java (in /tmp/bot/ea
sybuild/build/Java/17.0.6/system-system)
== 2024-05-29 04:43:41,954 run.py:247 INFO running cmd: patchelf --print-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java
== 2024-05-29 04:43:41,959 run.py:689 DEBUG cmd "patchelf --print-rpath /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java" exited with exit code 0 and ou
tput:
$ORIGIN:$ORIGIN/../lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64

== 2024-05-29 04:43:41,959 run.py:725 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2024-05-29 04:43:41,959 java.py:165 DEBUG RPATH for /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java (after shrinking): $ORIGIN:$ORIGIN/../lib:/cvmfs
/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64

@casparvl
Copy link
Contributor Author

After building with:

$ module list

Currently Loaded Modules:
  1) EESSI/2023.06   2) EasyBuild/4.9.4   3) EESSI-extend/2023.06-easybuild
$ eb Java-17.0.6.eb --rebuild --include-easyblocks-from-commit c2e0795fff0223793f94aed2c133a6e92e047607

I get the same (as expected):

[casparl@tcn1 ~]$ readelf -l /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java | grep interpreter
      [Requesting program interpreter: /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/ld-linux-x86-64.so.2]
$ readelf --dynamic /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/bin/java

Dynamic section at offset 0x2d20 contains 33 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libz.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libjli.so]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN:$ORIGIN/../lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x
86_64/usr/lib64]
 0x000000000000000c (INIT)               0x1000
 0x000000000000000d (FINI)               0x141c
 0x0000000000000019 (INIT_ARRAY)         0x3d10
 0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x3d18
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x0000000000000004 (HASH)               0x5080
 0x000000006ffffef5 (GNU_HASH)           0x3c8
 0x0000000000000005 (STRTAB)             0x6000
 0x0000000000000006 (SYMTAB)             0x420
 0x000000000000000a (STRSZ)              953 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000015 (DEBUG)              0x0
 0x0000000000000003 (PLTGOT)             0x3f70
 0x0000000000000002 (PLTRELSZ)           264 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x8a0
 0x0000000000000007 (RELA)               0x7e0
 0x0000000000000008 (RELASZ)             192 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x0000000000000018 (BIND_NOW)
 0x000000006ffffffb (FLAGS_1)            Flags: NOW PIE
 0x000000006ffffffe (VERNEED)            0x7c0
 0x000000006fffffff (VERNEEDNUM)         1
 0x000000006ffffff0 (VERSYM)             0x796
 0x000000006ffffff9 (RELACOUNT)          4
 0x0000000000000000 (NULL)               0x0

and for libawt_xawt.so

$ readelf --dynamic /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so

Dynamic section at offset 0x5e278 contains 40 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libawt.so]
 0x0000000000000001 (NEEDED)             Shared library: [libXext.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libX11.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libXrender.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libXtst.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libXi.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libjava.so]
 0x0000000000000001 (NEEDED)             Shared library: [libjvm.so]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [libawt_xawt.so]
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64
]
 0x000000000000000c (INIT)               0x10000
 0x000000000000000d (FINI)               0x4ce34
 0x0000000000000019 (INIT_ARRAY)         0x5f230
 0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x5f238
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x0000000000000004 (HASH)               0x64000
 0x000000006ffffef5 (GNU_HASH)           0x1dc8
 0x0000000000000005 (STRTAB)             0x65bc8
 0x0000000000000006 (SYMTAB)             0x33b0
 0x000000000000000a (STRSZ)              22357 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000003 (PLTGOT)             0x5f538
 0x0000000000000002 (PLTRELSZ)           7608 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0xdbd8
 0x0000000000000007 (RELA)               0xd698
 0x0000000000000008 (RELASZ)             1344 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x0000000000000018 (BIND_NOW)
 0x000000006ffffffb (FLAGS_1)            Flags: NOW ORIGIN
 0x000000006ffffffe (VERNEED)            0xd608
 0x000000006fffffff (VERNEEDNUM)         4
 0x000000006ffffff0 (VERSYM)             0xd018
 0x000000006ffffff9 (RELACOUNT)          32
 0x0000000000000000 (NULL)               0x0

Great, means we have retained the old behavior throughout the restructuring of the code. Next, we try with an Easyconfig at the GCCcore level which has X11 as dependency. This should make sure that the $EBROOTX11/lib gets added to the RPATH of libawt_xawt.so. Note that currently:

$ ldd /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so
ldd: warning: you do not have execution permission for `/home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt_xawt.so'
        linux-vdso.so.1 (0x00007ffc78995000)
        /opt/xalt/current/lib64/libxalt_init.so (0x000014cba6729000)
        libm.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libm.so.6 (0x000014cba6649000)
        libawt.so => /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libawt.so (0x000014cba6553000)
        libXext.so.6 => not found
        libX11.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64/libX11.so.6 (0x000014cba6405000)
        libXrender.so.1 => not found
        libdl.so.2 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libdl.so.2 (0x000014cba63fe000)
        libXtst.so.6 => not found
        libXi.so.6 => not found
        libjava.so => /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6/lib/libjava.so (0x000014cba63cf000)
        libjvm.so => not found
        libpthread.so.0 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libpthread.so.0 (0x000014cba63c8000)
        libc.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libc.so.6 (0x000014cba61f7000)
        /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/ld-linux-x86-64.so.2 (0x000014cba67c5000)
        libjvm.so => not found
        libxcb.so.1 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64/libxcb.so.1 (0x000014cba61cc000)
        libjvm.so => not found
        libXau.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64/libXau.so.6 (0x000014cba61c5000)
        libXdmcp.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64/libXdmcp.so.6 (0x000014cba61bd000)

Clearly, that is not right: a properly RPATH-ed library should be able to find all its dependencies.

@casparvl
Copy link
Contributor Author

Just as a test:

$ cat Java-17.0.6-GCCcore-13.2.0.eb
name = 'Java'
version = '17.0.6'
local_build = '10'

homepage = 'http://openjdk.java.net'
description = """Java Platform, Standard Edition (Java SE) lets you develop and deploy
Java applications on desktops and servers."""

toolchain = {'name': 'GCCcore', 'version': '13.2.0'}

local_tarball_tmpl = 'OpenJDK%%(version_major)sU-jdk_%s_linux_hotspot_%%(version)s_%s.tar.gz'

# Using the Adoptium Eclipse Temurin builds, recommended by https://whichjdk.com/#distributions

source_urls = ['https://github.com/adoptium/temurin%%(version_major)s-binaries/releases/download/jdk-%%(version)s+%s/'
               % local_build]
sources = [local_tarball_tmpl % ('%(jdkarch)s', local_build)]

checksums = [
    {
        local_tarball_tmpl % ('x64', local_build):
            'a0b1b9dd809d51a438f5fa08918f9aca7b2135721097f0858cf29f77a35d4289',
        local_tarball_tmpl % ('aarch64', local_build):
            '9e0e88bbd9fa662567d0c1e22d469268c68ac078e9e5fe5a7244f56fec71f55f',
        local_tarball_tmpl % ('ppc64le', local_build):
            'cb772c3fdf3f9fed56f23a37472acf2b80de20a7113fe09933891c6ef0ecde95',
    }
]

dependencies = [
    ('X11', '20231019'),
]

moduleclass = 'lang'

Installing this:

eb Java-17.0.6-GCCcore-13.2.0.eb --rebuild --include-easyblocks-from-commit c2e0795fff0223793f94aed2c133a6e92e047607

With that, the X11 libraries, including libXext.so are now picked up from the software-layer:

[casparl@tcn1 java_gcccore]$ readelf -a /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libawt_xawt.so | grep RUNPATH
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib:/cvmfs/software.eessi.io/
versions/2023.06/compat/linux/x86_64/lib64]
[casparl@tcn1 java_gcccore]$ ldd /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libawt_xawt.so
ldd: warning: you do not have execution permission for `/home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libawt_xawt.so'
        linux-vdso.so.1 (0x00007ffde8d92000)
        /opt/xalt/current/lib64/libxalt_init.so (0x000014ba7bcae000)
        libm.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libm.so.6 (0x000014ba7bbce000)
        libawt.so => /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libawt.so (0x000014ba7bad8000)
        libXext.so.6 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib/libXext.so.6 (0x000014ba7bac1000)
        libX11.so.6 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib/libX11.so.6 (0x000014ba7b975000)
        libXrender.so.1 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib/libXrender.so.1 (0x000014ba7b967000)
        libdl.so.2 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libdl.so.2 (0x000014ba7b962000)
        libXtst.so.6 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib/libXtst.so.6 (0x000014ba7b959000)
        libXi.so.6 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib/libXi.so.6 (0x000014ba7b946000)
        libjava.so => /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libjava.so (0x000014ba7b915000)
        libjvm.so => not found
        libpthread.so.0 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libpthread.so.0 (0x000014ba7b905000)
        libc.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libc.so.6 (0x000014ba7b734000)
        /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/ld-linux-x86-64.so.2 (0x000014ba7bd4a000)
        libjvm.so => not found
        libxcb.so.1 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib/libxcb.so.1 (0x000014ba7b705000)
        libjvm.so => not found
        libXau.so.6 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib/libXau.so.6 (0x000014ba7b6fd000)
        libXdmcp.so.6 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib/libXdmcp.so.6 (0x000014ba7b6f5000)

Note that libjvm.so is still not found. This is located in $EBROOTJAVA/lib/server/libjvm.so. I'll add that too by adding $ORIGIN/../lib/server (in addition to the conventional $ORIGIN/../lib)

@casparvl
Copy link
Contributor Author

Installing with

eb Java-17.0.6-GCCcore-13.2.0.eb --rebuild --include-easyblocks-from-commit b0128470319aa51861157e27f8608c202ef8408b

We now get:

$ ldd /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libawt_xawt.so
ldd: warning: you do not have execution permission for `/home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libawt_xawt.so'
        linux-vdso.so.1 (0x00007ffd5e9ae000)
        /opt/xalt/current/lib64/libxalt_init.so (0x0000154f56ac7000)
        libm.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libm.so.6 (0x0000154f569e7000)
        libawt.so => /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libawt.so (0x0000154f568f1000)
        libXext.so.6 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib/libXext.so.6 (0x0000154f568da000)
        libX11.so.6 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib/libX11.so.6 (0x0000154f5678e000)
        libXrender.so.1 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib/libXrender.so.1 (0x0000154f56780000)
        libdl.so.2 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libdl.so.2 (0x0000154f5677b000)
        libXtst.so.6 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib/libXtst.so.6 (0x0000154f56772000)
        libXi.so.6 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib/libXi.so.6 (0x0000154f5675f000)
        libjava.so => /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libjava.so (0x0000154f5672e000)
        libjvm.so => /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/../lib/server/libjvm.so (0x0000154f553e7000)
        libpthread.so.0 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libpthread.so.0 (0x0000154f553e2000)
        libc.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libc.so.6 (0x0000154f55211000)
        /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/ld-linux-x86-64.so.2 (0x0000154f56b63000)
        libxcb.so.1 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib/libxcb.so.1 (0x0000154f551e2000)
        librt.so.1 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/librt.so.1 (0x0000154f551dd000)
        libXau.so.6 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib/libXau.so.6 (0x0000154f551d5000)
        libXdmcp.so.6 => /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib/libXdmcp.so.6 (0x0000154f551cd000)
[casparl@tcn1 java_gcccore]$ readelf -a /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libawt_xawt.so | grep RUNPATH
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN:$ORIGIN/../lib/server:/cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen2/software/X11/20231019-GCCcore-13.2.0/lib:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64]

That looks completely fine, now everything is found.

@casparvl
Copy link
Contributor Author

Next step, I'll make some GCCcore based java easyconfigs, that can utilize this Easyblock PR properly

@casparvl
Copy link
Contributor Author

casparvl commented Jan 30, 2025

I'm... confused. To test the RPATH sanity check, I removed the X11 dependency in my Easyconfig file. I see

== 2025-01-30 03:04:11,172 run.py:702 DEBUG cmd "ldd /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libawt_xawt.so" exited with exit code 0 and output:
ldd: warning: you do not have execution permission for `/home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libawt_xawt.so'
        linux-vdso.so.1 (0x00007fffa03f3000)
        /opt/xalt/current/lib64/libxalt_init.so (0x0000147844ac1000)
        libm.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libm.so.6 (0x00001478449e1000)
        libawt.so => /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libawt.so (0x00001478448eb000)
        libXext.so.6 => not found
        libX11.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64/libX11.so.6 (0x000014784479b000)
        libXrender.so.1 => not found
        libdl.so.2 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libdl.so.2 (0x0000147844796000)
        libXtst.so.6 => not found
        libXi.so.6 => not found
        libjava.so => /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libjava.so (0x0000147844767000)
        libjvm.so => /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/../lib/server/libjvm.so (0x000014784341f000)
        libpthread.so.0 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libpthread.so.0 (0x000014784341a000)
        libc.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libc.so.6 (0x0000147843249000)
        /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/ld-linux-x86-64.so.2 (0x0000147844b5d000)
        libxcb.so.1 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64/libxcb.so.1 (0x000014784321e000)
        librt.so.1 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/librt.so.1 (0x0000147843219000)
        libXau.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64/libXau.so.6 (0x0000147843212000)
        libXdmcp.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/usr/lib64/libXdmcp.so.6 (0x000014784320a000)

== 2025-01-30 03:04:11,172 run.py:738 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2025-01-30 03:04:11,175 easyblock.py:3180 WARNING Library libXext.so.6 not found for /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libawt_xawt.so
== 2025-01-30 03:04:11,175 easyblock.py:3180 WARNING Library libXrender.so.1 not found for /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libawt_xawt.so
== 2025-01-30 03:04:11,175 easyblock.py:3180 WARNING Library libXtst.so.6 not found for /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libawt_xawt.so
== 2025-01-30 03:04:11,175 easyblock.py:3180 WARNING Library libXi.so.6 not found for /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib/libawt_xawt.so

Why doesn't this result in a failed sanity check?

Edit: I may know, I probably should have returned the values from the RPATH sanity check step, as those are the failure message. I.e. the function sanity_check_rpath should return something non-empty upon failure.

@casparvl
Copy link
Contributor Author

Ok, so even with the correct deps, it is failing, because my patchelf sets RUNPATH instead of RPATH. I'll have to use --force-rpath

@casparvl
Copy link
Contributor Author

Ok, this is for tomorrow to solve: some library is still not found in the RPATH:

ldd: warning: you do not have execution permission for `/home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib64/libjsound.so'
        linux-vdso.so.1 (0x00007ffcff3d1000)
        /opt/xalt/current/lib64/libxalt_init.so (0x00001534539a1000)
        libjava.so => /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib64/libjava.so (0x0000153453972000)
        libjvm.so => /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib64/../lib/server/libjvm.so (0x000015345262b000)
        libasound.so.2 => not found
        libc.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libc.so.6 (0x000015345244d000)
        libm.so.6 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libm.so.6 (0x000015345236d000)
        libdl.so.2 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libdl.so.2 (0x0000153452368000)
        libpthread.so.0 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/libpthread.so.0 (0x0000153452363000)
        librt.so.1 => /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/librt.so.1 (0x000015345235e000)
        /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64/ld-linux-x86-64.so.2 (0x00001534539e6000)

== 2025-01-30 03:22:24,035 run.py:738 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
...
== 2025-01-30 03:22:24,042 run.py:702 DEBUG cmd "readelf -d /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib64/libjsound.so" exited with exit code
0 and output:

Dynamic section at offset 0xf960 contains 32 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libjava.so]
 0x0000000000000001 (NEEDED)             Shared library: [libjvm.so]
 0x0000000000000001 (NEEDED)             Shared library: [libasound.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [libjsound.so]
 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN:$ORIGIN/../lib/server:/cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib64]
 0x000000000000000c (INIT)               0x5000
 0x000000000000000d (FINI)               0xcb30
 0x0000000000000019 (INIT_ARRAY)         0x10950
 0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x10958
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x0000000000000004 (HASH)               0x12000
 0x000000006ffffef5 (GNU_HASH)           0xae8
 0x0000000000000005 (STRTAB)             0x128e8
 0x0000000000000006 (SYMTAB)             0xda8
 0x000000000000000a (STRSZ)              7448 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000003 (PLTGOT)             0x10ba0
 0x0000000000000002 (PLTRELSZ)           3192 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x39c8
 0x0000000000000007 (RELA)               0x3920
 0x0000000000000008 (RELASZ)             168 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x0000000000000018 (BIND_NOW)
 0x000000006ffffffb (FLAGS_1)            Flags: NOW ORIGIN
 0x000000006ffffffe (VERNEED)            0x38c0
 0x000000006fffffff (VERNEEDNUM)         2
 0x000000006ffffff0 (VERSYM)             0x373c
 0x000000006ffffff9 (RELACOUNT)          3
 0x0000000000000000 (NULL)               0x0

@casparvl
Copy link
Contributor Author

Ooff, Java is messy:

No '(RPATH)' found in 'readelf -d' output for /home/casparl/eessi/versions/2023.06/software/linux/x86_64/amd/zen2/software/Java/17.0.6-GCCcore-13.2.0/lib64/jexec:
Dynamic section at offset 0x2d50 contains 28 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000c (INIT)               0x1000
 0x000000000000000d (FINI)               0x16bc
 0x0000000000000019 (INIT_ARRAY)         0x3d40
 0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x3d48
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x0000000000000004 (HASH)               0x2e8
 0x000000006ffffef5 (GNU_HASH)           0x3d8
 0x0000000000000005 (STRTAB)             0x5e0
 0x0000000000000006 (SYMTAB)             0x400
 0x000000000000000a (STRSZ)              256 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000015 (DEBUG)              0x0
 0x0000000000000003 (PLTGOT)             0x3f50
 0x0000000000000002 (PLTRELSZ)           360 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x7f0
 0x0000000000000007 (RELA)               0x748
 0x0000000000000008 (RELASZ)             168 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x0000000000000018 (BIND_NOW)
 0x000000006ffffffb (FLAGS_1)            Flags: NOW PIE
 0x000000006ffffffe (VERNEED)            0x708
 0x000000006fffffff (VERNEEDNUM)         1
 0x000000006ffffff0 (VERSYM)             0x6e0
 0x000000006ffffff9 (RELACOUNT)          3
 0x0000000000000000 (NULL)               0x0

The reason there is no RPATH is because this is an executable that is located in /lib.

I see two options here: either I find all files under /lib and check with file if they are ELF executables - and if so, I add the RPATH, or I manually add these files to the list of files to be RPATH-ed. The advantage of the first approach is that if things get added in the future, we won't hit the same issue. However, if /lib contains a lot of files, this becomes quite expensive.

Right now, there are only two files that this concerns, so I think hardcoding is the way to go here. The good news is that if an additional executable ends up in /lib in the future, the RPATH sanity check will catch it.

@casparvl
Copy link
Contributor Author

Hm, we actually do the checking for dynamically linked executables for the RPATH sanity check. So maybe it's not too expensive. I'll just do that instead, it'll be more robust and we can spare the extra few seconds.

@casparvl
Copy link
Contributor Author

Hm, I think we can improve here:

  1. I think we should make this post_install_cmds at the Binary level, so it can be used for all Binary installations (though maybe the default should not be 'on' yet?)
  2. I think we should respect the bin_lib_subdirs when setting RPATH. This means that for java, we'd have to add lib/server to the bin_lib_subdirs, probably by overwriting the option with a different default (like we do now for run_rpath_sanity_check).
  3. I think we should change the default RPATH sanity check behavior. I don't think it should respect bin_lib_subdirs, I think it should just check for all dynamically linked files in bin and lib, including subdirs (i.e. with an os.walk). The sanity check should be a catch-all. If it is not checked here, it will be missed. The sanity check is a nice way to find out that we should add something to bin_lib_subdirs for a particular software in order to make sure it gets RPATH-ed.
  4. If we are using bin_lib_subdirs in combination with os.listdir, we should check for (and RPATH-patch) all dynamically linked files, not just shared libs under /lib and /lib64.

@casparvl
Copy link
Contributor Author

casparvl commented Jan 30, 2025

Hm, I think we can improve here:

  1. I think we should make this post_install_cmds at the Binary level, so it can be used for all Binary installations. The default should be 'off'.
  2. I think we should respect the bin_lib_subdirs when setting RPATH. This means that for java, we'd have to add lib/server to the bin_lib_subdirs, probably by overwriting the option with a different default (like we do now for run_rpath_sanity_check).
  3. I think we should change the default RPATH sanity check behavior. I don't think it should respect bin_lib_subdirs, I think it should just check for all dynamically linked files in bin and lib, including subdirs (i.e. with an os.walk). The sanity check should be a catch-all. If it is not checked here, it will be missed. The sanity check is a nice way to find out that we should add something to bin_lib_subdirs for a particular software in order to make sure it gets RPATH-ed.
  4. If we are using bin_lib_subdirs in combination with os.listdir, we should check for (and RPATH-patch) all dynamically linked files, not just shared libs under /lib and /lib64.
  5. The RPATH patching should respect filter_rpath_sanity_libs (and maybe we should rename that option to something not-specific to the sanity, something like filter_rpath_libs) and NOT set an RPATH on any libs or executables in this option.
  6. Licensing may not allow modification of headers in binaries, so long term, the best defaults would be to not do patching of RPATH, but do run the RPATH sanity check for all binary installs. This way, contributors of binary-based installs will have to make a conscious choice to either skip the RPATH sanity check (and probably set LD_LIBRARY_PATH=$LIBRARY_PATH for these modules to make sure the correct libs are still found) OR they have to enable the RPATH patching.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant