Building formulae from source on aarch64 keeps automatically symlinking to /lib64/ld-linux-x86-64.so.2

[heaths]

Additional Info

• I am using Windows Subsystem for Linux

Output of brew config

HOMEBREW_VERSION: 4.5.2
ORIGIN: https://github.com/Homebrew/brew
HEAD: 6f39076b3c2251994419215279d0525ef667fc31
Last commit: 12 days ago
Branch: stable
Core tap JSON: 21 May 23:34 UTC
HOMEBREW_PREFIX: /home/linuxbrew/.linuxbrew
HOMEBREW_BROWSER: wslview
HOMEBREW_CASK_OPTS: []
HOMEBREW_DISPLAY: :0
HOMEBREW_EDITOR: hx
HOMEBREW_MAKE_JOBS: 12
Homebrew Ruby: 3.4.3 => /home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/vendor/portable-ruby/3.4.3/bin/ruby
CPU: dodeca-core 64-bit arm
Clang: 20.1.5
Git: 2.43.0 => /bin/git
Curl: 8.5.0 => /bin/curl
Kernel: Linux 5.15.167.4-microsoft-standard-WSL2 aarch64 GNU/Linux
OS: Ubuntu 24.04.2 LTS (noble)
WSL: 2 (Microsoft Store)
Host glibc: 2.39
/usr/bin/gcc: 13.3.0
/usr/bin/ruby: N/A
glibc: N/A
gcc@11: N/A
gcc: 14.2.0_1
xorg: N/A

Output of brew doctor

Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: The following formulae have the same name as core formulae:
  jandedobbeleer/oh-my-posh/oh-my-posh
Some of these can be resolved with:
  brew untap jandedobbeleer/oh-my-posh

Warning: Your CPU architecture (arm64) is not supported. We only support
x86_64 CPU architectures. You will be unable to use binary packages (bottles).

This is a Tier 2 configuration:
  https://docs.brew.sh/Support-Tiers#tier-2
Do not report any issues to Homebrew/* repositories!
Read the above document instead before opening any issues or PRs.

Description of issue

I'm trying to run brew install --build-from-source gh, for example, but every time on a do on an aarch64 machine set up to cross-build for amd64, /home/linuxbrew/.linuxbrew/lib/ld.so gets symlinked to /lib64/ld-linux-x86-64.so.2 and I can't figure out why. I've deleted /home/linuxbrew/.linuxbrew/lib/ld.so and symlinked to /lib/ld-linux-aarch64.so.1 numerous times. Any installed formulae and even brew doctor and brew config work until I re-run a command like brew install --build-from-source {formulae}.

To repro on my system:

$ ls -l /home/linuxbrew/.linuxbrew/lib/ld.so
lrwxrwxrwx 1 root root 26 May 21 16:40 /home/linuxbrew/.linuxbrew/lib/ld.so -> /lib/ld-linux-aarch64.so.1

$ brew install --build-from-source gh
gh 2.72.0 is already installed but outdated (so it will be upgraded).
Warning: Your CPU architecture (arm64) is not supported. We only support
x86_64 CPU architectures. You will be unable to use binary packages (bottles).

This is a Tier 2 configuration:
  https://docs.brew.sh/Support-Tiers#tier-2
Do not report any issues to Homebrew/* repositories!
Read the above document instead before opening any issues or PRs.

==> Fetching gh
==> Downloading https://raw.githubusercontent.com/Homebrew/homebrew-core/6e524fb47e61d295a2c73a125e472a690f556b54/Formula/g/gh.rb
Already downloaded: /home/heaths/.cache/Homebrew/downloads/deae80d3dd1d05351e22804638b21506610a47bf534e5dd0359cee2b6f07f2ed--gh.rb
==> Downloading https://github.com/cli/cli/archive/refs/tags/v2.73.0.tar.gz
Already downloaded: /home/heaths/.cache/Homebrew/downloads/c9977ee941167417ad194687ad0c2250afaef6d3e40300260112ff73fc674a15--cli-2.73.0.tar.gz
==> Upgrading gh
  2.72.0 -> 2.73.0
Error: An exception occurred within a child process:
  Errno::ELIBBAD: Accessing a corrupted shared library - /home/linuxbrew/.linuxbrew/bin/gcc-14

$ ls -l /home/linuxbrew/.linuxbrew/lib/ld.so
lrwxrwxrwx 1 heaths heaths 27 May 21 16:49 /home/linuxbrew/.linuxbrew/lib/ld.so -> /lib64/ld-linux-x86-64.so.2

Notice how it points back to /lib64/ld-linux-x86-64.so.2. I'm not sure how or why this happens.

I'm using ubuntu-24.04 and added am64 arch support. A few packages are installed e.g.,

$ apt list --installed | grep amd64

gcc-14-base/noble-updates,noble-security,now 14.2.0-4ubuntu2~24.04 amd64 [installed,automatic]
libasan8-amd64-cross/noble-updates,noble-updates,noble-security,noble-security,now 14.2.0-4ubuntu2~24.04cross1 all [installed,automatic]
libatomic1-amd64-cross/noble-updates,noble-updates,noble-security,noble-security,now 14.2.0-4ubuntu2~24.04cross1 all [installed,automatic]
libc6-amd64-cross/noble,noble,now 2.39-0ubuntu8cross1 all [installed]
libc6-dev-amd64-cross/noble,noble,now 2.39-0ubuntu8cross1 all [installed,automatic]
libc6/noble-updates,noble-security,now 2.39-0ubuntu8.4 amd64 [installed]
libgcc-13-dev-amd64-cross/noble-updates,noble-updates,noble-security,noble-security,now 13.3.0-6ubuntu2~24.04cross1 all [installed,automatic]
libgcc-s1-amd64-cross/noble-updates,noble-updates,noble-security,noble-security,now 14.2.0-4ubuntu2~24.04cross1 all [installed,automatic]
libgcc-s1/noble-updates,noble-security,now 14.2.0-4ubuntu2~24.04 amd64 [installed,automatic]
libgomp1-amd64-cross/noble-updates,noble-updates,noble-security,noble-security,now 14.2.0-4ubuntu2~24.04cross1 all [installed,automatic]
libhwasan0-amd64-cross/noble-updates,noble-updates,noble-security,noble-security,now 14.2.0-4ubuntu2~24.04cross1 all [installed,automatic]
libidn2-0/noble-updates,now 2.3.7-2build1.1 amd64 [installed,automatic]
libitm1-amd64-cross/noble-updates,noble-updates,noble-security,noble-security,now 14.2.0-4ubuntu2~24.04cross1 all [installed,automatic]
liblsan0-amd64-cross/noble-updates,noble-updates,noble-security,noble-security,now 14.2.0-4ubuntu2~24.04cross1 all [installed,automatic]
libquadmath0-amd64-cross/noble-updates,noble-updates,noble-security,noble-security,now 14.2.0-4ubuntu2~24.04cross1 all [installed,automatic]
libstdc++6-amd64-cross/noble-updates,noble-updates,noble-security,noble-security,now 14.2.0-4ubuntu2~24.04cross1 all [installed,automatic]
libtsan2-amd64-cross/noble-updates,noble-updates,noble-security,noble-security,now 14.2.0-4ubuntu2~24.04cross1 all [installed,automatic]
libubsan1-amd64-cross/noble-updates,noble-updates,noble-security,noble-security,now 14.2.0-4ubuntu2~24.04cross1 all [installed,automatic]
libunistring5/noble-updates,now 1.1-2build1.1 amd64 [installed,automatic]
linux-libc-dev-amd64-cross/noble,noble,now 6.8.0-25.25cross1 all [installed,automatic]
pkg-config/noble,now 1.8.1-2build1 amd64 [installed]
pkgconf/noble,now 1.8.1-2build1 amd64 [installed,automatic]

[heaths]

Tracing even a dry-run, I see brew is updating ld.so:

$ strace brew install --build-from-source -n gh

...
newfstatat(AT_FDCWD, "/home/linuxbrew/.linuxbrew/opt/glibc/bin/ld.so", 0xfffff9e891f8, 0) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/lib64/ld-linux-x86-64.so.2", {st_mode=S_IFREG|0755, st_size=236616, ...}, 0) = 0
geteuid()                               = 1000
getegid()                               = 1000
getuid()                                = 1000
getgid()                                = 1000
faccessat(AT_FDCWD, "/lib64/ld-linux-x86-64.so.2", X_OK) = 0
newfstatat(AT_FDCWD, "/home/linuxbrew/.linuxbrew/lib/ld.so", {st_mode=S_IFREG|0755, st_size=203968, ...}, 0) = 0
geteuid()                               = 1000
getegid()                               = 1000
getuid()                                = 1000
getgid()                                = 1000
faccessat(AT_FDCWD, "/home/linuxbrew/.linuxbrew/lib/ld.so", R_OK) = 0
readlinkat(AT_FDCWD, "/home/linuxbrew/.linuxbrew/lib/ld.so", "/lib/ld-linux-aarch64.so.1", 100) = 26
newfstatat(AT_FDCWD, "/home/linuxbrew/.linuxbrew/lib", {st_mode=S_IFDIR|0775, st_size=40960, ...}, 0) = 0
newfstatat(AT_FDCWD, "/home/linuxbrew/.linuxbrew/lib/ld.so", {st_mode=S_IFREG|0755, st_size=203968, ...}, 0) = 0
unlinkat(AT_FDCWD, "/home/linuxbrew/.linuxbrew/lib/ld.so", 0) = 0
symlinkat("/lib64/ld-linux-x86-64.so.2", AT_FDCWD, "/home/linuxbrew/.linuxbrew/lib/ld.so") = 0
mmap(NULL, 131072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xff13cc720000
munmap(0xff13cc720000, 65536)           = 0
newfstatat(AT_FDCWD, "/home/linuxbrew/.linuxbrew/opt/glibc/bin/ld.so", 0xfffff9e891f8, 0) = -1 ENOENT (No such file or directory)
newfstatat(AT_FDCWD, "/home/linuxbrew/.linuxbrew/opt/gcc", {st_mode=S_IFDIR|0755, st_size=4096, ...}, 0) = 0
...

According to brew info glibc, it's not installed.

I removed /etc/ld.so.cache and rebuilt it. ldconfig -p shows only aarch64 libraries but even after all that, the above steps - re-symlinking ld.so and doing a dry-run install - still overwrites the symlink to point to /lib64/ld-linux-x86-64.so.2.

[heaths]

What's odd is that it had been working. Could this be because I was working on a formulae to eventually build a bottle, but ended up pulling in gcc and that started this chain? My system had been configured for xplat for a while before that with no problems. It's not clear what might've triggered this.

[heaths]

FWIW, I put /lib/ld-linux-aarch64.so.1 at the top of DYNAMIC_LINKERS in Homebrew/Library/Homebrew/extend/os/linux/install.rb and not having any issues so far. /me knocks on wood

[heaths]

@Bo98 would you be open to a PR to conditionally order those libs on amd64 vs aarch64? While I'm a polyglot with expertise in quite a few languages, I only know enough Ruby to dabble. Can constants (can they be called constants?) like DYNAMIC_LINKERS be conditioned on platform invariants? I can look around online as well. The idea would be to keep the order as-is for amd64 but change it for aarch64.

[Bo98]

Yeah I will soon.

[bluefing]

I hit this (or something very similar) on ARM Linux containers (MacOS m3 + Colima) with libc6:amd64 installed (needed for x86_64 Android SDK binaries.

Brew install re-symlinks ld.so to /lib64/ld-linux-x86-64.so.2, breaking brew-installed ARM binaries with "accessing a corrupted shared library" error. System tools installed in the Dockerfile were ok, e.g. /usr/bin/ls but eza was corrupted.

Installing brew's own glibc fixed the symlink because symlink_ld_so checks $HOMEBREW_PREFIX/opt/glibc/bin/ld.so first, before scanning the DYNAMIC_LINKERS array.

The symlink then points to brew's native ARM glibc instead of the system x86_64 one, and appears to survive subsequent brew install calls - at least based on the limited testing i did.

gcc was also required, without it, brew bottles fail with libgcc_s.so.1: cannot open shared object file

The clue for the workaround was in the brew install logs. I don't recall exactly but there was something about the glibc version being out of date.

So the workaround for me was

brew install glibc gcc

The order matters, glibc/gcc must be installed after amd64 multi -arch libs.

Hope this helps someone - i lost literally more that a day banging my head against this.