llbuild: 'sqlite3.h' file not found

[SMillerDev]

Description

When trying to build swift 5.9.2 in Homebrew, the build fails with:

2023-12-24T16:55:37.9717810Z /tmp/swift-20231224-11721-bkt1we/llbuild/lib/Core/SQLiteBuildDB.cpp:28:10: fatal error: 'sqlite3.h' file not found
2023-12-24T16:55:37.9717913Z #include <sqlite3.h>
2023-12-24T16:55:37.9717995Z          ^~~~~~~~~~~
2023-12-24T16:55:37.9718084Z 1 error generated.

Despite sqlite3 being included as a dependency.

Reproduction

See Homebrew/homebrew-core#157113 for the full build

Expected behavior

Completed build

Environment

Swift 5.9.2

Additional information

While this very well might be a Homebrew issue, we could use some help trying to figure out if it is.

[compnerd]

How are you invoking swift build? You should be passing the header search path or ensuring that the systemLibrary dependency is set up with the right paths.

[Bo98]

How are you invoking swift build?

Via SwiftPM bootstrap rather than llbuild directly.

the systemLibrary dependency

Where is this dependency located? SQLite3 seems to only be passed as a linker flag here: https://github.com/apple/swift-llbuild/blob/783aec21649a6c47d1a8314db4144bdceb11df30/Package.swift#L85 and there's already an OpenBSD-specific hack for a similar issue: https://github.com/apple/swift-llbuild/blob/783aec21649a6c47d1a8314db4144bdceb11df30/Package.swift#L279-L287

It should be able to pick up the pkg-config like the CMake part of the build is able to.

[Bo98]

Issue seems to just be #733 again but s/OpenBSD/any other non-usr install/

[Bo98]

Reproduction steps:

$ docker run --rm -it swift:5.9.2
# apt update
# apt install g++ make cmake curl
# cd /root
# curl -LO https://www.sqlite.org/2024/sqlite-autoconf-3450100.tar.gz
# tar xf sqlite-autoconf-3450100.tar.gz
# cd sqlite-autoconf-3450100
# ./configure --prefix=/opt/prefix
# make install
# cd ..
# export PKG_CONFIG_PATH=/opt/prefix/lib/pkgconfig
# pkg-config --cflags sqlite3
-I/opt/prefix/include
# git clone https://github.com/apple/swift-package-manager
# cd swift-package-manager
# git checkout swift-5.9.2-RELEASE
# swift build
/root/swift-package-manager/.build/checkouts/swift-llbuild/lib/Core/SQLiteBuildDB.cpp:28:10: fatal error: 'sqlite3.h' file not found
#include <sqlite3.h>
         ^~~~~~~~~~~
1 error generated.

[finagolfin]

I ran another test in that linked 5.9.2 pull and I think he is right that llbuild should specify the systemLibrary dependency explicitly in Package.swift, or it fails for any non-standard directory where it may be installed, such as when cross-compiling.

I'm guessing he could make this more explicit by trying to build the llbuild package alone with SwiftPM in his example above and that would also fail, as the Swift toolchain build avoids that by building llbuild alone with CMake, obviously since SwiftPM isn't built yet when first building llbuild alone in the toolchain build.