Skip to content

llvm-quick-fuzz: Download LLVM versions through https://apt.llvm.org instead of downloading binary artifacts #330

@RyanGlScott

Description

@RyanGlScott

When attempting to upgrade llvm-quick-fuzz to support LLVM 19 and later recently, I hit a snag. Previously, we have been obtaining Clang/LLVM binaries for each version that llvm-pretty-bc-parser supports by downloading the corresponding binary artifact from https://github.com/llvm/llvm-project/releases. In particular, we have been downloading binary artifacts of the form clang+llvm-<VER>-x86_64-linux-gnu-ubuntu-*.tar.xz, which provides a relatively lightweight bindist (never more than 1GB) that contains clang and the assorted LLVM utilities that we need.

As of LLVM 19, however, the llvm-project repo no longer appears to offer such bindists. Instead, they now offer LLVM-<VER>-Linux-X64.tar.xz bindists. These bindists are much larger, apparently because they bundle many additional LLVM utilities, header files, and library files that llvm-pretty-bc-parser does not make use of. (For instance, LLVM-19.1.7-Linux-X64.tar.xz is about 1.5GB in size before being uncompressed.) While we could switch to using these bindists, I worry that doing so would be a bit wasteful.

What's more, the llvm-quick-fuzz job is currently using ubuntu-22.04, and we will eventually need to upgrade this to a more recent Ubuntu version. Some of the bindists that we are currently downloading rely on old versions of libtinfo, however, such as the bindists that target Ubuntu 18.04 or older. Some experimentation suggests that one simply cannot run these old LLVM binaries on Ubuntu 24.04, which will pose an obstacle when we need to upgrade llvm-quick-fuzz's Ubuntu version.

One possible solution to both problems is to stop directly downloading bindists from llvm-project and to switch to using https://apt.llvm.org as a source of Ubuntu binaries. That page contains an automatic installation script (llvm.sh) that installs the appropriate PPA for a given Ubuntu version and can then allow you to install a range of Clang/LLVM versions. This would make our LLVM installation in llvm-quick-fuzz much more future-proof and would likely dramatically simplify the details of actually downloading the binaries, as we would no longer need to hard-code long, bespoke URLs like we are currently doing.

The main downside to switching to https://apt.llvm.org is that the llvm.sh script only supports LLVM 9 through 21 as of the time of writing, so if we switched, we would have to drop llvm-quick-fuzz support for pre-9 versions of LLVM. Moreover, it's not clear to me if this script will continue to support old versions of LLVM, or if they will eventually be dropped.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions