Skip to content

ci: build WebKitGTK as two shards on two runners - #32

Merged
hanthor merged 1 commit into
fix/build-failures-batch1from
claude/utah-rawhide-build-iriu0i
Sep 2, 2026
Merged

ci: build WebKitGTK as two shards on two runners#32
hanthor merged 1 commit into
fix/build-failures-batch1from
claude/utah-rawhide-build-iriu0i

Conversation

@hanthor

@hanthor hanthor commented Sep 2, 2026

Copy link
Copy Markdown
Member

Experiment on top of #30, targeting its branch rather than main since the lane infrastructure lives there. Nothing here is meant to merge while main is active; it is the "shard the long build across runners" question answered with the one package that actually needs it.

What the numbers say

From run 33461820477, the last run where the heavy lane completed:

Package Job time
webkitgtk 5h 01m
glycin 27 min
mesa 25 min
mozjs140 19 min
librsvg2 15 min
everything else under 10 min

pipewire is not slow; it sits in a late lane for the abseil ordering, not for its runtime. WebKitGTK is the wall clock of the whole factory, and it runs five hours against a six-hour job limit, which is why the heavy lane has a 360-minute timeout and a UTAH_HEAVY_RUNNER hook. That build compiles the source twice, once per GTK port, and the two compiles share nothing but the tarball.

What this does

  • WebKitGTK builds as two shards on two runners. A source-manifest entry may now name another entry's recipe directory and the rpm defines to build it with (tools/recipe.py). webkitgtk builds the GTK 4 port with --without gtk3; the new webkit2gtk4.1 entry builds the GTK 3 port with --without gtk4. Expected: about 2.5 hours each, side by side, instead of 5 in sequence, and neither near the limit.
  • The spec gates each port behind a bcond (%bcond_without gtk4, %bcond_without gtk3), so a plain rpmbuild with no defines still builds both, exactly as Fedora does. Only the GTK 4 shard produces webkitgtk-debugsource; two RPMs of that name with different contents would otherwise collide in the repository.
  • The lane workflow resolves the recipe directory and defines outside the container and passes them in. The defines go to dnf builddep as well as to rpmbuild, so a shard resolves the BuildRequires of the port it will compile.
  • Build identity hashes the shared directory plus the entry (defines included), so the shards get distinct keys, and the prepare job treats an edit to the shared recipe as a change to both entries. dist_git_name stays what it was, the lookaside name; malcontent-bootstrap has both directories and its identity is unchanged.
  • mozjs140 leaves the heavy lane. At 18 minutes it belongs in the ordinary stage-1 lane, and gjs stops waiting behind WebKitGTK for it.

What was considered and not done

  • A true distributed compile of one port across three or four runners (sccache-dist with the runners meshed over a tailnet, one scheduler, the rest build servers). It is the only way to go below one port per runner, and it is feasible, but it needs a tailnet secret, a toolchain tarball shipped to Ubuntu hosts from the Fedora container, and unprivileged user namespaces re-enabled on 24.04 for bubblewrap. Worth a follow-up once the two-shard number is in hand; not worth guessing at before.
  • Dropping debuginfo for the factory build. It would take a large fixed cost off the tail of each shard (the spec asks for 32 GB per vCPU for debuginfo processing and gets 4), but it changes what the repository ships, so that is a decision, not an optimisation.

Validation

tools/validate.py, tools/check_workflow_quoting.py, and the unit tests (17, including new tests/test_recipe.py) pass; both workflows parse; the spec's conditionals balance. No rpm tooling is available where this was written, so the spec has not been run through rpmspec. The heavy lane on this PR is the test: two webkitgtk jobs in s1-heavy, each finishing with a full set of that port's subpackages.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KVSCoKWBWAuFFZyLwteEbB


Generated by Claude Code

Measured on run 33461820477: webkitgtk 5h01m, the next longest package
25 minutes (mesa), then mozjs140 at 18. One package is the wall clock,
and it sits five hours against a six-hour job limit. Its recipe compiles
the source twice, once per GTK port, and the two compiles share nothing
but the tarball.

So the recipe now builds twice on two runners. A source-manifest entry
may name another entry's recipe directory and the rpm defines to build
it with (tools/recipe.py); webkitgtk builds the GTK 4 port with
--without gtk3 and the new webkit2gtk4.1 entry builds the GTK 3 port with
--without gtk4. The spec gates each port behind a bcond, so a plain
rpmbuild with no defines still builds both, as Fedora does. Only the
GTK 4 shard produces the -debugsource package; two of the same name and
different contents would otherwise collide in the repository.

The lane workflow resolves the recipe directory and defines outside the
container and passes them in; the defines go to builddep as well as to
rpmbuild, so a shard resolves the BuildRequires of the port it will
compile. The build identity hashes the shared directory and the entry,
defines included, so the shards get distinct keys and an edit to the
recipe invalidates both. The source pipeline keeps dist_git_name for the
lookaside, which is a different question from which directory to build.

mozjs140 leaves the heavy lane: at eighteen minutes it belongs in the
ordinary stage-1 lane, and gjs stops waiting behind WebKitGTK for it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KVSCoKWBWAuFFZyLwteEbB
@hanthor
hanthor merged commit 60e8ca0 into fix/build-failures-batch1 Sep 2, 2026
155 of 156 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants