Skip to content

Commit 04b0b48

Browse files
mihalicynmihalicyn
committed
github: add deb-pkg build pipeline
Signed-off-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@futurfusion.io>
1 parent fe2da50 commit 04b0b48

4 files changed

Lines changed: 450 additions & 63 deletions

File tree

‎.github/workflows/build-deb.yml‎

Lines changed: 290 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,290 @@
1+
name: Build kernel .deb
2+
3+
# Manual-only and deliberately not on push/pull_request: this packages a real
4+
# kernel (make bindeb-pkg), which is an order of magnitude heavier than the
5+
# config-generation matrix in generate-config.yml -- much more disk, much
6+
# more time -- so it only runs when someone explicitly asks for a specific
7+
# kernel version, not on every commit.
8+
on:
9+
workflow_dispatch:
10+
inputs:
11+
kernel_version:
12+
description: >
13+
Exact kernel version to build, e.g. 6.19.4 or 7.1.0-rc2. No
14+
reference config is required for this -- genconfig.sh only needs
15+
one if you also turn on "validate" below.
16+
required: true
17+
type: string
18+
flavor:
19+
description: "Which flavor to build the .deb for."
20+
required: false
21+
default: generic
22+
type: choice
23+
options:
24+
- generic
25+
- incus-os
26+
validate:
27+
description: >
28+
Also compare the generated config against misc/<series>/zabbly-config
29+
for this kernel's series, purely informational (shown in the job
30+
summary). Off by default: a reference isn't needed to build a .deb,
31+
only to check fidelity against it, and this lets you package a
32+
kernel series that has no reference config yet. Turning this on
33+
fails the run if that series has no reference.
34+
required: false
35+
default: false
36+
type: boolean
37+
38+
jobs:
39+
build-deb:
40+
# Each target is a distro whose own toolchain (gcc, pahole, libc) the
41+
# package has to be built against -- a .deb built with debian-12's gcc
42+
# isn't guaranteed compatible with debian-13 or ubuntu-24.04, and things
43+
# this repo's own tooling probes at generation time (CC_VERSION_TEXT,
44+
# PAHOLE_VERSION) need to reflect whichever toolchain actually builds the
45+
# kernel, not the runner's own Ubuntu image. So generation AND build both
46+
# happen inside the matching container, not just the final `make`.
47+
name: ${{ matrix.target }} (${{ inputs.flavor }} / ${{ inputs.kernel_version }})
48+
runs-on: ubuntu-latest
49+
# Generous but explicit: a full-featured build with modules can run well
50+
# past an hour on a 4-core runner. Adjust if your flavor is smaller/bigger.
51+
timeout-minutes: 180
52+
53+
strategy:
54+
# One target failing (say, a package name that doesn't exist on
55+
# debian-13 yet) shouldn't hide the others' results.
56+
fail-fast: false
57+
matrix:
58+
target:
59+
#- debian-12
60+
- debian-13
61+
#- ubuntu-24.04
62+
include:
63+
# Fully-qualified references rather than short names ("debian:12")
64+
# -- rootless podman's unqualified-search-registry behavior is a
65+
# config setting that varies by host, and being explicit here means
66+
# this doesn't depend on however the runner image has it set.
67+
#- target: debian-12
68+
# image: docker.io/library/debian:12
69+
- target: debian-13
70+
image: docker.io/library/debian:13
71+
#- target: ubuntu-24.04
72+
# image: docker.io/library/ubuntu:24.04
73+
74+
env:
75+
FLAVOR: ${{ inputs.flavor }}
76+
KERNEL_VERSION: ${{ inputs.kernel_version }}
77+
78+
steps:
79+
- name: Check out
80+
uses: actions/checkout@v4
81+
with:
82+
submodules: recursive
83+
84+
# Always computed (used in the job summary either way), but only
85+
# required to resolve to an existing reference when --validate is on.
86+
- name: Determine kernel series
87+
run: |
88+
set -euo pipefail
89+
series="$(echo "${KERNEL_VERSION%%-*}" | cut -d. -f1,2)"
90+
echo "series=$series" >> "$GITHUB_ENV"
91+
echo "Kernel series: $series"
92+
93+
# Only matters if --validate is going to be passed to genconfig.sh
94+
# later: fail fast, before spending time on a download, rather than
95+
# discovering the missing reference after the kernel fetch. When
96+
# validate is off, generation needs no reference at all, so this step
97+
# is skipped entirely.
98+
- name: Check the kernel series has a reference config
99+
if: inputs.validate
100+
run: |
101+
set -euo pipefail
102+
if [ ! -f "misc/${series}/zabbly-config" ]; then
103+
echo "error: no misc/${series}/zabbly-config for kernel version '$KERNEL_VERSION' (series '$series')" >&2
104+
echo " available series: $(ls misc | tr '\n' ' ')" >&2
105+
echo " or turn off 'validate' to build without comparing against a reference" >&2
106+
exit 1
107+
fi
108+
echo "Building $FLAVOR for Linux $KERNEL_VERSION (series $series), will validate against misc/${series}/zabbly-config"
109+
110+
# A kernel .deb build is one of the few things that reliably runs a
111+
# GitHub-hosted runner out of disk: default free space is ~25-29 GB out
112+
# of ~84 GB total, most of the rest being preinstalled toolchains
113+
# (Android SDK, .NET, multiple JDKs, GHC, ...) this job never touches.
114+
# Clearing them first is the standard workaround -- and matters even
115+
# more here, since podman's own image/container storage lands on the
116+
# same root filesystem as everything else.
117+
- name: Free up disk space
118+
run: |
119+
set -x
120+
df -h /
121+
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc \
122+
/opt/hostedtoolcache/CodeQL /usr/local/.ghcup \
123+
/usr/local/share/powershell /usr/local/share/chromium \
124+
/usr/share/swift || true
125+
df -h /
126+
127+
# Podman ships preinstalled on the GitHub-hosted Ubuntu runner images,
128+
# so there's nothing to install here -- just pull the target image as
129+
# its own step for clearer logs/timing, separate from the build itself.
130+
- name: Pull the ${{ matrix.target }} image
131+
run: podman pull "${{ matrix.image }}"
132+
133+
# Shared across all three matrix legs (same key, keyed only on kernel
134+
# version) since the source tarball itself doesn't depend on which
135+
# distro it's eventually packaged for. If two legs race on a cold
136+
# cache, the loser just re-downloads -- harmless, not a failure.
137+
- name: Cache kernel source tree
138+
id: kernel-cache
139+
uses: actions/cache@v4
140+
with:
141+
path: ~/kernel/linux-${{ inputs.kernel_version }}
142+
key: linux-src-${{ inputs.kernel_version }}
143+
144+
- name: Download kernel source tree
145+
if: steps.kernel-cache.outputs.cache-hit != 'true'
146+
run: |
147+
set -euo pipefail
148+
major="${KERNEL_VERSION%%.*}"
149+
url="https://cdn.kernel.org/pub/linux/kernel/v${major}.x/linux-${KERNEL_VERSION}.tar.xz"
150+
mkdir -p ~/kernel
151+
echo "Fetching $url"
152+
curl -fSL --retry 3 "$url" | tar -xJ -C ~/kernel
153+
test -f ~/kernel/linux-"$KERNEL_VERSION"/Makefile
154+
155+
# Written with container-internal paths, not host ones -- this .env is
156+
# only ever read from inside the container below. /kernel and /repo are
157+
# bind mounts set up in the podman run step further down.
158+
- name: Create .env
159+
run: |
160+
set -euo pipefail
161+
sed -e "s|^KERNEL_TREE_PATH=.*|KERNEL_TREE_PATH=/kernel/linux-${KERNEL_VERSION}|" \
162+
-e "s|^KERNEL_TREE_BUILD_PATH=.*|KERNEL_TREE_BUILD_PATH=/kbuild|" \
163+
.env.example > .env
164+
cat .env
165+
166+
# Everything from config generation through package collection runs as
167+
# one script inside the container, rather than as separate podman run
168+
# invocations per step -- state (the generated config, the build
169+
# directory) only needs to survive within a single container's
170+
# lifetime, not across steps. Written into the checkout itself so it
171+
# rides along on the /repo bind mount below without a mount of its own;
172+
# it's throwaway CI scaffolding, never committed.
173+
- name: Write the in-container build script
174+
run: |
175+
cat > .ci-build-deb.sh << 'SCRIPT'
176+
#!/bin/bash
177+
set -euo pipefail
178+
set -x
179+
180+
apt-get update
181+
apt-get install -y --no-install-recommends \
182+
build-essential flex bison bc libelf-dev libssl-dev dwarves \
183+
python3 fakeroot dpkg-dev rsync ca-certificates debhelper libdw-dev kmod
184+
185+
./check_slices.py "$FLAVOR"
186+
187+
# Always normalized here, unlike the regular config-generation
188+
# workflow where it's a toggle: a .deb has to come from a config
189+
# the kernel's own Kconfig actually accepts. --validate is passed
190+
# only when asked for -- generation itself needs no reference.
191+
if [ "$VALIDATE" = "true" ]; then
192+
./genconfig.sh "$FLAVOR" --normalize --validate
193+
else
194+
./genconfig.sh "$FLAVOR" --normalize --no-validate
195+
fi
196+
197+
# genconfig.sh --validate, as its last step, runs the reference
198+
# config through fix-config.sh too, reusing KERNEL_TREE_BUILD_PATH
199+
# as scratch space -- so when validate is on, by the time it
200+
# returns, that directory's .config is the *reference's* normalized
201+
# config, not ours. Restore ours before building. GENERATED_CONFIG_PATH
202+
# as read from .env is unsuffixed; genconfig.sh appends "-$FLAVOR"
203+
# internally for every flavor but generic, and that never gets
204+
# written back to .env, so mirror the rule here too.
205+
source .env
206+
if [ "$FLAVOR" != "generic" ]; then
207+
GENERATED_CONFIG_PATH="${GENERATED_CONFIG_PATH}-${FLAVOR}"
208+
fi
209+
cp "$GENERATED_CONFIG_PATH" "${KERNEL_TREE_BUILD_PATH}/.config"
210+
211+
# LOCALVERSION disambiguates flavors within one target -- without
212+
# it, generic and incus-os would produce identically-named .debs
213+
# for the same kernel version and target, since neither flavor sets
214+
# its own CONFIG_LOCALVERSION.
215+
make -C "$KERNEL_TREE_PATH" O="$KERNEL_TREE_BUILD_PATH" \
216+
-j"$(nproc)" LOCALVERSION="-${FLAVOR}" bindeb-pkg
217+
218+
# bindeb-pkg's output location has moved around across kernel
219+
# versions (srctree/.. vs objtree/..), so search both rather than
220+
# hardcode one, and copy into /out -- the one directory in this
221+
# container that's bind-mounted back to the host, so anything not
222+
# copied there is lost the moment the container is removed.
223+
mkdir -p /out
224+
found=0
225+
for dir in "$(dirname "$KERNEL_TREE_PATH")" "$(dirname "$KERNEL_TREE_BUILD_PATH")"; do
226+
while IFS= read -r -d '' f; do
227+
cp "$f" /out/
228+
found=1
229+
done < <(find "$dir" -maxdepth 1 -name '*.deb' -print0 2>/dev/null)
230+
done
231+
if [ "$found" -eq 0 ]; then
232+
echo "error: bindeb-pkg produced no .deb files -- the build must have failed silently" >&2
233+
exit 1
234+
fi
235+
ls -lh /out
236+
SCRIPT
237+
chmod +x .ci-build-deb.sh
238+
239+
- name: Prepare the output directory
240+
run: mkdir -p "$RUNNER_TEMP/out-${{ matrix.target }}"
241+
242+
# /repo is the checkout itself (rw) -- .env, the generated config, and
243+
# output/<flavor>/ all land there and are visible on the host
244+
# afterwards for free, no extra mount needed. /kernel is the cached
245+
# source tree, shared across all three targets. /out is where the
246+
# script above copies finished .debs before the container is removed.
247+
# KERNEL_TREE_BUILD_PATH (/kbuild) is deliberately NOT mounted: it's
248+
# pure scratch space, already extracted from before the container exits.
249+
- name: Build inside ${{ matrix.target }}
250+
run: |
251+
podman run --rm \
252+
-e FLAVOR \
253+
-e VALIDATE="${{ inputs.validate }}" \
254+
-v "$PWD":/repo \
255+
-v "$HOME/kernel":/kernel \
256+
-v "$RUNNER_TEMP/out-${{ matrix.target }}":/out \
257+
-w /repo \
258+
"${{ matrix.image }}" \
259+
bash /repo/.ci-build-deb.sh
260+
261+
# Rootless podman commonly maps the container's root user to a
262+
# subordinate UID range on the host, so files the script above wrote
263+
# into bind mounts as "root" may not be owned by the runner's own user
264+
# afterwards. Reads have generally been fine in testing (default
265+
# permissions are world-readable), but reclaiming ownership explicitly
266+
# is cheap insurance against a confusing permission error on upload.
267+
- name: Fix up ownership of container-written files
268+
if: always()
269+
run: |
270+
sudo chown -R "$(id -u):$(id -g)" \
271+
"$RUNNER_TEMP/out-${{ matrix.target }}" \
272+
"$GITHUB_WORKSPACE" || true
273+
274+
- name: Show diff against the reference config
275+
if: inputs.validate
276+
run: |
277+
echo "## Building $FLAVOR / $KERNEL_VERSION (${{ matrix.target }})" >> "$GITHUB_STEP_SUMMARY"
278+
echo >> "$GITHUB_STEP_SUMMARY"
279+
if [ -f "output/${FLAVOR}/diff" ]; then
280+
lines=$(wc -l < "output/${FLAVOR}/diff")
281+
echo "Config diff against misc/${series}/zabbly-config: $lines line(s)." >> "$GITHUB_STEP_SUMMARY"
282+
fi
283+
284+
- name: Upload .deb packages
285+
if: always()
286+
uses: actions/upload-artifact@v4
287+
with:
288+
name: ${{ inputs.flavor }}-${{ inputs.kernel_version }}-${{ matrix.target }}-deb
289+
path: ${{ runner.temp }}/out-${{ matrix.target }}/*.deb
290+
if-no-files-found: warn

‎.github/workflows/generate-config.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -147,9 +147,9 @@ jobs:
147147
- name: Generate the config
148148
run: |
149149
if [ "${{ inputs.normalize }}" = "true" ] || [ -z "${{ inputs.normalize }}" ]; then
150-
./genconfig.sh "$FLAVOR" --normalize
150+
./genconfig.sh "$FLAVOR" --normalize --validate
151151
else
152-
./genconfig.sh "$FLAVOR" --no-normalize
152+
./genconfig.sh "$FLAVOR" --no-normalize --validate
153153
fi
154154
155155
- name: Show diff against the reference config

0 commit comments

Comments
 (0)