Skip to content

Commit 5273931

Browse files
authored
docs(ci): add build-and-test-libremesh workflow documentation (#65)
* docs(ci): add build-and-test-libremesh workflow documentation * docs(ci-runner): add workflows table and troubleshooting section * docs(labgrid): document helper scripts generate_places_yaml, resolve_target, provision_mesh_ip * docs(rack): add CI workflow quick reference section * docs(firmware): add pre-built firmwares directory overview * docs(som): add CI firmware build workflow to automation section * docs(rack): add switch and power scripts reference section * docs(running-tests): add section on using CI-built firmware artifacts * docs(developer): add section on triggering CI workflow from GitHub * docs(diseno): add lime-packages CI firmware build flow page * docs(diseno): add lime-packages CI hardware test flow page * docs(nav): add lime-packages CI flow pages to mkdocs nav * docs(index): add quick actions table to home page * docs(integration): add CI workflow entries to design reading guide * docs(demos): add index table to video demos page * docs(new-lab): add CI build and test link to related pages * docs(routine): add CI runner verification section * docs(operar): add virtual mesh local setup guide from vms/README
1 parent 7daa7be commit 5273931

17 files changed

Lines changed: 503 additions & 2 deletions

docs/configuracion/ci-runner.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,3 +73,28 @@ To move the runner from one repo to another (or user to org):
7373
## 6. Ownership transfer (if needed in near future)
7474

7575
When the repo transfers to an org, attached runners move with it. The systemd service name may still reference the old owner; this should not affect operation.
76+
77+
---
78+
79+
## 7. Workflows that use this runner
80+
81+
The following workflows in `fcefyn-testbed/fcefyn_testbed_utils` target this runner with `runs-on: [self-hosted, testbed-fcefyn]`:
82+
83+
| Workflow | Trigger | What it does on the runner |
84+
|----------|---------|---------------------------|
85+
| `build-and-test-libremesh.yml` | Manual (`workflow_dispatch`) | Downloads firmware artifact built on GitHub-hosted runners, reserves the DUT via labgrid, loads firmware, and runs libremesh-tests with pytest. |
86+
87+
The `build` job of that workflow runs on GitHub-hosted runners (no lab hardware needed). Only the `flash_and_test` job lands on this runner.
88+
89+
See [CI: Build & Test](../operar/ci-build-and-test.md) for full usage instructions.
90+
91+
---
92+
93+
## 8. Troubleshooting
94+
95+
| Symptom | Cause | Fix |
96+
|---------|-------|-----|
97+
| Runner shows **Offline** in GitHub | Service stopped | `sudo systemctl restart actions.runner.*` |
98+
| `flash_and_test` job queued but never starts | Runner offline or label mismatch | Check runner labels include `testbed-fcefyn` |
99+
| `PermissionError` on labgrid coordinator | Wrong `/etc/labgrid` ownership | See §4 above |
100+
| Job fails at "Reserve DUT" | DUT locked by a previous run | `labgrid-client -p <place> unlock` |

docs/demos.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,11 @@
22

33
This section collects demo videos showing various operations on the testbed.
44

5+
| Demo | Topic |
6+
|------|-------|
7+
| [Remote access to the HIL testbed](#remote-access-to-the-hil-testbed) | Administrator access and lab infrastructure recovery |
8+
| [Developer remote access through Labgrid](#developer-remote-access-through-labgrid) | Developer SSH + multi-node mesh test run |
9+
510
---
611

712
## Remote access to the HIL testbed

docs/diseno/integration-overview.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -165,3 +165,6 @@ sequenceDiagram
165165
| Set up the WireGuard tunnel and contribute hardware to openwrt-tests | [openwrt-tests onboarding](openwrt-tests-onboarding.md) |
166166
| VLAN design, `switch-vlan` CLI, switch configuration | [Lab architecture](lab-architecture.md) |
167167
| Virtual mesh tests with QEMU and vwifi | [Virtual mesh](virtual-mesh.md) |
168+
| Build and test lime-packages from a PR via CI | [CI: Build & Test](../operar/ci-build-and-test.md) |
169+
| Understand the lime-packages fork CI build pipeline | [lime-packages CI build](lime-packages-ci-flow.md) |
170+
| Understand the lime-packages fork CI test pipeline | [lime-packages CI tests](lime-packages-test-flow.md) |

docs/diseno/lime-packages-ci-flow.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
12
# lime-packages CI: firmware build pipeline
23

34
How the **fcefyn-testbed/lime-packages** fork builds per-device LibreMesh
@@ -271,3 +272,4 @@ consumes this artifact.
271272
To add a new board to this pipeline see the [add-device guide][add].
272273

273274
[add]: lime-packages-add-device.md
275+

docs/diseno/lime-packages-test-flow.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -376,4 +376,4 @@ The workflow only references `environment: physical-lab` by name. All
376376
governance (teams, rulesets, environment reviewers) lives in GitHub
377377
repository/org settings, not in the YAML. Any organisation adopting
378378
this workflow creates its own `physical-lab` environment and teams
379-
without modifying the workflow file.
379+
without modifying the workflow file.

docs/diseno/new-lab-contribution.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,3 +126,4 @@ A lab can contribute to both paths at once: the infrastructure is shared. Scenar
126126
- [Switch](../configuracion/switch-config.md) - VLAN layout, `switch.conf` multi-user setup, `switch-vlan` invocation.
127127
- [Lab architecture](lab-architecture.md) - per-lab coordinator, VLAN scheduling, locking.
128128
- [CI runner](../configuracion/ci-runner.md) - self-hosted runner setup (Scenario B).
129+
- [CI: Build & Test](../operar/ci-build-and-test.md) - automated firmware build and test workflow using the self-hosted runner.

docs/index.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,3 +21,16 @@ Relationship between orchestration host, switch, gateway, DUTs, power, and seria
2121
![High-level diagram of the testbed and main components](img/diagrams/general-design-overview.png)
2222

2323
The design builds on the **remote lab** model from [openwrt-tests](https://github.com/aparcar/openwrt-tests), but scope is not limited to **adding devices** to that network. It also **reuses and extends** the approach with **local infrastructure**, along the same axes (orchestration, network, power, serial) with a focus on **LibreMesh** testing.
24+
25+
---
26+
27+
## Quick actions
28+
29+
| I want to… | Go to |
30+
|---|---|
31+
| Run tests on lab hardware from my machine | [Developer quickstart](operar/developer-remote-access.md) |
32+
| Build a LibreMesh firmware and test it automatically | [CI: Build & Test](operar/ci-build-and-test.md) |
33+
| SSH into a DUT | [SSH access to DUTs](operar/dut-ssh-access.md) |
34+
| Check the rack layout and device IPs | [Rack quick reference](operar/rack-cheatsheets.md) |
35+
| Build firmware manually | [Build firmware](operar/build-firmware-manual.md) |
36+
| Add a new device to the lab | [Adding a DUT](operar/dut-onboarding.md) |

docs/operar/build-firmware-manual.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -182,6 +182,34 @@ Firmware appears under `bin/` as in standard OpenWrt.
182182

183183
---
184184

185+
## Pre-built firmwares (`firmwares/`)
186+
187+
The `firmwares/` directory in this repo contains pre-built images ready to use in the lab without compiling. Useful for quick tests or when the build pipeline is unavailable.
188+
189+
```
190+
firmwares/
191+
├── belkin_rt3200/
192+
│ ├── libremesh/ ← LibreMesh images (initramfs + sysupgrade)
193+
│ └── openwrt/ ← Stock OpenWrt images
194+
├── bananapi_bpi-r4/
195+
├── librerouter_librerouter-v1/
196+
├── openwrt_one/
197+
├── tplink-wdr3500/
198+
└── qemu/
199+
```
200+
201+
Each device folder has:
202+
203+
| File pattern | Purpose |
204+
|---|---|
205+
| `*initramfs*.bin` / `*initramfs*.itb` | TFTP boot — loads into RAM, flash untouched |
206+
| `*sysupgrade*.bin` / `*sysupgrade*.itb` | Flash write via `sysupgrade` |
207+
| `*sdcard*.img.gz` | SD card image (Banana Pi R4 only) |
208+
209+
To use a pre-built initramfs in tests, set `LG_IMAGE` to the file path. See [Running tests](lab-running-tests.md).
210+
211+
---
212+
185213
## Automatic builds via PR (lime-packages fork)
186214

187215
The **fcefyn-testbed/lime-packages** fork includes `.github/workflows/build-firmware.yml`: on pull requests (and manual dispatch), GitHub Actions builds a **local lime_packages feed** with the OpenWrt SDK, then one **firmware image per row** in `.github/ci/targets.yml` using ImageBuilder. Successful runs upload **`firmware-<device>.*`** and **`lime-feed-<arch>`** artifacts.
@@ -192,3 +220,5 @@ For architecture, caching, and feed indexing details, see:
192220
- [lime-packages CI: hardware tests](../diseno/lime-packages-test-flow.md) (downstream **libremesh-tests** on the `testbed-fcefyn` self-hosted runner)
193221

194222
Manual procedures above still apply when you build outside CI or need a custom `menuconfig` / full Buildroot tree.
223+
224+
For the automated CI workflow that builds and tests firmware directly in this repo, see [CI: Build & Test](ci-build-and-test.md).

docs/operar/ci-build-and-test.md

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
# CI: Build & Test LibreMesh
2+
3+
This workflow builds a LibreMesh firmware image from source and runs automated
4+
tests on a physical device in the FCEFYN lab.
5+
6+
It is triggered **manually** — it does not run automatically on every commit.
7+
You decide when to run it and with which parameters.
8+
9+
---
10+
11+
## When to use it
12+
13+
Run this workflow when you want to:
14+
15+
- Test a specific version or branch of
16+
[lime-packages](https://github.com/libremesh/lime-packages) on real hardware
17+
- Verify that a set of packages installs and boots correctly on a lab device
18+
- Produce a firmware image with custom packages for a specific device
19+
20+
---
21+
22+
## How to run it
23+
24+
1. Go to the repository on GitHub
25+
2. Click the **Actions** tab
26+
3. Select **Build LibreMesh and Test on DUT** in the left panel
27+
4. Click **Run workflow**
28+
5. Fill in the inputs (see below) and click the green **Run workflow** button
29+
30+
---
31+
32+
## Inputs
33+
34+
| Input | Required | Default | Description |
35+
|-------|----------|---------|-------------|
36+
| `duts` | yes | `belkin_rt3200` | Device(s) to build and test. Comma-separated. Use `all` for every lab device. |
37+
| `lime_ref` | yes | `v2024.1` | Branch, tag, or commit SHA of lime-packages to build from. |
38+
| `openwrt_version` | no | `23.05.5` | OpenWrt version to use. Must be compatible with `lime_ref`. |
39+
| `extra_packages` | no | _(empty)_ | Space-separated packages to add or remove from the base set. Prefix with `-` to remove. Example: `luci-app-dawn -lime-proto-batadv` |
40+
| `config_file` | no | _(empty)_ | Repo-relative path to a config file to inject as `/etc/config/<name>` in the firmware. Example: `firmware/configs/belkin_rt3200.conf` |
41+
42+
### Supported devices
43+
44+
| `duts` value | Hardware | OpenWrt target |
45+
|---|---|---|
46+
| `belkin_rt3200` | Belkin RT3200 / Linksys E8450 | `mediatek/mt7622` |
47+
| `openwrt_one` | OpenWrt One | `mediatek/filogic` |
48+
| `bananapi_r4` | Banana Pi R4 | `mediatek/filogic` |
49+
| `librerouter` | LibreRouter v1 | `ath79/generic` |
50+
51+
### OpenWrt / lime-packages compatibility
52+
53+
| `openwrt_version` | Compatible `lime_ref` |
54+
|---|---|
55+
| `23.05.5` | `v2024.1` |
56+
| `24.10.5` | `master` or newer tags |
57+
| `25.12.0` | `master` or newer tags |
58+
59+
---
60+
61+
## What happens when you run it
62+
63+
The workflow has three jobs that run in sequence:
64+
65+
### 1. Resolve matrix (~5 seconds, GitHub-hosted)
66+
67+
Parses the `duts` input and builds a job matrix so each device runs in
68+
parallel. For example, `"belkin_rt3200,librerouter"` becomes two independent
69+
build jobs.
70+
71+
### 2. Build (~20–25 min per device, GitHub-hosted)
72+
73+
For each device, two Docker containers run back to back:
74+
75+
**Step 1 — OpenWrt SDK**
76+
Downloads `ghcr.io/openwrt/sdk:<target>-<subtarget>-v<openwrt_version>` and
77+
compiles the lime-packages listed below from source, using the exact git ref
78+
you specified in `lime_ref`. This is the slow step.
79+
80+
Packages compiled:
81+
82+
- `lime-system`
83+
- `lime-proto-babeld`
84+
- `lime-proto-batadv`
85+
- `lime-proto-anygw`
86+
- `lime-hwd-openwrt-wan`
87+
- `lime-app`
88+
- `shared-state` + `shared-state-babeld_hosts` + `shared-state-bat_hosts` + `shared-state-nodes_and_links`
89+
- `babeld-auto-gw-mode`
90+
- anything you add via `extra_packages`
91+
92+
**Step 2 — OpenWrt ImageBuilder**
93+
Downloads `ghcr.io/openwrt/imagebuilder:<target>-<subtarget>-v<openwrt_version>`
94+
and assembles the compiled `.ipk` packages into a complete firmware image
95+
(`.bin` or `.itb`). This step takes ~2–3 minutes.
96+
97+
The firmware is uploaded as a GitHub Actions artifact named
98+
`firmware-<dut>-<lime_ref>-<short_sha>` and kept for 7 days.
99+
100+
### 3. Flash and test (lab self-hosted runner, `testbed-fcefyn`)
101+
102+
Runs on the physical T430 machine in the FCEFYN lab.
103+
104+
1. Downloads the firmware artifact from step 2
105+
2. Reserves the target device via [labgrid](https://labgrid.readthedocs.io)
106+
(waits if the device is busy)
107+
3. Loads the firmware onto the device
108+
4. Runs the [libremesh-tests](https://github.com/fcefyn-testbed/libremesh-tests)
109+
test suite with pytest
110+
5. Releases the device when done (even if tests fail)
111+
112+
---
113+
114+
## Examples
115+
116+
**Test the latest stable lime-packages on the Belkin RT3200:**
117+
```
118+
duts: belkin_rt3200
119+
lime_ref: v2024.1
120+
openwrt_version: 23.05.5
121+
```
122+
123+
**Test a feature branch on all devices:**
124+
```
125+
duts: all
126+
lime_ref: my-feature-branch
127+
openwrt_version: 24.10.5
128+
```
129+
130+
**Add a package and remove another:**
131+
```
132+
duts: belkin_rt3200
133+
lime_ref: v2024.1
134+
extra_packages: luci-app-dawn -lime-proto-batadv
135+
```
136+
137+
**Inject a custom network config:**
138+
```
139+
duts: belkin_rt3200
140+
lime_ref: v2024.1
141+
config_file: firmware/configs/belkin_rt3200.conf
142+
```
143+
144+
---
145+
146+
## Testing the build locally with `act`
147+
148+
[act](https://github.com/nektos/act) lets you run the build job on your own
149+
machine without pushing to GitHub. Only the `build` job works locally — the
150+
`flash_and_test` job requires physical lab hardware and is automatically
151+
skipped.
152+
153+
```bash
154+
act workflow_dispatch \
155+
--workflows .github/workflows/build-and-test-libremesh.yml \
156+
--job build \
157+
--input duts="belkin_rt3200" \
158+
--input lime_ref="v2024.1" \
159+
--input openwrt_version="23.05.5" \
160+
--input config_file="" \
161+
--input extra_packages="" \
162+
-P ubuntu-latest=catthehacker/ubuntu:act-22.04 \
163+
--artifact-server-path /tmp/act-artifacts
164+
```
165+
166+
The firmware is saved to `/tmp/act-artifacts` and to `./images/` in the repo
167+
root (not committed).

docs/operar/developer-remote-access.md

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -305,7 +305,24 @@ ZeroTier is **not** required for developer access. It is used only by lab admins
305305

306306
---
307307

308-
## 10. Reference
308+
## 10. Triggering the CI workflow as a developer
309+
310+
Instead of running tests manually, you can trigger the full build + test pipeline from GitHub:
311+
312+
1. Go to [fcefyn-testbed/fcefyn_testbed_utils → Actions → Build LibreMesh and Test on DUT](https://github.com/fcefyn-testbed/fcefyn_testbed_utils/actions/workflows/build-and-test-libremesh.yml)
313+
2. Click **Run workflow** and fill in:
314+
- `duts`: the device you want to test
315+
- `lime_ref`: the lime-packages branch, tag, or commit you want to validate
316+
- `openwrt_version`: must be compatible with `lime_ref`
317+
3. The build runs on GitHub's servers (~20 min). The `flash_and_test` job then runs automatically on the lab hardware.
318+
319+
This does **not** require SSH access to the lab. The only requirement is having a GitHub account with access to the repository.
320+
321+
For full workflow documentation: [CI: Build & Test](ci-build-and-test.md).
322+
323+
---
324+
325+
## 11. Reference
309326

310327
- [Running tests (host-side)](lab-running-tests.md)
311328
- [SSH access to DUTs](dut-ssh-access.md) - VLAN lifecycle and mesh SSH

0 commit comments

Comments
 (0)