From 272ac94fd0f6a737ce4d5f1b886a0c897d64647a Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:30:12 -0300 Subject: [PATCH 01/18] docs(ci): add build-and-test-libremesh workflow documentation --- docs/operar/build-firmware-manual.md | 2 + docs/operar/ci-build-and-test.md | 167 +++++++++++++++++++++++++++ mkdocs.yml | 1 + 3 files changed, 170 insertions(+) create mode 100644 docs/operar/ci-build-and-test.md diff --git a/docs/operar/build-firmware-manual.md b/docs/operar/build-firmware-manual.md index d499e00..e829ab3 100644 --- a/docs/operar/build-firmware-manual.md +++ b/docs/operar/build-firmware-manual.md @@ -192,3 +192,5 @@ For architecture, caching, and feed indexing details, see: - [lime-packages CI: hardware tests](../diseno/lime-packages-test-flow.md) (downstream **libremesh-tests** on the `testbed-fcefyn` self-hosted runner) Manual procedures above still apply when you build outside CI or need a custom `menuconfig` / full Buildroot tree. + +For the automated CI workflow that builds and tests firmware directly in this repo, see [CI: Build & Test](ci-build-and-test.md). diff --git a/docs/operar/ci-build-and-test.md b/docs/operar/ci-build-and-test.md new file mode 100644 index 0000000..2c922d6 --- /dev/null +++ b/docs/operar/ci-build-and-test.md @@ -0,0 +1,167 @@ +# CI: Build & Test LibreMesh + +This workflow builds a LibreMesh firmware image from source and runs automated +tests on a physical device in the FCEFYN lab. + +It is triggered **manually** — it does not run automatically on every commit. +You decide when to run it and with which parameters. + +--- + +## When to use it + +Run this workflow when you want to: + +- Test a specific version or branch of + [lime-packages](https://github.com/libremesh/lime-packages) on real hardware +- Verify that a set of packages installs and boots correctly on a lab device +- Produce a firmware image with custom packages for a specific device + +--- + +## How to run it + +1. Go to the repository on GitHub +2. Click the **Actions** tab +3. Select **Build LibreMesh and Test on DUT** in the left panel +4. Click **Run workflow** +5. Fill in the inputs (see below) and click the green **Run workflow** button + +--- + +## Inputs + +| Input | Required | Default | Description | +|-------|----------|---------|-------------| +| `duts` | yes | `belkin_rt3200` | Device(s) to build and test. Comma-separated. Use `all` for every lab device. | +| `lime_ref` | yes | `v2024.1` | Branch, tag, or commit SHA of lime-packages to build from. | +| `openwrt_version` | no | `23.05.5` | OpenWrt version to use. Must be compatible with `lime_ref`. | +| `extra_packages` | no | _(empty)_ | Space-separated packages to add or remove from the base set. Prefix with `-` to remove. Example: `luci-app-dawn -lime-proto-batadv` | +| `config_file` | no | _(empty)_ | Repo-relative path to a config file to inject as `/etc/config/` in the firmware. Example: `firmware/configs/belkin_rt3200.conf` | + +### Supported devices + +| `duts` value | Hardware | OpenWrt target | +|---|---|---| +| `belkin_rt3200` | Belkin RT3200 / Linksys E8450 | `mediatek/mt7622` | +| `openwrt_one` | OpenWrt One | `mediatek/filogic` | +| `bananapi_r4` | Banana Pi R4 | `mediatek/filogic` | +| `librerouter` | LibreRouter v1 | `ath79/generic` | + +### OpenWrt / lime-packages compatibility + +| `openwrt_version` | Compatible `lime_ref` | +|---|---| +| `23.05.5` | `v2024.1` | +| `24.10.5` | `master` or newer tags | +| `25.12.0` | `master` or newer tags | + +--- + +## What happens when you run it + +The workflow has three jobs that run in sequence: + +### 1. Resolve matrix (~5 seconds, GitHub-hosted) + +Parses the `duts` input and builds a job matrix so each device runs in +parallel. For example, `"belkin_rt3200,librerouter"` becomes two independent +build jobs. + +### 2. Build (~20–25 min per device, GitHub-hosted) + +For each device, two Docker containers run back to back: + +**Step 1 — OpenWrt SDK** +Downloads `ghcr.io/openwrt/sdk:--v` and +compiles the lime-packages listed below from source, using the exact git ref +you specified in `lime_ref`. This is the slow step. + +Packages compiled: + +- `lime-system` +- `lime-proto-babeld` +- `lime-proto-batadv` +- `lime-proto-anygw` +- `lime-hwd-openwrt-wan` +- `lime-app` +- `shared-state` + `shared-state-babeld_hosts` + `shared-state-bat_hosts` + `shared-state-nodes_and_links` +- `babeld-auto-gw-mode` +- anything you add via `extra_packages` + +**Step 2 — OpenWrt ImageBuilder** +Downloads `ghcr.io/openwrt/imagebuilder:--v` +and assembles the compiled `.ipk` packages into a complete firmware image +(`.bin` or `.itb`). This step takes ~2–3 minutes. + +The firmware is uploaded as a GitHub Actions artifact named +`firmware---` and kept for 7 days. + +### 3. Flash and test (lab self-hosted runner, `testbed-fcefyn`) + +Runs on the physical T430 machine in the FCEFYN lab. + +1. Downloads the firmware artifact from step 2 +2. Reserves the target device via [labgrid](https://labgrid.readthedocs.io) + (waits if the device is busy) +3. Loads the firmware onto the device +4. Runs the [libremesh-tests](https://github.com/fcefyn-testbed/libremesh-tests) + test suite with pytest +5. Releases the device when done (even if tests fail) + +--- + +## Examples + +**Test the latest stable lime-packages on the Belkin RT3200:** +``` +duts: belkin_rt3200 +lime_ref: v2024.1 +openwrt_version: 23.05.5 +``` + +**Test a feature branch on all devices:** +``` +duts: all +lime_ref: my-feature-branch +openwrt_version: 24.10.5 +``` + +**Add a package and remove another:** +``` +duts: belkin_rt3200 +lime_ref: v2024.1 +extra_packages: luci-app-dawn -lime-proto-batadv +``` + +**Inject a custom network config:** +``` +duts: belkin_rt3200 +lime_ref: v2024.1 +config_file: firmware/configs/belkin_rt3200.conf +``` + +--- + +## Testing the build locally with `act` + +[act](https://github.com/nektos/act) lets you run the build job on your own +machine without pushing to GitHub. Only the `build` job works locally — the +`flash_and_test` job requires physical lab hardware and is automatically +skipped. + +```bash +act workflow_dispatch \ + --workflows .github/workflows/build-and-test-libremesh.yml \ + --job build \ + --input duts="belkin_rt3200" \ + --input lime_ref="v2024.1" \ + --input openwrt_version="23.05.5" \ + --input config_file="" \ + --input extra_packages="" \ + -P ubuntu-latest=catthehacker/ubuntu:act-22.04 \ + --artifact-server-path /tmp/act-artifacts +``` + +The firmware is saved to `/tmp/act-artifacts` and to `./images/` in the repo +root (not committed). diff --git a/mkdocs.yml b/mkdocs.yml index c8da772..8957b95 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -87,6 +87,7 @@ nav: - Routine operations: operar/lab-routine-operations.md - Adding a DUT: operar/dut-onboarding.md - Build firmware: operar/build-firmware-manual.md + - "CI: Build & Test": operar/ci-build-and-test.md - Wake-on-LAN: operar/wake-on-lan-setup.md - ZeroTier (admin-only): operar/zerotier-remote-access.md - Component configuration: From e370db1b635f4971ec206b24675ac52037de45e4 Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:37:32 -0300 Subject: [PATCH 02/18] docs(ci-runner): add workflows table and troubleshooting section --- docs/configuracion/ci-runner.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/docs/configuracion/ci-runner.md b/docs/configuracion/ci-runner.md index ff3ae53..5e9813b 100644 --- a/docs/configuracion/ci-runner.md +++ b/docs/configuracion/ci-runner.md @@ -73,3 +73,28 @@ To move the runner from one repo to another (or user to org): ## 6. Ownership transfer (if needed in near future) When the repo transfers to an org, attached runners move with it. The systemd service name may still reference the old owner; this should not affect operation. + +--- + +## 7. Workflows that use this runner + +The following workflows in `fcefyn-testbed/fcefyn_testbed_utils` target this runner with `runs-on: [self-hosted, testbed-fcefyn]`: + +| Workflow | Trigger | What it does on the runner | +|----------|---------|---------------------------| +| `build-and-test-libremesh.yml` | Manual (`workflow_dispatch`) | Downloads firmware artifact built on GitHub-hosted runners, reserves the DUT via labgrid, loads firmware, and runs libremesh-tests with pytest. | + +The `build` job of that workflow runs on GitHub-hosted runners (no lab hardware needed). Only the `flash_and_test` job lands on this runner. + +See [CI: Build & Test](../operar/ci-build-and-test.md) for full usage instructions. + +--- + +## 8. Troubleshooting + +| Symptom | Cause | Fix | +|---------|-------|-----| +| Runner shows **Offline** in GitHub | Service stopped | `sudo systemctl restart actions.runner.*` | +| `flash_and_test` job queued but never starts | Runner offline or label mismatch | Check runner labels include `testbed-fcefyn` | +| `PermissionError` on labgrid coordinator | Wrong `/etc/labgrid` ownership | See §4 above | +| Job fails at "Reserve DUT" | DUT locked by a previous run | `labgrid-client -p unlock` | From d73f2959ec56de637a52400740a793e4ff9c6bb4 Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:38:10 -0300 Subject: [PATCH 03/18] docs(labgrid): document helper scripts generate_places_yaml, resolve_target, provision_mesh_ip --- docs/operar/labgrid-useful-commands.md | 46 ++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/docs/operar/labgrid-useful-commands.md b/docs/operar/labgrid-useful-commands.md index 644df81..e579050 100644 --- a/docs/operar/labgrid-useful-commands.md +++ b/docs/operar/labgrid-useful-commands.md @@ -134,3 +134,49 @@ ansible-playbook playbook_labgrid.yml -l labgrid-fcefyn (Playbook path and inventory as in [ansible-labgrid](../configuracion/ansible-labgrid.md).) VLAN or switch issues are outside `labgrid-client`; see [Routine operations - DUTs and VLANs](lab-routine-operations.md#duts-and-vlans) and [switch-config](../configuracion/switch-config.md). + +--- + +## Helper scripts + +These scripts live in `scripts/` and complement the labgrid workflow. + +### generate_places_yaml.py + +Generates `places.yaml` for the labgrid coordinator from `labnet.yaml` and a Jinja2 template. Run this after adding or removing DUTs from `labnet.yaml`. + +```bash +# Generate for the default lab (labgrid-fcefyn) +python3 scripts/generate_places_yaml.py + +# Generate for a different lab +python3 scripts/generate_places_yaml.py --lab labgrid-hsn + +# Custom paths +python3 scripts/generate_places_yaml.py \ + --labnet /path/to/labnet.yaml \ + --output ~/labgrid-coordinator/places.yaml +``` + +### resolve_target.py + +Resolves the labgrid target file for a given device name. Useful for debugging which `targets/.yaml` would be used by pytest. + +```bash +python3 scripts/resolve_target.py belkin_rt3200_1 +# Output: targets/linksys_e8450.yaml +``` + +!!! note + This script is optional. When running pytest, `LG_ENV` is resolved automatically from `LG_PLACE`. Use this only for debugging or manual environment setup. + +### provision_mesh_ip.py + +Sets up the per-DUT mesh SSH/control IP (`10.13.200.x`) on `br-lan` via serial. Run once per DUT after flashing. + +```bash +python3 scripts/provision_mesh_ip.py --all # all DUTs +python3 scripts/provision_mesh_ip.py --all --dry-run # verify without applying +``` + +See [duts-config](../configuracion/duts-config.md) for the per-DUT IP table. From e19b31f5b7082afe9f5625ee40199ef37538626b Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:38:55 -0300 Subject: [PATCH 04/18] docs(rack): add CI workflow quick reference section --- docs/operar/rack-cheatsheets.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/docs/operar/rack-cheatsheets.md b/docs/operar/rack-cheatsheets.md index 6ddf164..f27d1f6 100644 --- a/docs/operar/rack-cheatsheets.md +++ b/docs/operar/rack-cheatsheets.md @@ -37,6 +37,24 @@ LibreMesh feeds, `menuconfig`, lime packages, QEMU/vwifi: [build-firmware-manual --- +## CI workflow quick reference + +Trigger a firmware build and test from GitHub Actions without touching the lab manually. + +**Go to:** GitHub → Actions → **Build LibreMesh and Test on DUT** → Run workflow + +| Input | Example | Notes | +|-------|---------|-------| +| `duts` | `belkin_rt3200` or `all` | Comma-separated or `all` | +| `lime_ref` | `v2024.1` | Branch, tag, or commit SHA | +| `openwrt_version` | `23.05.5` | Must match `lime_ref` | +| `extra_packages` | `luci-app-dawn` | Prefix with `-` to remove | +| `config_file` | `firmware/configs/belkin_rt3200.conf` | Optional, injected as `/etc/config/` | + +The `flash_and_test` job runs on the **T430Runner** (`testbed-fcefyn`). Full guide: [CI: Build & Test](ci-build-and-test.md). + +--- + ## SSH: Oracle VPS and OpenWrt gateway From the **orchestration host** (same machine as Labgrid). Requires `~/.ssh/config` per repo templates. From ce951105037bcdf7a47199ebdcc4b054def1cffb Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:40:35 -0300 Subject: [PATCH 05/18] docs(firmware): add pre-built firmwares directory overview --- docs/operar/build-firmware-manual.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/docs/operar/build-firmware-manual.md b/docs/operar/build-firmware-manual.md index e829ab3..016ebb3 100644 --- a/docs/operar/build-firmware-manual.md +++ b/docs/operar/build-firmware-manual.md @@ -182,6 +182,34 @@ Firmware appears under `bin/` as in standard OpenWrt. --- +## Pre-built firmwares (`firmwares/`) + +The `firmwares/` directory in this repo contains pre-built images ready to use in the lab without compiling. Useful for quick tests or when the build pipeline is unavailable. + +``` +firmwares/ +├── belkin_rt3200/ +│ ├── libremesh/ ← LibreMesh images (initramfs + sysupgrade) +│ └── openwrt/ ← Stock OpenWrt images +├── bananapi_bpi-r4/ +├── librerouter_librerouter-v1/ +├── openwrt_one/ +├── tplink-wdr3500/ +└── qemu/ +``` + +Each device folder has: + +| File pattern | Purpose | +|---|---| +| `*initramfs*.bin` / `*initramfs*.itb` | TFTP boot — loads into RAM, flash untouched | +| `*sysupgrade*.bin` / `*sysupgrade*.itb` | Flash write via `sysupgrade` | +| `*sdcard*.img.gz` | SD card image (Banana Pi R4 only) | + +To use a pre-built initramfs in tests, set `LG_IMAGE` to the file path. See [Running tests](lab-running-tests.md). + +--- + ## Automatic builds via PR (lime-packages fork) The **fcefyn-testbed/lime-packages** fork includes `.github/workflows/build-firmware.yml`: on pull requests (and manual dispatch), GitHub Actions builds a **local lime_packages feed** with the OpenWrt SDK, then one **firmware image per row** in `.github/ci/targets.yml` using ImageBuilder. Successful runs upload **`firmware-.*`** and **`lime-feed-`** artifacts. From 317a120765509cd90eeb3a23d4e5687b3c15b793 Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:41:46 -0300 Subject: [PATCH 06/18] docs(som): add CI firmware build workflow to automation section --- docs/operar/system-operation-manual.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/operar/system-operation-manual.md b/docs/operar/system-operation-manual.md index 445630c..b516b73 100644 --- a/docs/operar/system-operation-manual.md +++ b/docs/operar/system-operation-manual.md @@ -78,6 +78,7 @@ Automation of necessary operations is achieved using: - **Ansible:** `ansible/playbook_labgrid.yml` (exporter, places, users). See [ansible-labgrid](../configuracion/ansible-labgrid.md). - **Dynamic VLAN:** `labgrid-switch-abstraction` (used by libremesh-tests in CI); manual ops: `switch-vlan`. Details in [Lab architecture](../diseno/lab-architecture.md). +- **CI firmware build:** `build-and-test-libremesh.yml` compiles lime-packages from a specific git ref using the OpenWrt SDK and ImageBuilder on GitHub-hosted runners, then runs libremesh-tests on the physical DUTs via the `testbed-fcefyn` self-hosted runner. Triggered manually. See [CI: Build & Test](ci-build-and-test.md). --- From b32008b27a16f69683ca2eaefe8cc3e40f87438b Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:42:13 -0300 Subject: [PATCH 07/18] docs(rack): add switch and power scripts reference section --- docs/operar/rack-cheatsheets.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/docs/operar/rack-cheatsheets.md b/docs/operar/rack-cheatsheets.md index f27d1f6..c621333 100644 --- a/docs/operar/rack-cheatsheets.md +++ b/docs/operar/rack-cheatsheets.md @@ -66,6 +66,25 @@ From the **orchestration host** (same machine as Labgrid). Requires `~/.ssh/conf --- +## Switch and power scripts + +| Script | Purpose | Example | +|--------|---------|---------| +| `scripts/switch/poe_switch_control.py` | Control PoE ports on the TP-Link switch | `python3 scripts/switch/poe_switch_control.py off 1` | +| `scripts/switch/dut_gateway.py` | Update default gateway on DUTs via SSH after VLAN change | `python3 scripts/switch/dut_gateway.py --dut belkin_rt3200` | + +```bash +# Power cycle OpenWRT One (PoE port 1) +python3 scripts/switch/poe_switch_control.py off 1 +sleep 3 +python3 scripts/switch/poe_switch_control.py on 1 + +# Set SWITCH_PASSWORD env var or use ~/.config/switch.conf +export SWITCH_PASSWORD=yourpassword +``` + +--- + ## TP-Link SG2016P switch | Field | Value | From a66039e98183b8330f7fe7ada76ecf1b97cfa4c8 Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:42:48 -0300 Subject: [PATCH 08/18] docs(running-tests): add section on using CI-built firmware artifacts --- docs/operar/lab-running-tests.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/docs/operar/lab-running-tests.md b/docs/operar/lab-running-tests.md index 435eef5..0944916 100644 --- a/docs/operar/lab-running-tests.md +++ b/docs/operar/lab-running-tests.md @@ -69,6 +69,24 @@ The test suite locates `labnet.yaml` via `LABNET_PATH`, `OPENWRT_TESTS_DIR/labne --- +## Using CI-built firmware in tests + +The `build-and-test-libremesh.yml` workflow builds a firmware artifact and runs `flash_and_test` automatically. If you want to run the tests manually with a firmware built by CI: + +1. Go to GitHub → Actions → **Build LibreMesh and Test on DUT** → select a run +2. Download the artifact `firmware---` +3. Extract the `.bin` or `.itb` file and point `LG_IMAGE` to it: + +```bash +LG_PLACE=labgrid-fcefyn-belkin_rt3200_1 \ +LG_IMAGE=/path/to/openwrt-23.05.5-mediatek-mt7622-linksys_e8450-squashfs-sysupgrade.bin \ +uv run pytest tests/ -v +``` + +Artifacts are kept for 7 days. For automated end-to-end runs, trigger the workflow directly — it handles the full build → flash → test pipeline without manual steps. + +--- + ## Remote coordinator access (openwrt-tests) ```bash From e62534f850b7d128548bc6bf4ec7e0b33077588c Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:43:22 -0300 Subject: [PATCH 09/18] docs(developer): add section on triggering CI workflow from GitHub --- docs/operar/developer-remote-access.md | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/docs/operar/developer-remote-access.md b/docs/operar/developer-remote-access.md index 19802c9..ab7da17 100644 --- a/docs/operar/developer-remote-access.md +++ b/docs/operar/developer-remote-access.md @@ -313,7 +313,24 @@ ZeroTier is **not** required for developer access. It is used only by lab admins --- -## 10. Reference +## 10. Triggering the CI workflow as a developer + +Instead of running tests manually, you can trigger the full build + test pipeline from GitHub: + +1. Go to [fcefyn-testbed/fcefyn_testbed_utils → Actions → Build LibreMesh and Test on DUT](https://github.com/fcefyn-testbed/fcefyn_testbed_utils/actions/workflows/build-and-test-libremesh.yml) +2. Click **Run workflow** and fill in: + - `duts`: the device you want to test + - `lime_ref`: the lime-packages branch, tag, or commit you want to validate + - `openwrt_version`: must be compatible with `lime_ref` +3. The build runs on GitHub's servers (~20 min). The `flash_and_test` job then runs automatically on the lab hardware. + +This does **not** require SSH access to the lab. The only requirement is having a GitHub account with access to the repository. + +For full workflow documentation: [CI: Build & Test](ci-build-and-test.md). + +--- + +## 11. Reference - [Running tests (host-side)](lab-running-tests.md) - [SSH access to DUTs](dut-ssh-access.md) - VLAN lifecycle and mesh SSH From 003008b1eaa3ffa200ea01f3863325a0eeda7490 Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:45:17 -0300 Subject: [PATCH 10/18] docs(diseno): add lime-packages CI firmware build flow page --- docs/diseno/lime-packages-ci-flow.md | 72 ++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) create mode 100644 docs/diseno/lime-packages-ci-flow.md diff --git a/docs/diseno/lime-packages-ci-flow.md b/docs/diseno/lime-packages-ci-flow.md new file mode 100644 index 0000000..4a4f307 --- /dev/null +++ b/docs/diseno/lime-packages-ci-flow.md @@ -0,0 +1,72 @@ +# lime-packages CI: firmware build + +The [fcefyn-testbed/lime-packages](https://github.com/fcefyn-testbed/lime-packages) fork adds a +CI pipeline that builds firmware images automatically on pull requests and manual dispatches, +without requiring a full OpenWrt buildroot. + +--- + +## Overview + +``` +PR or manual dispatch + ↓ + OpenWrt SDK (Docker) + compile lime-packages feed + ↓ + OpenWrt ImageBuilder (Docker) + assemble firmware per target + ↓ + Upload artifacts: + firmware-.* + lime-feed- +``` + +The SDK and ImageBuilder images come from `ghcr.io/openwrt/sdk` and +`ghcr.io/openwrt/imagebuilder`, pinned to a specific OpenWrt version. + +--- + +## Workflow file + +`.github/workflows/build-firmware.yml` in the fork. Triggered on: + +- Pull requests to the default branch +- Manual dispatch (`workflow_dispatch`) + +--- + +## Target matrix + +Build targets are defined in `.github/ci/targets.yml`. Each row specifies: + +| Field | Example | Description | +|-------|---------|-------------| +| `target` | `mediatek` | OpenWrt target architecture | +| `subtarget` | `mt7622` | OpenWrt subtarget | +| `profile` | `linksys_e8450-ubi` | Device profile | +| `openwrt_version` | `23.05.5` | OpenWrt release | + +One firmware image is produced per row, in parallel. + +--- + +## Artifacts + +Successful runs upload: + +- **`firmware-.*`** — firmware image (`.bin` or `.itb`) with lime-packages included +- **`lime-feed-`** — compiled `.ipk` feed for the target architecture, usable by an + ImageBuilder directly + +Artifacts are kept for a configurable number of days (default: 7). + +--- + +## Relationship to this repo + +The firmware artifacts from the lime-packages fork CI feed into the +`flash_and_test` job of [build-and-test-libremesh.yml](../operar/ci-build-and-test.md) +in this repo. Both pipelines use the same SDK + ImageBuilder approach. + +See also: [lime-packages CI: hardware tests](lime-packages-test-flow.md). From 05e4489721aabf10655aedd081f439a91ee6eaa1 Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:46:16 -0300 Subject: [PATCH 11/18] docs(diseno): add lime-packages CI hardware test flow page --- docs/diseno/lime-packages-test-flow.md | 66 ++++++++++++++++++++++++++ 1 file changed, 66 insertions(+) create mode 100644 docs/diseno/lime-packages-test-flow.md diff --git a/docs/diseno/lime-packages-test-flow.md b/docs/diseno/lime-packages-test-flow.md new file mode 100644 index 0000000..c00adc4 --- /dev/null +++ b/docs/diseno/lime-packages-test-flow.md @@ -0,0 +1,66 @@ +# lime-packages CI: hardware tests + +After the firmware build in [lime-packages CI: firmware build](lime-packages-ci-flow.md), +a downstream job runs the [libremesh-tests](https://github.com/fcefyn-testbed/libremesh-tests) +suite on physical hardware in the FCEFYN lab. + +--- + +## Flow + +``` +lime-packages PR build succeeds + ↓ + Download firmware artifact + ↓ + self-hosted runner (testbed-fcefyn) + Reserve DUT via labgrid + ↓ + Load firmware via TFTP (RAM boot) + No flash written + ↓ + pytest (libremesh-tests) + ↓ + Release DUT + Report results on PR +``` + +--- + +## Runner + +Tests run on the `testbed-fcefyn` self-hosted runner (the T430 machine in the FCEFYN lab). +This runner is shared with the `flash_and_test` job in +[build-and-test-libremesh.yml](../operar/ci-build-and-test.md). + +Setup: [GitHub Actions self-hosted runner](../configuracion/ci-runner.md). + +--- + +## DUT reservation + +The job uses `labgrid-client reserve --wait` so it queues until the requested device is +free. If multiple CI jobs run at the same time (e.g. two open PRs), the second job waits +rather than failing. + +--- + +## Test suite + +The [libremesh-tests](https://github.com/fcefyn-testbed/libremesh-tests) suite validates: + +- Device boots and reaches a shell +- LibreMesh packages are installed and services start +- Basic mesh configuration is applied +- Network interfaces are configured correctly + +Tests use the `LG_IMAGE` firmware loaded via TFTP — the device boots into RAM, runs the +test fixture, and is powered off at teardown. The flash is never written. + +--- + +## Relationship to this repo + +The `flash_and_test` job in `build-and-test-libremesh.yml` follows the same pattern but +uses firmware built by the SDK + ImageBuilder pipeline in this repo instead of the +lime-packages fork. See [CI: Build & Test](../operar/ci-build-and-test.md). From 1247a952e777c0c180de2a966f2a6883e18cee6e Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:46:45 -0300 Subject: [PATCH 12/18] docs(nav): add lime-packages CI flow pages to mkdocs nav --- mkdocs.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mkdocs.yml b/mkdocs.yml index 8957b95..34d98cd 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -114,5 +114,7 @@ nav: - openwrt-tests onboarding: diseno/openwrt-tests-onboarding.md - Contributing a new lab: diseno/new-lab-contribution.md - openwrt-tests CI flow: diseno/openwrt-tests-ci-flow.md + - lime-packages CI build: diseno/lime-packages-ci-flow.md + - lime-packages CI tests: diseno/lime-packages-test-flow.md - Virtual mesh: diseno/virtual-mesh.md - Demos: demos.md From b1ad93bdda6c64b313a30e62d48ddc22da4318b2 Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:47:37 -0300 Subject: [PATCH 13/18] docs(index): add quick actions table to home page --- docs/index.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/docs/index.md b/docs/index.md index fe455f5..2488f9f 100644 --- a/docs/index.md +++ b/docs/index.md @@ -21,3 +21,16 @@ Relationship between orchestration host, switch, gateway, DUTs, power, and seria ![High-level diagram of the testbed and main components](img/diagrams/general-design-overview.png) The design builds on the **remote lab** model from [openwrt-tests](https://github.com/aparcar/openwrt-tests), but scope is not limited to **adding devices** to that network. It also **reuses and extends** the approach with **local infrastructure**, along the same axes (orchestration, network, power, serial) with a focus on **LibreMesh** testing. + +--- + +## Quick actions + +| I want to… | Go to | +|---|---| +| Run tests on lab hardware from my machine | [Developer quickstart](operar/developer-remote-access.md) | +| Build a LibreMesh firmware and test it automatically | [CI: Build & Test](operar/ci-build-and-test.md) | +| SSH into a DUT | [SSH access to DUTs](operar/dut-ssh-access.md) | +| Check the rack layout and device IPs | [Rack quick reference](operar/rack-cheatsheets.md) | +| Build firmware manually | [Build firmware](operar/build-firmware-manual.md) | +| Add a new device to the lab | [Adding a DUT](operar/dut-onboarding.md) | From 08d1fd3ca6a604b591967c9df2fb0e2aa79b1a91 Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:48:47 -0300 Subject: [PATCH 14/18] docs(integration): add CI workflow entries to design reading guide --- docs/diseno/integration-overview.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/diseno/integration-overview.md b/docs/diseno/integration-overview.md index 9ce132c..2a791e8 100644 --- a/docs/diseno/integration-overview.md +++ b/docs/diseno/integration-overview.md @@ -163,3 +163,6 @@ sequenceDiagram | Set up the WireGuard tunnel and contribute hardware to openwrt-tests | [openwrt-tests onboarding](openwrt-tests-onboarding.md) | | VLAN design, `switch-vlan` CLI, switch configuration | [Lab architecture](lab-architecture.md) | | Virtual mesh tests with QEMU and vwifi | [Virtual mesh](virtual-mesh.md) | +| Build and test lime-packages from a PR via CI | [CI: Build & Test](../operar/ci-build-and-test.md) | +| Understand the lime-packages fork CI build pipeline | [lime-packages CI build](lime-packages-ci-flow.md) | +| Understand the lime-packages fork CI test pipeline | [lime-packages CI tests](lime-packages-test-flow.md) | From 4675dc0dcf38b71fe4c5ff497f4420ca878ce25e Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:49:15 -0300 Subject: [PATCH 15/18] docs(demos): add index table to video demos page --- docs/demos.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/demos.md b/docs/demos.md index 276a5e3..5c55afd 100644 --- a/docs/demos.md +++ b/docs/demos.md @@ -2,6 +2,11 @@ This section collects demo videos showing various operations on the testbed. +| Demo | Topic | +|------|-------| +| [Remote access to the HIL testbed](#remote-access-to-the-hil-testbed) | Administrator access and lab infrastructure recovery | +| [Developer remote access through Labgrid](#developer-remote-access-through-labgrid) | Developer SSH + multi-node mesh test run | + --- ## Remote access to the HIL testbed From 5383aa6770d042ceb1486bd77710dae67d8d0098 Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:49:57 -0300 Subject: [PATCH 16/18] docs(new-lab): add CI build and test link to related pages --- docs/diseno/new-lab-contribution.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/diseno/new-lab-contribution.md b/docs/diseno/new-lab-contribution.md index b79f5bc..ac1a0b3 100644 --- a/docs/diseno/new-lab-contribution.md +++ b/docs/diseno/new-lab-contribution.md @@ -125,3 +125,4 @@ A lab can contribute to both paths at once: the infrastructure is shared. Scenar - [Switch](../configuracion/switch-config.md) - VLAN layout, `switch.conf` multi-user setup, `switch-vlan` invocation. - [Lab architecture](lab-architecture.md) - shared coordinator, VLAN scheduling, locking. - [CI runner](../configuracion/ci-runner.md) - self-hosted runner setup (Scenario B). +- [CI: Build & Test](../operar/ci-build-and-test.md) - automated firmware build and test workflow using the self-hosted runner. From 824eb06956ce2cad024bfd636a1069749400ea29 Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 21:50:40 -0300 Subject: [PATCH 17/18] docs(routine): add CI runner verification section --- docs/operar/lab-routine-operations.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/docs/operar/lab-routine-operations.md b/docs/operar/lab-routine-operations.md index 864c7ec..30a8c96 100644 --- a/docs/operar/lab-routine-operations.md +++ b/docs/operar/lab-routine-operations.md @@ -136,3 +136,26 @@ systemctl status arduino-relay-daemon # Restart exporter sudo systemctl restart labgrid-exporter ``` + +--- + +## Verify CI runner {: #verify-ci-runner } + +```bash +# Check runner service +sudo systemctl status actions.runner.* + +# Restart runner +sudo systemctl restart actions.runner.* +``` + +The runner should show **Idle** in GitHub → Settings → Actions → Runners. +If a `flash_and_test` job is queued but never starts, check that the runner is online +and that the DUT is not locked by a previous run: + +```bash +labgrid-client reservations # check for stale reservations +labgrid-client -p unlock # release if needed +``` + +See [CI runner](../configuracion/ci-runner.md) for full setup and troubleshooting. From 4fbbe05a11c97768ca2f1b15ae86cf0afa686172 Mon Sep 17 00:00:00 2001 From: Constanza Date: Tue, 28 Apr 2026 22:03:22 -0300 Subject: [PATCH 18/18] docs(operar): add virtual mesh local setup guide from vms/README --- docs/operar/virtual-mesh-local.md | 111 ++++++++++++++++++++++++++++++ mkdocs.yml | 1 + 2 files changed, 112 insertions(+) create mode 100644 docs/operar/virtual-mesh-local.md diff --git a/docs/operar/virtual-mesh-local.md b/docs/operar/virtual-mesh-local.md new file mode 100644 index 0000000..d1ccfa4 --- /dev/null +++ b/docs/operar/virtual-mesh-local.md @@ -0,0 +1,111 @@ +# Running the virtual mesh locally + +How to launch QEMU-based virtual mesh nodes on a developer machine using +[vwifi](https://github.com/sysprog21/vwifi) for WiFi simulation, and run the +libremesh-tests suite against them without physical hardware. + +--- + +## 1. Check the source image + +The build source image lives in: + +``` +firmwares/qemu/libremesh/lime--viwifi-x86-64-generic-ext4-combined.img +``` + +The `vms/node*.img` files are **working copies** created by QEMU at runtime — +they are not the source image and should not be used as a base. + +```bash +# Check size (should be ~121 MB) and date +ls -lh firmwares/qemu/libremesh/*.img + +# Confirm it is a bootable x86 disk +file firmwares/qemu/libremesh/lime-*.img +# Expected: "DOS/MBR boot sector" + +# Check the LibreMesh version inside the image +strings firmwares/qemu/libremesh/lime-*.img | grep -i 'DISTRIB_RELEASE\|lime_release' + +# Hash for comparison with previous builds +md5sum firmwares/qemu/libremesh/lime-*.img +``` + +--- + +## 2. Launch the VMs + +```bash +# From the repo root +VIRTUAL_MESH_IMAGE=firmwares/qemu/libremesh/lime-*.img ./vms/launch_debug_vms.sh +``` + +Available environment variables: + +| Variable | Default | Description | +|---|---|---| +| `VIRTUAL_MESH_IMAGE` | — | Path to the image (required) | +| `VIRTUAL_MESH_NODES` | `2` | Number of VMs to start | +| `VIRTUAL_MESH_BOOT_TIMEOUT` | `120` | Seconds to wait for boot | +| `VIRTUAL_MESH_CONVERGENCE_WAIT` | `60` | Seconds to wait for mesh convergence | +| `VIRTUAL_MESH_SKIP_VWIFI` | `0` | Skip vwifi setup (useful for quick debugging) | + +### SSH into the nodes + +```bash +ssh -o StrictHostKeyChecking=no -p 2222 root@127.0.0.1 # VM 1 +ssh -o StrictHostKeyChecking=no -p 2223 root@127.0.0.1 # VM 2 +``` + +--- + +## 3. Run the tests + +With the VMs already running: + +```bash +# All tests +pytest tests/mesh/ -v + +# Node health only (interfaces, services, UCI, kernel) +pytest tests/mesh/test_mesh_node_basic.py -v + +# Network connectivity (ping bat0, unique IPs, inter-node visibility) +pytest tests/mesh/test_mesh_basic.py -v + +# batman-adv (TQ, originators, symmetry, statistics) +pytest tests/mesh/test_mesh_batman.py -v +``` + +With 3 nodes: + +```bash +VIRTUAL_MESH_NODES=3 VIRTUAL_MESH_IMAGE=firmwares/qemu/libremesh/lime-*.img \ + ./vms/launch_debug_vms.sh & + +# Wait for "Debug session ready", then: +VIRTUAL_MESH_NODES=3 pytest tests/mesh/ -v +``` + +--- + +## 4. Useful commands inside a node + +```bash +batctl n # direct batman-adv neighbours +batctl o # originator table (all mesh routes) +batctl if # batman slave interfaces +batctl s # traffic statistics +ip addr show bat0 # node IP on the mesh +logread | grep lime-config # verify lime-config ran +uci show vwifi # vwifi client config +``` + +--- + +## 5. Related pages + +- [Virtual mesh design](../diseno/virtual-mesh.md) — architecture, CI integration, fixture variables +- [Build firmware](build-firmware-manual.md) — how to build the QEMU vwifi image +- [Running tests](lab-running-tests.md) — physical DUT tests diff --git a/mkdocs.yml b/mkdocs.yml index 34d98cd..59a5080 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -88,6 +88,7 @@ nav: - Adding a DUT: operar/dut-onboarding.md - Build firmware: operar/build-firmware-manual.md - "CI: Build & Test": operar/ci-build-and-test.md + - Virtual mesh (local): operar/virtual-mesh-local.md - Wake-on-LAN: operar/wake-on-lan-setup.md - ZeroTier (admin-only): operar/zerotier-remote-access.md - Component configuration: