chore: sync develop → main - #90
Merged
Merged
Conversation
Adds a scheduled workflow that uses LIME_PACKAGES_TOKEN to download JUnit report.xml artifacts from recent fcefyn-testbed/lime-packages CI runs and commits them under docs/ci-results/results/. Fixes the 404 on the dashboard "Report ↗" links without requiring any change in lime-packages (the PR to publish from there will not be merged upstream).
ci(dashboard): pull lime-packages test reports from another side
develop branch protection blocks direct push from the bot (signed commits, PR-only, no push permission). Switch the workflow to use peter-evans/create-pull-request with sign-commits: true (commits get signed via GitHub API on the bot's behalf) and auto-merge the generated PR so the collection stays hands-off.
…r-flow ci(dashboard): collect-lime-results opens auto-merged PR
TESTBED_UTILS_TOKEN only exists in lime-packages (writing into this repo). The collect workflow runs the other direction, so use the auto-provided GITHUB_TOKEN, which already has pull-requests:write and contents:write thanks to the permissions block.
…oken-2 fix(ci): use default GITHUB_TOKEN for collect-lime-results PR
The org disables "write" workflow permissions, so the default GITHUB_TOKEN cannot open PRs or call gh pr merge --auto. Use the BOT_PR_TOKEN PAT (Contents + Pull Requests: write on this repo) for the PR-creating and merging steps. Checkout keeps GITHUB_TOKEN because it only needs to read the working tree.
…ot-pat fix(ci): use BOT_PR_TOKEN for collect-lime-results PR flow
The Report ↗ links were 404-ing because PAGES_BASE pointed at fcefyn-testbed.github.io/lime-packages/, but the report.xml files are published under this repo's Pages (fcefyn-testbed.github.io/fcefyn_testbed_utils/ci-results/results/).
The publish pipeline now lives entirely in fcefyn_testbed_utils via the collect-lime-results.yml workflow that pulls report.xml artifacts from lime-packages instead of relying on a workflow change there. Replaces the old TESTBED_UTILS_TOKEN-based push diagram with the new pull sequence, lists artifact→path mappings, and documents required secrets (LIME_PACKAGES_TOKEN, BOT_PR_TOKEN) plus the repo-level toggles.
PAGES_BASE in dashboard.html points to this repo's Pages (fcefyn-testbed.github.io/fcefyn_testbed_utils/...), not lime-packages. Also remove the stale "publish-results job in build-firmware.yml" reference; reports now come from collect-lime-results.yml on this side.
Replace the TESTBED_UTILS_TOKEN rotation procedure with the two fine-grained PATs the new pipeline uses (LIME_PACKAGES_TOKEN for reading lime-packages artifacts, BOT_PR_TOKEN for opening the auto-merged PR). Expand the troubleshooting table with every failure mode we hit while wiring this up: org-level Actions toggle, repo-level auto-merge toggle, develop branch protection signed-commits gotcha, and the PAGES_BASE host mismatch that surfaced as 404 on Report links.
Place names use bananapi_bpi-r4 / openwrt_one (no _1 suffix), single Belkin entries are 24.10.6-only on the lab, and 25.12.2 currently runs only on QEMU. Add the physical mesh / mesh-pairs rows that the dashboard now surfaces.
Adds the lab-overview.json dashboard to the dashboards table and bumps the Orchestrator Host panel count to ~40 to reflect the new Lab Services and WireGuard sections.
Adds rows to the Orchestrator Host sections table describing the new "Lab Services" panels (labgrid-exporter, pdudaemon, ser2net via node_systemd_unit_state) and the "WireGuard (wg0)" tunnel panels (bandwidth, packets, errors, link status).
So the inventory of dashboard JSONs in the repo matches what Ansible actually provisions.
Per-section breakdown of lab-overview.json so the doc matches what Grafana actually shows: scrape up cards, uptime/CPU/RAM stat panels, consolidated device table, and the firmware/target table sourced from node_openwrt_info labels.
The duts-node.json dashboard has a per-interface node_network_*_errs_total / *_drop_total rate panel. Was missing from the section table.
The bpi_r4_1 / openwrt_one_1 place names never existed in the labgrid config — the actual places are bananapi_bpi-r4 and openwrt_one. Also drop the 25.12.2 physical rows (that release is QEMU-only on the lab right now) and add the physical mesh / mesh-pair entries that prepare_matrix.sh emits. The "Adding a new device" steps now point at the pull-based collect workflow.
\`bpi\` no longer matches anything — the place is bananapi_bpi-r4. Adds \`bananapi\` and \`openwrt\` as more useful examples that match the real labgrid place names.
Central to the collect-lime-results.yml pipeline, worth defining alongside the other CI/Labgrid terms.
Both metrics are referenced by the new Grafana panels (Lab Overview firmware table, Orchestrator Host Lab Services section) and worth calling out so the dashboard descriptions are self-contained.
Lists the two PATs the pipeline actually uses (LIME_PACKAGES_TOKEN, BOT_PR_TOKEN) so a reader skimming the glossary can find what secret does what without digging into publishing.md.
Mention that the same test-results-* artifacts surface on the CI dashboard via collect-lime-results.yml, so a reader debugging a failure knows where to find the post-hoc report view without digging through Actions artifacts.
Section 7 used to describe a publish-results job inside lime-packages that pushed report.xml into this repo via TESTBED_UTILS_TOKEN. That job never landed upstream. The replacement describes the inverse flow that ships from fcefyn_testbed_utils today: collect-lime-results.yml fetches artifacts, opens an auto-merged PR with BOT_PR_TOKEN, and pages.yml deploys. Also updates the sequence diagram.
…etail Replace the stale publish-results reference with the actual workflow that pulls report.xml into the dashboard and include the manual trigger instructions, so an operator hitting this case can act on it.
…hestrator Panel stat con 100 * node_nf_conntrack_entries / node_nf_conntrack_entries_limit. Relevante en el orchestrator porque mantiene autossh + SSH-proxy por DUT más los flujos de Prometheus/Grafana
Panel stat por DUT que lee node_load1, con thresholds en 1 y 2. Sirve para detectar DUTs trabajando bajo presión aunque el % CPU instantáneo se vea bajo, complementando el panel de CPU busy ya existente.
Panel stat por DUT que muestra el % de swap en uso: 100 * (SwapTotal - SwapFree) / max(SwapTotal, 1).
Sirve para ver de un vistazo si algún DUT está viendo tráfico inesperado durante tests o si quedó algo silencioso.
8 reglas para provisionar en Grafana (formato Grafana managed alerting v1): - fcefyn-target-down: target Prometheus inalcanzable > 2 min - fcefyn-high-cpu: CPU del orchestrator > 85% por 5 min - fcefyn-high-memory: RAM > 90% por 5 min - fcefyn-low-disk: filesystem raíz < 10% libre - fcefyn-tunnel-down: autossh DUT tunnel caído > 3 min - fcefyn-systemd-failed: alguna systemd unit en estado failed - fcefyn-network-errors: errores rx/tx > 0 sostenidos por 5 min se traen al flujo principal para que el Ansible role los provisione junto con los dashboards.
Signed-off-by: Franco <franco.riba@mi.unc.edu.ar>
Signed-off-by: Franco <franco.riba@mi.unc.edu.ar>
* docs(ci): add build-and-test-libremesh workflow documentation * docs(ci-runner): add workflows table and troubleshooting section * docs(labgrid): document helper scripts generate_places_yaml, resolve_target, provision_mesh_ip * docs(rack): add CI workflow quick reference section * docs(firmware): add pre-built firmwares directory overview * docs(som): add CI firmware build workflow to automation section * docs(rack): add switch and power scripts reference section * docs(running-tests): add section on using CI-built firmware artifacts * docs(developer): add section on triggering CI workflow from GitHub * docs(diseno): add lime-packages CI firmware build flow page * docs(diseno): add lime-packages CI hardware test flow page * docs(nav): add lime-packages CI flow pages to mkdocs nav * docs(index): add quick actions table to home page * docs(integration): add CI workflow entries to design reading guide * docs(demos): add index table to video demos page * docs(new-lab): add CI build and test link to related pages * docs(routine): add CI runner verification section * docs(operar): add virtual mesh local setup guide from vms/README
…os (#89) * docs(ci-results): rewrite publishing.md for pull-based flow The publish pipeline now lives entirely in fcefyn_testbed_utils via the collect-lime-results.yml workflow that pulls report.xml artifacts from lime-packages instead of relying on a workflow change there. Replaces the old TESTBED_UTILS_TOKEN-based push diagram with the new pull sequence, lists artifact→path mappings, and documents required secrets (LIME_PACKAGES_TOKEN, BOT_PR_TOKEN) plus the repo-level toggles. * docs(ci-results): fix architecture.md report-source URLs PAGES_BASE in dashboard.html points to this repo's Pages (fcefyn-testbed.github.io/fcefyn_testbed_utils/...), not lime-packages. Also remove the stale "publish-results job in build-firmware.yml" reference; reports now come from collect-lime-results.yml on this side. * docs(ci-results): rewrite maintenance.md for pull pipeline Replace the TESTBED_UTILS_TOKEN rotation procedure with the two fine-grained PATs the new pipeline uses (LIME_PACKAGES_TOKEN for reading lime-packages artifacts, BOT_PR_TOKEN for opening the auto-merged PR). Expand the troubleshooting table with every failure mode we hit while wiring this up: org-level Actions toggle, repo-level auto-merge toggle, develop branch protection signed-commits gotcha, and the PAGES_BASE host mismatch that surfaced as 404 on Report links. * docs(ci-results): fix device coverage table Place names use bananapi_bpi-r4 / openwrt_one (no _1 suffix), single Belkin entries are 24.10.6-only on the lab, and 25.12.2 currently runs only on QEMU. Add the physical mesh / mesh-pairs rows that the dashboard now surfaces. * docs(observability): list Lab Overview dashboard alongside the others Adds the lab-overview.json dashboard to the dashboards table and bumps the Orchestrator Host panel count to ~40 to reflect the new Lab Services and WireGuard sections. * docs(observability): document Lab Services and WireGuard sections Adds rows to the Orchestrator Host sections table describing the new "Lab Services" panels (labgrid-exporter, pdudaemon, ser2net via node_systemd_unit_state) and the "WireGuard (wg0)" tunnel panels (bandwidth, packets, errors, link status). * docs(observability): list lab-overview.json in Key files table So the inventory of dashboard JSONs in the repo matches what Ansible actually provisions. * docs(observability): describe Lab Overview dashboard sections Per-section breakdown of lab-overview.json so the doc matches what Grafana actually shows: scrape up cards, uptime/CPU/RAM stat panels, consolidated device table, and the firmware/target table sourced from node_openwrt_info labels. * docs(observability): note network errors and drops panel on DUTs The duts-node.json dashboard has a per-interface node_network_*_errs_total / *_drop_total rate panel. Was missing from the section table. * docs(ci-results): fix registered devices table The bpi_r4_1 / openwrt_one_1 place names never existed in the labgrid config — the actual places are bananapi_bpi-r4 and openwrt_one. Also drop the 25.12.2 physical rows (that release is QEMU-only on the lab right now) and add the physical mesh / mesh-pair entries that prepare_matrix.sh emits. The "Adding a new device" steps now point at the pull-based collect workflow. * docs(ci-results): update search examples to current place names \`bpi\` no longer matches anything — the place is bananapi_bpi-r4. Adds \`bananapi\` and \`openwrt\` as more useful examples that match the real labgrid place names. * docs(glossary): add auto-merge entry Central to the collect-lime-results.yml pipeline, worth defining alongside the other CI/Labgrid terms. * docs(glossary): define node_openwrt_info and node_systemd_unit_state Both metrics are referenced by the new Grafana panels (Lab Overview firmware table, Orchestrator Host Lab Services section) and worth calling out so the dashboard descriptions are self-contained. * docs(glossary): add PAT entry Lists the two PATs the pipeline actually uses (LIME_PACKAGES_TOKEN, BOT_PR_TOKEN) so a reader skimming the glossary can find what secret does what without digging into publishing.md. * docs(test-flow): cross-link to CI dashboard publishing Mention that the same test-results-* artifacts surface on the CI dashboard via collect-lime-results.yml, so a reader debugging a failure knows where to find the post-hoc report view without digging through Actions artifacts. * docs(test-run-flow): replace publish-results section with pull flow Section 7 used to describe a publish-results job inside lime-packages that pushed report.xml into this repo via TESTBED_UTILS_TOKEN. That job never landed upstream. The replacement describes the inverse flow that ships from fcefyn_testbed_utils today: collect-lime-results.yml fetches artifacts, opens an auto-merged PR with BOT_PR_TOKEN, and pages.yml deploys. Also updates the sequence diagram. * docs(debugging-faq): point to collect-lime-results when card has no detail Replace the stale publish-results reference with the actual workflow that pulls report.xml into the dashboard and include the manual trigger instructions, so an operator hitting this case can act on it. * feat(grafana): agregar panel de offset NTP en dashboard del orchestrator * feat(grafana): agregar panel de uso de conntrack al dashboard del orchestrator Panel stat con 100 * node_nf_conntrack_entries / node_nf_conntrack_entries_limit. Relevante en el orchestrator porque mantiene autossh + SSH-proxy por DUT más los flujos de Prometheus/Grafana * feat(grafana): agregar fila de load1 al dashboard Lab Overview Panel stat por DUT que lee node_load1, con thresholds en 1 y 2. Sirve para detectar DUTs trabajando bajo presión aunque el % CPU instantáneo se vea bajo, complementando el panel de CPU busy ya existente. * feat(grafana): agregar fila de swap al dashboard Lab Overview Panel stat por DUT que muestra el % de swap en uso: 100 * (SwapTotal - SwapFree) / max(SwapTotal, 1). * feat(grafana): agregar fila de tráfico de red al dashboard Lab Overview Sirve para ver de un vistazo si algún DUT está viendo tráfico inesperado durante tests o si quedó algo silencioso. * feat(observability): agregar reglas de alerting de Grafana 8 reglas para provisionar en Grafana (formato Grafana managed alerting v1): - fcefyn-target-down: target Prometheus inalcanzable > 2 min - fcefyn-high-cpu: CPU del orchestrator > 85% por 5 min - fcefyn-high-memory: RAM > 90% por 5 min - fcefyn-low-disk: filesystem raíz < 10% libre - fcefyn-tunnel-down: autossh DUT tunnel caído > 3 min - fcefyn-systemd-failed: alguna systemd unit en estado failed - fcefyn-network-errors: errores rx/tx > 0 sostenidos por 5 min se traen al flujo principal para que el Ansible role los provisione junto con los dashboards. * fix
…les nuev…" This reverts commit 4de3d19.
…esults-flow Revert "docs + grafana: align con pipeline pull-based, alerting, pane…
docs + grafana: paneles, alerting y pipeline pull-based
ccasanueva7
enabled auto-merge
May 24, 2026 17:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Trae a main lo acumulado en develop: PR #65 (docs CI workflow), PR #88 (PAGES_BASE fix), PR #89 (docs + Grafana + alerting), más todo lo del pipeline pull-based de CI dashboard que se fue agregando en los últimos días.
Conflicts esperados
Test plan