|
| 1 | +# v1.3.2 — AppSec fail-open (bug fix) |
| 2 | + |
| 3 | +Patch release. Fixes an availability regression where a |
| 4 | +misconfigured or missing AppSec sidecar would cascade into HTTP 500 |
| 5 | +responses for every host on the panel. |
| 6 | + |
| 7 | +## The bug |
| 8 | + |
| 9 | +Pre-v1.3.2, the panel emitted `apps.crowdsec.appsec_url` in its |
| 10 | +Caddy config whenever `appsec.mode != disabled`. The bouncer plugin |
| 11 | +(`hslatman/caddy-crowdsec-bouncer/appsec`) defaults to |
| 12 | +fail-**closed** behaviour: if the AppSec HTTP endpoint is |
| 13 | +unreachable, the handler returns an error and Caddy serves HTTP 500. |
| 14 | + |
| 15 | +That default is dangerous for argos-edge installations because: |
| 16 | + |
| 17 | +- The CrowdSec image ships with **zero** AppSec collections by |
| 18 | + default. Ports `:7422` and `:7423` refuse connections until |
| 19 | + `setup-appsec.sh` is run. |
| 20 | +- AppSec mode defaults to `detect` at first boot, which means the |
| 21 | + panel emits the `appsec_url` from minute zero. |
| 22 | +- A single dead sidecar = every host on the panel 500s. Twenty |
| 23 | + hosts, one broken CrowdSec, zero working sites. |
| 24 | + |
| 25 | +Confirmed in the wild on a freshly-deployed v1.3.1: |
| 26 | + |
| 27 | +``` |
| 28 | +caddy_error.log → "crowdsec.appsec" "appsec component unavailable" |
| 29 | + "dial tcp 172.20.0.2:7423: connect: connection refused" |
| 30 | +curl archive.cmos486.es → HTTP 500 |
| 31 | +``` |
| 32 | + |
| 33 | +## The fix |
| 34 | + |
| 35 | +The plugin already exposes an `appsec_fail_open` knob on its app- |
| 36 | +level config (v0.12.1). v1.3.2 starts emitting it, defaulted to |
| 37 | +`true`. Requests now pass through to the backend when the AppSec |
| 38 | +sidecar is unreachable or errors; the WAF-inline round-trip is |
| 39 | +skipped for that request. |
| 40 | + |
| 41 | +Operators who actively run AppSec and want strict enforcement can |
| 42 | +flip the new **AppSec → Fail policy** card in the panel from |
| 43 | +"Fail-open (default, recommended)" to "Fail-closed (strict)". |
| 44 | + |
| 45 | +### Pieces shipped |
| 46 | + |
| 47 | +- `caddycfg` emits `appsec_fail_open: true|false` inside the |
| 48 | + `apps.crowdsec` block whenever `appsec_url` is set. Absent when |
| 49 | + AppSec is fully disabled so Caddy doesn't get an orphan field. |
| 50 | +- New setting `appsec.fail_open` (bool, default `true`) wired |
| 51 | + through `api/settings.go` + the reconciler. |
| 52 | +- New notification event `appsec_unavailable` — severity warning, |
| 53 | + fires on the reachable → unreachable transition of a background |
| 54 | + 5-minute probe of the AppSec URL. `appsec.fail_open` true means |
| 55 | + traffic keeps flowing; the notification is how the operator |
| 56 | + learns their WAF-inline silently degraded. |
| 57 | +- New UI card **AppSec → Fail policy** with a two-radio chooser |
| 58 | + (fail-open vs fail-closed). Only shown when AppSec mode is not |
| 59 | + `disabled`. |
| 60 | +- New troubleshooting section in `docs/operations/troubleshooting.md` |
| 61 | + covering the "connection refused on :7423" symptom, the |
| 62 | + setup-appsec.sh runbook, and how to hook up the new |
| 63 | + notification rule. |
| 64 | + |
| 65 | +## What's NOT here |
| 66 | + |
| 67 | +- **Automatic setup-appsec.sh execution**: the panel does not try |
| 68 | + to install AppSec collections for you. Fail-open prevents |
| 69 | + breakage; getting AppSec actually running is still an operator |
| 70 | + task. |
| 71 | +- **Block-mode auto-downgrade**: the panel does not flip a host |
| 72 | + from block to detect when AppSec is down. Fail-open is per- |
| 73 | + request; the policy is at the bouncer level, not the host level. |
| 74 | +- **AppSec health badge in the status card**: out of scope for the |
| 75 | + hotfix. Today the signal is via the notification event + |
| 76 | + `GET /api/appsec/status` consumed by the card as before. |
| 77 | + |
| 78 | +## Migration |
| 79 | + |
| 80 | +Drop-in. The new setting defaults to `"true"` without a DB |
| 81 | +migration (it's a key/value setting, read-on-demand; absence = |
| 82 | +default). Existing stacks pick up the fix on the next reconcile |
| 83 | +after deploying the new panel + caddy images. |
| 84 | + |
| 85 | +```bash |
| 86 | +cd argos-edge |
| 87 | +git pull |
| 88 | +docker compose build |
| 89 | +docker compose up -d |
| 90 | +``` |
| 91 | + |
| 92 | +No data touched. The `appsec.fail_open` setting surfaces in the UI |
| 93 | +immediately as "Fail-open (default)". |
| 94 | + |
| 95 | +## Rollback |
| 96 | + |
| 97 | +- `git checkout v1.3.1` |
| 98 | +- `docker compose build && up -d` |
| 99 | + |
| 100 | +The setting row may remain in `settings` on rollback — v1.3.1's |
| 101 | +`settingWhitelist` doesn't know about it, but the row is inert: |
| 102 | +the older reconciler never reads it. Harmless orphan, gone on the |
| 103 | +next v1.3.2+ upgrade. |
| 104 | + |
| 105 | +## Related |
| 106 | + |
| 107 | +- [AppSec troubleshooting section](../operations/troubleshooting.md) |
| 108 | + — the "connection refused on :7423" runbook. |
| 109 | +- [Notifications event catalog](../features/notifications.md) — |
| 110 | + `appsec_unavailable` is the new event. |
| 111 | +- [v1.3.1](v1.3.1.md) — previous release (screenshots patch). |
0 commit comments