Guacamole behind Pangolin/Traefik/Gerbil broke after upgrade to Pangolin 1.14.1 and Newt 1.8.1

[DaveCorsello]

After upgrading Pangolin from 1.13.1 to 1.14.1, Newt from 1.7.0 to 1.8.1 and Traefik from 3.6.5 to 3.6.6., logging into Apache Guacamole, a public resource in Pangolin, became unreliable. After entering MFA in Guacamole, its UI showed the generic message:

“An error has occurred and this action cannot be completed…”

Key observations:

Traefik access logs showed POST /guacamole/api/tokens alternating between:

• 502 (upstream unreachable / bad gateway),

• 500 with no ServiceName (request handled before routing; suggests router/service missing temporarily / dynamic config instability),

• and occasionally 403 from the backend (Guacamole returned JSON 403).

Pangolin-generated Traefik config routed Guacamole to a dynamic overlay endpoint:

• 1-Guacamole-service -> http://100.89.128.16:<ephemeral_port>

I have searched for other recent discussions in this forum on the topic of api tokens and have found none.

Gerbil's WireGuard interface:

• wg0 on 100.89.128.1/24,

Newt 1.8.1 logs on the remote side repeatedly showed:

• Ping failed ... recurring every minute (this happens across multiple versions of Newt and doesn't seem to me to be related to the Guacamole problems)

• Connection to server lost ... reconnecting,

• then Tunnel connection established ... Started tcp proxy to <guacamole's private IP>:8080

This reconnection loop correlated with the 502/500 bursts and errors/inaccessible pages in the Guacamole UI.

Rollback and roll forward results:

Rolling back Pangolin to 1.13.1, Newt to 1.7.0 and Traefik to 3.6.5 restored Guacamole stability.
Rolling forward Pangolin to 1.14.1 and Newt to 1.8.1 while keeping Traefik fixed at 3.6.5 resulted in the Guacamole issues described above.
Rolling back Pangolin and Newt to version 1.13.1 and 1.7.0 respectively again restored Guacamole stability.

Conclusion:

It appears that upgrading to Pangolin 1.14.1 and Newt 1.8.1 results in errors in Guacamole.

[didotb]

You might want to transfer this as an issue to the main pangolin repo rather than a discussion.

[DaveCorsello]

Thanks for your suggestion.