Testing: evaluate TLS-Anvil against Reseau's TLS client & server

[quinnj]

Summary

Evaluate TLS-Anvil against Reseau's native TLS client and server, triage the report, and decide on an ongoing cadence.

Motivation

Reseau implements the TLS 1.2 and 1.3 client + server state machines natively (src/tls/handshake_client_tls12.jl, handshake_client_tls13.jl, handshake_server_tls12.jl, handshake_server_tls13.jl, record_tls12.jl, record_tls13.jl). We have solid known-answer coverage for handshake-message encode/decode (test/tls_handshake_messages_tests.jl) and several native end-to-end handshake tests, but no systematic adversarial protocol-conformance suite. The negative tests we have are hand-picked (cases we thought of); a catalogue-driven suite finds the cases we didn't.

TLS-Anvil is an automated, RFC- and CVE-derived TLS test suite (USENIX Security '22) that exercises state-machine correctness, illegal message sequences, record/padding edge cases, extension handling, downgrade protection, etc.

Proposal

1. Stand up a minimal Reseau TLS echo/test server with a fixed cert (reuse test/resources), exposed on host:port.
2. Run TLS-Anvil (Docker image available) against it; collect the HTML/JSON report.
3. Triage each result into: PASS / expected-deviation (feature we intentionally don't support) / real bug. Record expected deviations as an allow-list.
4. If TLS-Anvil's client-testing mode is mature enough, repeat with Reseau as the client.

Caveats / notes

• TLS-Anvil is JVM-based and historically server-focused (client support has improved but verify before relying on it).
• First deliverable is a manual run + triage + a documented runbook, not CI automation. Decide cadence afterward (likely nightly or on-demand, not PR-blocking).

Acceptance / scope

• A runbook in test/ (or docs/) describing how to launch the test server and run TLS-Anvil against it.
• A triaged report with any real bugs split out into their own issues, and an allow-list of intentional deviations.

Links

• TLS-Anvil: https://github.com/tls-attacker/TLS-Anvil
• Paper (USENIX Security '22) — "TLS-Anvil: Adapting Combinatorial Testing for TLS Libraries": https://www.usenix.org/conference/usenixsecurity22/presentation/maehren
• Underlying framework (TLS-Attacker): https://github.com/tls-attacker/TLS-Attacker
• Docs: https://github.com/tls-attacker/TLS-Anvil/wiki

Files of interest

src/tls/handshake_*_tls1{2,3}.jl, src/tls/record_tls1{2,3}.jl, src/tls/handshake_messages.jl.

---

One of a set of TLS test-infrastructure hardening initiatives (differential fuzzing / TLS-Anvil / Wycheproof / BoGo).