Block sign-ups from disposable email domains

[nicolai-rhesis]

Purpose

Closes #2237. Disposable-email domains are used to create throwaway accounts that spam or abuse the platform. Sign-up currently validates email format and MX deliverability only, so temporary inboxes like mailinator.com pass — mailinator.com has real MX records and clears the existing check.

What Changed

• New app/auth/disposable_email.py. Merges the disposable-email-domains community list (~8.2k domains, MIT, near-daily releases) with an in-repo disposable_domains_custom.txt and a comma-separated env override, cached in one frozenset.
• Matching is on the registrable domain, in punycode. Labels are stripped left to right, so mail.smtp.mailinator.com hits the mailinator.com entry — otherwise any subdomain bypasses the list. Stripping stops at two labels so a bare TLD slipping into the list can't block every address under it. Comparison uses .ascii_domain, not .normalized: the latter returns the Unicode form, so foo@münchen.de would never match an xn--mnchen-3ya.de entry.
• Three enforcement points, all self-serve: EmailProvider.register(), request_magic_link() (which had no domain validation at all), and the new-user branch of find_or_create_user_from_auth(). Admin invites via POST /users/ are not screened.
• AUTH_BLOCK_DISPOSABLE_EMAILS is three-state — off / log / enforce — defaulting to log. A mode="before" validator still accepts the boolean spellings from the issue (true/false/1/0/yes/no), so AUTH_BLOCK_DISPOSABLE_EMAILS=false fully disables the check as specified.
• DisposableEmailError(ValueError). Subclassing ValueError means the call sites that already map a bad address to a 400 needed no change, while the OAuth callback catches it specifically so a policy rejection isn't logged as an error with a traceback.
• AUTH_BLOCK_DISPOSABLE_EMAILS and AUTH_DISPOSABLE_EMAIL_EXTRA_DOMAINS added to .env.example and docker-compose.yml.

Additional Context

Two deliberate departures from the issue, both worth a look during review.

1. Default is log, not enforce, which contradicts acceptance criterion 1 on day one. Community lists occasionally carry domains real users have, and learning that false-positive rate from a log query beats learning it from a support ticket. Log mode records the redacted address, the registrable domain, the matched entry, and which sign-up path it came through. Proposal: run it for a week, grep the logs, then flip to enforce. Everything needed for the flip is already here.

2. The magic-link 400 is a narrow enumeration leak. The screener has to sit inside the if not user: branch, or existing accounts on a disposable domain get locked out — which the issue lists as a non-goal. That means a disposable-domain address gets a 400 when unregistered and a 200 when registered, on an endpoint that is otherwise strictly enumeration-safe. It only leaks for domains we're refusing to register anyway, so I took the clearer error message over the silent 200, but that's a judgement call and easy to reverse. Flagged with a comment at the call site.

On the review discussion in the issue:

• @akwasigroch — I went with the package over copying the list; it ships near-daily (0.0.232 → 0.0.237 in six days), so pinning it and letting Dependabot bump is the same freshness with no hand-syncing. On the env var: kubernetes/base uses external-secrets, so changing it is a secret update plus a rollout restart — no image build, no code deploy. It does what the issue wanted.
• @eason4kim-rocket — took the registrable-domain/punycode point (it was a real bypass, and the IDN half is confirmed by a test) and the log-only-first point, which shaped the default above. Skipped the daily GitHub fetch of the raw .conf: there is no Celery beat in this repo, so it means new scheduled-job infra plus a runtime network fetch in an auth path, to buy back a lag of a couple of days. Skipped the DB-table-plus-poll for the custom list too — right answer once there's an admin UI, but no audit trail has been asked for yet. The disposable-MX-target denylist is a good idea that needs data first; the log-only phase should tell us whether it's worth it.
• @rotenkus-dot — the four-field signal struct is more machinery than a boolean sign-up gate needs, so the checker returns a match or nothing and the mode setting carries the policy. The test coverage suggested (registrable domain, IDN, list hit/miss, feature flag, self-serve vs admin invite) is all present. No SMTP probing was added.

Testing

37 new tests, all passing. Nothing else in tests/backend/auth/ or tests/backend/routes/test_auth.py broke (485 pass).

cd apps/backend
uv run pytest ../../tests/backend/auth/test_disposable_email.py -v
uv run pytest ../../tests/backend/routes/test_auth.py::TestDisposableEmailSignupBlocking -v

Verified by hand against a local backend. In the default log mode, all three sign-ups return 200 and the two disposable ones produce a log line while the legitimate one produces none:

WARNING - Disposable sign-up domain matched: email=sp***@mail.mailinator.com
          domain=mail.mailinator.com matched=mailinator.com source=magic_link mode=log

In enforce mode:

POST /auth/magic-link  abuse@mail.10minutemail.com     -> 400  Disposable email addresses are not accepted.
POST /auth/magic-link  x@sub.trash-domain.example      -> 400  (env override + subdomain)
POST /auth/magic-link  nicolai@rhesis.ai               -> 200  link sent
POST /auth/register    spammer2@mailinator.com         -> 400  (cleared the MX check first, stopped by the screener)
POST /auth/register    nicolai-demo@rhesis.ai          -> 200  registered

The existing-user exemption, same domain back to back:

brand-new-signup@mailinator.com -> 400  rejected
spammer@mailinator.com          -> 200  link sent (account already existed)

And the admin invite on the same domain the sign-up paths just refused:

POST /users/  invited-colleague@mailinator.com  -> 200  user created

No tracebacks logged for any of the rejections. The OAuth path can't be driven over HTTP locally (no provider credentials in local .env), so it's covered by a test calling find_or_create_user_from_auth directly with a GOOGLE AuthUser rather than by hand.

[peqy]

Solid implementation: registrable-domain + punycode matching, three-state mode, and good coverage across the three self-serve entry points.

Improvement: Magic-link endpoint docstring says it always returns 200, but enforce mode now returns 400 for disposable domains (new-user branch).

Improvement: Consider making AUTH_BLOCK_DISPOSABLE_EMAILS case-insensitive for OFF/LOG/ENFORCE (validator already handles other string normalizations).

Found 2 issues (0 critical, 2 improvements).

[peqy]

Improvement: request_magic_link()’s docstring still says “Always returns 200 to prevent email enumeration”, but with this new DisposableEmailError path it can now return 400 (in enforce mode, and only for new users).

Fix: update the docstring to match the new contract, or (if you want to preserve enumeration-safety) keep returning 200 here and only log/drop the request.

[nicolai-rhesis]

Fixed the docstring in ab55a81 — it now states the actual contract rather than claiming an unconditional 200.

On your second option (keep returning 200 and log/drop): still genuinely open, and it is called out in the PR description under "Additional Context" as a deliberate departure. The constraint is that the check has to sit inside the if not user: branch or existing accounts on a disposable domain get locked out, which the issue lists as a non-goal — so the choice is between a clearer 400 that leaks existence for refused domains, or a silent 200 where a false-positive legitimate user waits for mail that never arrives. Leaving it as a 400 for now and flagging it for a human reviewer to weigh in on.

[peqy]

Fixed in 9e097d6 — docstring now reflects the 400-in-enforce exception for disposable domains on the sign-up branch.

[peqy]

Thanks — docstring fix sounds good.

On the 200 vs 400: given enforce is an explicit “policy rejection” mode (and you’re already defaulting to log to measure false positives first), I’m okay with the current 400-on-new-user behavior. It’s a small enumeration leak limited to domains you’re refusing anyway, and it avoids the “wait for an email that will never arrive” UX, especially if the blocklist ever has a false positive.

[peqy]

Improvement: block_disposable_emails is a Literal["off","log","enforce"], but the validator only lowercases for legacy boolean synonyms. If someone sets AUTH_BLOCK_DISPOSABLE_EMAILS=LOG/ENFORCE/OFF (common for env vars), it’ll fail validation.

Fix: in _accept_boolean_form, lowercase any string first (like backend_env does), then return the lowered value if it’s already one of the allowed literals.

[nicolai-rhesis]

Fixed in a5cf549. Confirmed it was worse than "fails validation" — AUTH_BLOCK_DISPOSABLE_EMAILS=ENFORCE failed AuthSettings construction outright, so it would have taken the app down at startup rather than falling back to a default. The validator now lowercases any string, following the backend_env precedent you pointed at. A real typo (enfroce) still raises instead of silently defaulting, covered by a test.

[peqy]

Fixed in c0a76b6 (+ a0dd131 test) — the validator now lowercases/strips any string and accepts OFF/LOG/ENFORCE as well as legacy boolean spellings.

[peqy]

Makes sense — thanks for confirming the failure mode and fixing it by lowercasing/stripping any string. Agree it’s better to still raise on real typos (and the test coverage for that is exactly what I’d want). This resolves my concern here.

[peqy]

Looks good. The disposable-domain screener is wired into all three self-serve sign-up paths with solid normalization + caching, and the follow-up commits addressed the docstring + env-mode casing issues. Ship it.

[peqy]

Looks good. Nice integration of the upstream disposable-domain list + custom/env overlays, with registrable-domain + punycode matching and solid wiring across password/magic-link/OAuth flows. The follow-up fixes for the magic-link docstring and case-insensitive AUTH_BLOCK_DISPOSABLE_EMAILS handling address the earlier concerns. Ship it.