Skip to content

fix: pass re.IGNORECASE via flags instead of deprecated positional arg - #2029

Open
TrueFurina wants to merge 1 commit into
smicallef:masterfrom
TrueFurina:pr7-ignorecase
Open

fix: pass re.IGNORECASE via flags instead of deprecated positional arg#2029
TrueFurina wants to merge 1 commit into
smicallef:masterfrom
TrueFurina:pr7-ignorecase

Conversation

@TrueFurina

Copy link
Copy Markdown

Summary

sflib.py removeUrlCreds() (line 1117):

ret = re.sub(pat, pats[pat], ret, re.IGNORECASE)

The 4th positional argument of re.sub() is count, not flags. As a
result:

  1. Case-insensitivity silently never worked — mixed-case credentials
    (KEY=, PassWord=, APIKEY=…) are NOT redacted from URLs.
  2. Each pattern replaces at most the first 2 occurrences.
  3. Python 3.13 emits a DeprecationWarning: 'count' is passed as positional argument (and a future Python may make this an error).

removeUrlCreds is the log-sanitization path for URLs, so this is a real
leak vector for credentials ending up in logs.

Fix: pass the flag by keyword.

ret = re.sub(pat, pats[pat], ret, flags=re.IGNORECASE)

Verification

Behavior comparison on
https://USER:PassWord=secret@example.com/?APIKEY=abc&key=1&key=2&key=3:

result leaks credential values?
before (count=2) https://USER:PassWord=secret@example.com/?APIKEY=abc&key=XXX yes (PassWord, secret, APIKEY)
after (flags=IGNORECASE) https://USER:password=XXX no
  • Existing test test_remove_url_creds_should_remove_credentials_from_url
    passes; pytest test/unit/test_spiderfoot.py → 74 passed (the 2 remaining
    failures are the known network-dependent tests: resolve_host6 DNS and the
    flaky fetchUrl against spiderfoot.net — unrelated to this change).

Files changed

  • sflib.py (1 line)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant