Skip to content

Add proxy auth support and status/metrics commands#81

Open
nik-kale wants to merge 3 commits into
Cisco-Talos:mainfrom
nik-kale:main
Open

Add proxy auth support and status/metrics commands#81
nik-kale wants to merge 3 commits into
Cisco-Talos:mainfrom
nik-kale:main

Conversation

@nik-kale

@nik-kale nik-kale commented Jan 30, 2026

Copy link
Copy Markdown
  • Add authenticated proxy support via environment variables or config
  • Send proxy credentials as a Basic Proxy-Authorization header (Digest and NTLM are not supported)
  • Add 'cvd status' command for database health checks
  • Add 'cvd metrics' command for Prometheus monitoring
  • Bump version to 1.3.0

Closes #7 and #9. All existing tests pass, plus new unit tests covering the proxy, status, and metrics paths, including Click-level CLI tests.

- Add authenticated proxy support via env vars or config
- Support Basic, Digest, NTLM, and certificate-based (mTLS) auth
- Add 'cvd status' command for database health checks
- Add 'cvd metrics' command for Prometheus monitoring
- Bump version to 1.3.0

Closes Cisco-Talos#7, Cisco-Talos#9, Cisco-Talos#30

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a84f72c1e1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread cvdupdate/cvdupdate.py Outdated
Comment thread cvdupdate/__main__.py
@nik-kale

nik-kale commented Apr 2, 2026

Copy link
Copy Markdown
Author

Updated based on Codex review feedback:

  • Fixed proxy auth: switched from requests.auth objects (which set Authorization instead of Proxy-Authorization) to embedding credentials directly in the proxy URL
  • Fixed UnboundLocalError when using --json --check together
  • Removed --proxy-auth-type option and NTLM dependency (no longer needed with URL-embedded credentials)
  • Updated tests and docs to match

@val-ms val-ms left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution. The proxy auth and monitoring additions are useful directions, but I think this needs changes before we can merge.

Blocking issues:

  • cvd config set is currently broken for normal use. The new optional --proxy-cert and --proxy-cert-key options use click.Path(exists=True) with default="", so Click validates the empty default and exits before the command runs. I reproduced this locally with cvd config set --dbdir /tmp/db, which fails with Invalid value for '--proxy-cert': Path '' does not exist. These should default to None or otherwise avoid validating an absent optional path, and the CLI behavior should be covered by tests.

  • The PR says it closes #30, but the update path still requires DNS before any HTTP/proxy download can happen. db_update() still calls _query_dns_txt_entry() and returns 1 when dns_version_tokens is empty, so environments without direct DNS still cannot update through an HTTP proxy. Please either remove the Closes #30 claim or implement and test a real no-DNS fallback.

  • Proxy credentials can leak in logs/config output when credentials are embedded in proxy_url. _get_proxy_configuration() logs the full proxy URL in the unauthenticated branch, so http://user:pass@proxy.example.com:8080 exposes the secret. config_show() only masks proxy_pass, not credentials already present in proxy_url. Please sanitize userinfo in all displayed/logged proxy URLs.

Other requested changes:

  • The PR body still claims Basic, Digest, NTLM, and certificate-based auth. The current implementation embeds credentials in the proxy URL, which Requests turns into Basic Proxy-Authorization; it does not implement Digest or NTLM. Please narrow the claim or add real support/tests for those auth types.

  • cvdupdate_last_check_timestamp is documented as a Unix timestamp but emits milliseconds (time.time() * 1000). Please either emit seconds or rename/re-document it as milliseconds.

  • Prometheus label values are interpolated without escaping. Custom database names containing quotes, backslashes, or newlines will produce invalid exposition output. Please escape label values per Prometheus text format rules.

  • CHANGES.md still contains placeholder PR links (pull/XX) and says this closes #30. Please update those before merge.

Compatibility note:

This PR also conflicts with PR #88. I checked the current heads with git merge-tree; #81 merges cleanly with #87, but conflicts with #88 in cvdupdate/__main__.py and cvdupdate/cvdupdate.py. The conflict is semantic as well as textual: both PRs introduce or redefine status, both change config set, and #88 restructures config/state handling that #81 builds on. If #88 moves forward, this PR will need to be rebased and reconciled around the CLI design, especially the meaning of status.

Local verification: the PR test suite passes for me (37 passed in 3.05s), and the CLI/config regressions above were reproduced in an isolated temporary venv.

@val-ms val-ms mentioned this pull request May 29, 2026
@nik-kale nik-kale force-pushed the main branch 2 times, most recently from 5ce44a4 to 00e2b51 Compare May 31, 2026 21:23
nik-kale added 2 commits May 31, 2026 14:45
Switch from requests.auth objects (which set the wrong Authorization
header) to embedding credentials in the proxy URL, which is the correct
way to trigger Proxy-Authorization in the requests library. Remove the
now-unnecessary --proxy-auth-type option and NTLM dependency.
Also fix an UnboundLocalError when using cvd status --json --check
by moving the summary assignment above the output format branch.
Update tests and documentation to match.
Remove the proxy client-certificate options. The requests cert parameter is presented to the destination server during the TLS handshake, not to the proxy during CONNECT, so --proxy-cert and --proxy-cert-key never provided proxy mTLS. Removing them also fixes the config set failure where an empty default path was validated by click.Path(exists=True).

Mask proxy credentials wherever they are logged or displayed. The proxy log line and config show now redact userinfo, while the proxy dict returned for use still carries the real credentials. Warn when a proxy URL has no scheme.

Report the status last-check time in UTC. Distinguish an unverified version from an outdated one when DNS is unavailable, and stop treating file age alone as critical so a current mirror holding an infrequently changing database is not flagged. Cache status between metrics scrapes so the server does not run a DNS query on every request.

Emit cvdupdate_last_check_timestamp in seconds and escape Prometheus label values per the text exposition format.

Add Click-level tests for config set, status, and metrics, plus tests for credential masking, label escaping, and the unknown-version case.

Narrow CHANGES.md and the docs to Basic proxy authentication, drop the Closes Cisco-Talos#30 claim, and point the links at pull/81.
@nik-kale

Copy link
Copy Markdown
Author

Thanks for the detailed review @val-ms. I have addressed each point.

Blocking issues:

  1. config set. I removed --proxy-cert and --proxy-cert-key entirely rather than only fixing the empty-default validation. On inspection the requests cert parameter is presented to the destination server during the TLS handshake, not to the proxy during CONNECT, so those options never provided proxy mTLS. Removing them fixes the config set regression and drops a feature that did not do what it claimed. config set is now covered by CliRunner tests.

  2. Closes cvdupdate doesn't work without DNS #30. Dropped. A correct no-DNS path is constrained by the bandwidth design in Disable db version check over HTTP when using database.clamav.net #19, and an HTTP-only fallback for custom databases would not help the cvdupdate doesn't work without DNS #30 case, which uses the official databases behind a proxy. I would rather not claim it than ship a partial fix. CHANGES.md now lists Unable to use proxy #7 and Does cvdupdate work with proxy #9 only.

  3. Credential leakage. Proxy credentials are now masked wherever they are logged or displayed. _get_proxy_configuration logs a sanitized URL in both branches, and config show masks userinfo embedded in proxy_url in addition to proxy_pass. Both are covered by tests.

Other requested changes:

  • Auth claim narrowed to Basic Proxy-Authorization in the PR description and CHANGES.md. Digest and NTLM are not implemented.
  • cvdupdate_last_check_timestamp now emits seconds, matching the name and help text.
  • Prometheus label values are escaped for backslash, double quote, and newline.

While addressing the above I fixed a few related issues:

  • status printed local time labeled UTC; it now reports real UTC.
  • status used file age as the health signal, which flagged infrequently changing databases such as main.cvd as stale or critical even on a current mirror. Version state now drives health, age is informational, and an unverifiable version (DNS unavailable) reports as unknown rather than outdated.
  • metrics --serve ran a live DNS query on every scrape. Status is now cached with a configurable TTL, default 60 seconds.
  • A scheme-less proxy URL now produces a clear warning.

On the #88 overlap: agreed. If #88 lands first I will rebase and reconcile the status command and config set design. Happy to coordinate on ordering.

Test suite passes locally, 51 passed.

@nik-kale nik-kale requested a review from val-ms May 31, 2026 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unable to use proxy

2 participants