diff --git a/CHANGES.md b/CHANGES.md index ec35ff9..30aff26 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,6 +1,7 @@ # Notable Changes > _Note_: Changes should be grouped by release and use these icons: +> > - Added: ➕ > - Changed: 🌌 > - Deprecated: 👇 @@ -8,6 +9,38 @@ > - Fixed: 🐛 > - Security: 🛡 +## Version 1.3.0 + +- ➕ Add authenticated proxy support. Users can configure proxy credentials + via environment variables (CVDUPDATE_PROXY_URL, CVDUPDATE_PROXY_USER, + CVDUPDATE_PROXY_PASS) or via the config file using `cvd config set`. + Credentials are embedded in the proxy URL so that a Basic + `Proxy-Authorization` header is sent. Digest and NTLM proxy schemes are + not supported. Credentials are masked in logs and in `cvd config show`. + Feature courtesy of Nik Kale. + + Closes #7, #9. + + [GitHub Pull-Request](https://github.com/Cisco-Talos/cvdupdate/pull/81) + +- ➕ Add `cvd status` command for database health checks. The new command + reports the health and currency of downloaded databases, including + version comparisons, age status, and identification of stale or + missing databases. Supports JSON output for monitoring integration + and a --check flag for scripted health checks. + Feature courtesy of Nik Kale. + + [GitHub Pull-Request](https://github.com/Cisco-Talos/cvdupdate/pull/81) + +- ➕ Add `cvd metrics` command for Prometheus monitoring integration. The + new command outputs database status as Prometheus-format metrics. + Supports one-shot output to stdout and a persistent HTTP server mode + for Prometheus scraping. Exposes per-database version, age, and status + metrics plus aggregate health metrics. + Feature courtesy of Nik Kale. + + [GitHub Pull-Request](https://github.com/Cisco-Talos/cvdupdate/pull/81) + ## Version 1.2.0 - ➕ Support for downloading CVD and CDIFF digital signatures. @@ -108,6 +141,7 @@ `CVDUPDATE_NAMESERVER` environment variable as a comma separated list. E.g.: + ```bash CVDUPDATE_NAMESERVER=1.1.1.1,8.8.8.8 cvd update ``` @@ -123,8 +157,8 @@ Fix courtesy of Brent Clark. - - 🌌 CVD-Update will no longer remove extra files from the database directory - when you run `cvd clean dbs`. It will only remove those file managed by the - CVD-Update tool. + when you run `cvd clean dbs`. It will only remove those file managed by the + CVD-Update tool. This means that you can now store third-party extra signature databases in the CVD-Update database directory and CVD-Update will not delete them if you run @@ -143,6 +177,7 @@ Improvement courtesy of Bill Sanders. Special thanks to: + - Bill Sanders - Brent Clark - Michael Callahan @@ -213,7 +248,6 @@ Special thanks to: - ➕ Two ways to set a custom DNS nameserver. DNS queries are required to check the latest available database versions. - 1. Set the nameserver in the config. Eg: ```bash diff --git a/README.md b/README.md index 7bda2f2..e24d6e4 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ Run this tool as often as you like, but it will only download new content if the - Python 3.6 or newer. - An internet connection with DNS enabled. -- The following Python packages. These will be installed automatically if you use `pip`, but may need to be installed manually otherwise: +- The following Python packages. These will be installed automatically if you use `pip`, but may need to be installed manually otherwise: - `click` v7.0 or newer - `colorlog` v6.7 or newer - `colorama` @@ -44,6 +44,7 @@ python3 -m pip install --user cvdupdate ### Updating Your Installation When running `cvd update` to update the databases, it will also check if there is a new version of the `cvdupdate` package on Python's PyPI package repository. If there is a newer version of `cvdupdate`, you will see a message prompting you to upgrade. It will look someething like this: + ``` WARNING You are running cvdupdate version: 1.1.0. WARNING There is a newer version on PyPI: 1.1.1. Please update! @@ -94,11 +95,13 @@ cvd serve > _Disclaimer_: The `cvd serve` feature is not intended for production use, just for testing. You probably want to use a more robust HTTP server for production work. Install ClamAV if you don't already have it and, in another terminal window, modify your `freshclam.conf` file. Replace: + ``` DatabaseMirror database.clamav.net ``` ... with: + ``` DatabaseMirror http://localhost:8000 ``` @@ -107,7 +110,7 @@ DatabaseMirror http://localhost:8000 Now, run `freshclam -v` or `freshclam.exe -v` to see what happens. You should see FreshClam successfully update it's own database directory from your private database server. -Run `cvd update` as often as you need. Maybe put it in a `cron` job. +Run `cvd update` as often as you need. Maybe put it in a `cron` job. > _Tip_: Each command supports a `--verbose` (`-V`) mode, which often provides more details about what's going on under the hood. @@ -141,6 +144,18 @@ I selected `30 */4 * * *` to run at minute 30 past every 4th hour. CVD-Update us CVD-Update will write logs to the `~/.cvdupdate/logs` directory, which is why I directed `stdout` and `stderr` to `/dev/null` instead of a log file. You can use the `cvd config set` command to customize the log directory if you like, or redirect `stdout` and `stderr` to a log file if you prefer everything in one log instead of separate daily logs. +### Monitoring Example + +You can integrate CVD-Update status checks with your monitoring system: + +```bash +# Check status and alert if unhealthy +cvd status --check || send_alert "ClamAV databases unhealthy" + +# Get JSON status for monitoring dashboard +cvd status --json | curl -X POST -d @- https://monitoring.example.com/metrics +``` + ## Optional Functionality ### Using a custom DNS server @@ -161,7 +176,7 @@ DNS is required for CVD-Update to function properly (to gather the TXT record co The environment variable will take precedence over the nameserver config setting. -Note: Both options can be used to provide a comma-delimited list of nameservers to utilize for resolution. +Note: Both options can be used to provide a comma-delimited list of nameservers to utilize for resolution. ### Using a proxy @@ -190,23 +205,140 @@ export https_proxy cvd update -V ``` -> _Disclaimer_: CVD-Update doesn't support proxies that require authentication at this time. If your network admin allows it, you may be able to work around it by updating your proxy to allow HTTP requests through unauthenticated if the User-Agent matches your specific CVD-Update user agent. The CVD-Update User-Agent follows the form `CVDUPDATE/ ()` where the `uuid` is unique to your installation and can be found in the `~/.cvdupdate/state.json` file (or `~/.cvdupdate/config.json` for cvdupdate <=1.0.2). See https://github.com/Cisco-Talos/cvdupdate/issues/9 for more details. -> -> Adding support for proxy authentication is a ripe opportunity for a community contribution to the project. +### Using an authenticated proxy + +CVD-Update supports authenticated proxy connections. Credentials are embedded in the proxy URL so that the `Proxy-Authorization` header is sent correctly. + +**Using environment variables:** + +```bash +CVDUPDATE_PROXY_URL="http://proxy.example.com:8080" \ +CVDUPDATE_PROXY_USER="myuser" \ +CVDUPDATE_PROXY_PASS="mypassword" \ +cvd update -V +``` + +Environment variables: + +- `CVDUPDATE_PROXY_URL` - Proxy URL (e.g., http://proxy.example.com:8080) +- `CVDUPDATE_PROXY_USER` - Proxy username +- `CVDUPDATE_PROXY_PASS` - Proxy password + +**Using config file settings:** + +```bash +cvd config set --proxy-url http://proxy.example.com:8080 +cvd config set --proxy-user myuser +cvd config set --proxy-pass # Will prompt for password +``` + +Environment variables take precedence over config file settings. Special characters in credentials are automatically URL-encoded. + +The proxy URL credentials are sent as a Basic `Proxy-Authorization` header. Digest and NTLM proxy schemes are not supported. + +### Check database status + +Check the health and currency of your downloaded databases: + +```bash +cvd status +``` + +For JSON output (useful for monitoring integration): + +```bash +cvd status --json +``` + +For scripted health checks (returns non-zero exit code on problems): + +```bash +cvd status --check +if [ $? -ne 0 ]; then + echo "Database health check failed" +fi +``` + +The status command reports: + +- Local and remote database versions +- Database age and staleness +- Missing or corrupted databases +- Databases on rate-limit cooldown +- CDIFF patch file counts + +### Prometheus Metrics + +CVD-Update can expose database status as Prometheus metrics for monitoring integration. + +One-shot metrics output: + +```bash +cvd metrics +``` + +Example output: + +``` +# HELP cvdupdate_database_version Local version of ClamAV database +# TYPE cvdupdate_database_version gauge +cvdupdate_database_version{database="main.cvd"} 62 +cvdupdate_database_version{database="daily.cvd"} 27456 +# HELP cvdupdate_databases_total Total number of configured databases +# TYPE cvdupdate_databases_total gauge +cvdupdate_databases_total 3 +# HELP cvdupdate_health_status Overall health (0=critical, 1=warning, 2=healthy) +# TYPE cvdupdate_health_status gauge +cvdupdate_health_status 2 +``` + +Start a persistent metrics server for Prometheus scraping: + +```bash +cvd metrics --serve --port 9090 --bind 0.0.0.0 +``` + +Add to your Prometheus configuration: + +```yaml +scrape_configs: + - job_name: 'cvdupdate' + static_configs: + - targets: ['localhost:9090'] + scrape_interval: 5m +``` + +Available metrics: + +| Metric | Type | Description | +| --------------------------------- | ----- | -------------------------------- | +| cvdupdate_database_version | gauge | Local version of each database | +| cvdupdate_database_remote_version | gauge | Latest available version | +| cvdupdate_database_age_seconds | gauge | Age of database in seconds | +| cvdupdate_database_current | gauge | 1 if current, 0 if outdated | +| cvdupdate_database_missing | gauge | 1 if missing, 0 if present | +| cvdupdate_database_cooldown | gauge | 1 if on cooldown, 0 otherwise | +| cvdupdate_database_size_bytes | gauge | Size of database file | +| cvdupdate_databases_total | gauge | Total configured databases | +| cvdupdate_databases_current | gauge | Number of current databases | +| cvdupdate_databases_stale | gauge | Number of stale databases | +| cvdupdate_health_status | gauge | 0=critical, 1=warning, 2=healthy | ## Files and directories created by CVD-Update This tool is to creates the following directories: - - `~/.cvdupdate` - - `~/.cvdupdate/logs` - - `~/.cvdupdate/databases` + +- `~/.cvdupdate` +- `~/.cvdupdate/logs` +- `~/.cvdupdate/databases` This tool creates the following files: - - `~/.cvdupdate/config.json` - - `~/.cvdupdate/state.json` - - `~/.cvdupdate/databases/.cvd` - - `~/.cvdupdate/databases/-.cdiff` - - `~/.cvdupdate/logs/.log` + +- `~/.cvdupdate/config.json` +- `~/.cvdupdate/state.json` +- `~/.cvdupdate/databases/.cvd` +- `~/.cvdupdate/databases/-.cdiff` +- `~/.cvdupdate/logs/.log` > _Tip_: You can set custom `database` and `logs` directories with the `cvd config set` command. It is likely you will want to customize the `database` directory to point to your HTTP server's `www` directory (or equivalent). Bare in mind that if you already downloaded the databases to the old directory, you may want to move them to the new directory. @@ -337,36 +469,43 @@ docker run -d \ -v /var/log/cvdupdate:/cvdupdate/logs \ -e CRON='0 0 * * *' \ cvdupdate:latest - ``` +``` + ## Use Docker Compose A Docker `compose.yaml` is provided to: + 1. Regularly update a Docker volume with the latest ClamAV databases. -2. Serve a database mirror on port 8000 using the Apache webserver. +2. Serve a database mirror on port 8000 using the Apache webserver. Edit the `compose.yaml` file if you need to change the default values: -* Port 8000 -* USER_ID=0 -* CRON=30 */4 * * * +- Port 8000 +- USER_ID=0 +- CRON=30 _/4 _ \* \* ### Build + ```bash docker compose build ``` ### Start + ```bash docker compose up -d ``` ### Stop + ```bash docker compose down ``` ### Volumes + Volumes are defined in `compose.yaml` and will be auto-created when you run `docker compose up` + ``` DRIVER VOLUME NAME local cvdupdate_database @@ -383,13 +522,13 @@ Join the ClamAV community on the [ClamAV Discord chat server](https://discord.gg ### Report issues -If you find an issue with CVD-Update or the CVD-Update documentation, please submit an issue to our [GitHub issue tracker](https://github.com/Cisco-Talos/cvdupdate/issues). Before you submit, please check to if someone else has already reported the issue. +If you find an issue with CVD-Update or the CVD-Update documentation, please submit an issue to our [GitHub issue tracker](https://github.com/Cisco-Talos/cvdupdate/issues). Before you submit, please check to if someone else has already reported the issue. ### Development If you find a bug and you're able to craft a fix yourself, consider submitting the fix in a [pull request](https://github.com/Cisco-Talos/cvdupdate/pulls). Your help will be greatly appreciated. -If you want to contribute to the project and don't have anything specific in mind, please check out our [issue tracker](https://github.com/Cisco-Talos/cvdupdate/issues). Perhaps you'll be able to fix a bug or add a cool new feature. +If you want to contribute to the project and don't have anything specific in mind, please check out our [issue tracker](https://github.com/Cisco-Talos/cvdupdate/issues). Perhaps you'll be able to fix a bug or add a cool new feature. _By submitting a contribution to the project, you acknowledge and agree to assign Cisco Systems, Inc the copyright for the contribution. If you submit a significant contribution such as a new feature or capability or a large amount of code, you may be asked to sign a contributors license agreement comfirming that Cisco will have copyright license and patent license and that you are authorized to contribute the code._ @@ -399,23 +538,23 @@ The following steps are intended to help users that wish to contribute to develo 1. Create a fork of the [CVD-Update git repository](https://github.com/Cisco-Talos/cvdupdate), and then clone your fork to a local directory. - For example: + For example: - ```bash - git clone https://github.com//cvdupdate.git - ``` + ```bash + git clone https://github.com//cvdupdate.git + ``` -2. Make sure CVD-Update is not already installed. If it is, remove it. +2. Make sure CVD-Update is not already installed. If it is, remove it. - ```bash - python3 -m pip uninstall cvdupdate - ``` + ```bash + python3 -m pip uninstall cvdupdate + ``` 3. Use pip to install CVD-Update in "edit" mode. - ```bash - python3 -m pip install -e --user ./cvdupdate - ``` + ```bash + python3 -m pip install -e --user ./cvdupdate + ``` Once installed in "edit" mode, any changes you make to your clone of the CVD-Update code will be immediately usable simply by running the `cvdupdate` / `cvd` commands. diff --git a/cvdupdate/__main__.py b/cvdupdate/__main__.py index ce7a64b..c858bf5 100644 --- a/cvdupdate/__main__.py +++ b/cvdupdate/__main__.py @@ -153,19 +153,43 @@ def config(): @click.option("--logdir", "-l", type=click.Path(), required=False, default="", help="Set a custom log directory. [optional]") @click.option("--dbdir", "-d", type=click.Path(), required=False, default="", help="Set a custom database directory. [optional]") @click.option("--nameserver", "-n", type=click.STRING, required=False, default="", help="Set a custom DNS nameserver. [optional]") -def config_set(config: str, verbose: bool, logdir: str, dbdir: str, nameserver: str): +@click.option("--proxy-url", type=click.STRING, required=False, default="", help="Proxy URL (e.g., http://proxy.example.com:8080). [optional]") +@click.option("--proxy-user", type=click.STRING, required=False, default="", help="Proxy username. [optional]") +@click.option("--proxy-pass", type=click.STRING, required=False, default="", is_flag=False, flag_value="__PROMPT__", help="Proxy password. Use flag without value to prompt for input. [optional]") +def config_set(config: str, verbose: bool, logdir: str, dbdir: str, nameserver: str, proxy_url: str, proxy_user: str, proxy_pass: str): """ Set up first time configuration. The default directories will be in ~/.cvdupdate """ - CVDUpdate( + # Handle password prompt if flag was used without value + if proxy_pass == "__PROMPT__": + proxy_pass = click.prompt("Proxy password", hide_input=True) + + m = CVDUpdate( config=config, verbose=verbose, log_dir=logdir, db_dir=dbdir, nameserver=nameserver) + # Update proxy settings if provided + need_save = False + if proxy_url: + m.config['proxy_url'] = proxy_url + need_save = True + if proxy_user: + m.config['proxy_user'] = proxy_user + need_save = True + if proxy_pass: + m.config['proxy_pass'] = proxy_pass + need_save = True + + if need_save: + m._save_config() + if verbose: + print("Proxy configuration updated.") + @config.command("show") @click.option("--config", "-c", type=click.Path(), required=False, default="", help="Config path. [optional]") @click.option("--verbose", "-V", is_flag=True, default=False, help="Verbose output. [optional]") @@ -212,6 +236,205 @@ def clean_all(config: str, verbose: bool): m.clean_all() +@cli.command("status") +@click.option("--config", "-c", type=click.Path(), required=False, default="", help="Config path. [optional]") +@click.option("--verbose", "-V", is_flag=True, default=False, help="Verbose output. [optional]") +@click.option("--json", "-j", "output_json", is_flag=True, default=False, help="Output in JSON format. [optional]") +@click.option("--check", is_flag=True, default=False, help="Exit with non-zero status if databases are not healthy. [optional]") +def db_status(config: str, verbose: bool, output_json: bool, check: bool): + """ + Check the health and currency of downloaded databases. + + Reports each database's local and remote version, file age, and version + status (current, behind, unknown, or missing). Use --check for scripted + health checks that return a non-zero exit code on problems. + + Version status comes from comparing the local version against the version + advertised over DNS. When DNS is unavailable the status is reported as + unknown rather than behind. The Age column is informational, since some + databases such as main.cvd change infrequently and a current mirror can + hold an old but correct file. Age thresholds are fixed at 24, 48, and 72 + hours and are not yet configurable per database. + """ + import json as json_module + import datetime + + m = CVDUpdate(config=config, verbose=verbose) + status = m.db_status() + summary = status['summary'] + + if output_json: + click.echo(json_module.dumps(status, indent=2)) + else: + # Human-readable output + databases = status['databases'] + warnings = status['warnings'] + + # Format overall status + overall = summary['overall_status'].upper() + if overall == 'HEALTHY': + status_color = Fore.GREEN + elif overall == 'WARNING': + status_color = Fore.YELLOW + else: + status_color = Fore.RED + + last_check = datetime.datetime.fromtimestamp(summary['last_check'], tz=datetime.timezone.utc).strftime('%Y-%m-%d %H:%M:%S UTC') + + click.echo("") + click.echo("CVD-Update Database Status") + click.echo("==========================") + click.echo(f"Overall Status: {status_color}{overall}{Style.RESET_ALL}") + click.echo(f"Last Check: {last_check}") + click.echo("") + + # Table header + click.echo(f"{'Database':<16} {'Local':<8} {'Remote':<8} {'Age':<10} {'Status':<12} {'Size':<10}") + click.echo(f"{'-'*16} {'-'*8} {'-'*8} {'-'*10} {'-'*12} {'-'*10}") + + for db in databases: + name = db['name'] + local_ver = str(db['local_version']) if db['local_version'] is not None else '-' + remote_ver = str(db['remote_version']) if db['remote_version'] is not None else '-' + + # Format age + if db['age_hours'] is None: + age_str = '-' + elif db['age_hours'] < 1: + age_str = f"{int(db['age_hours'] * 60)}m" + elif db['age_hours'] < 24: + age_str = f"{int(db['age_hours'])}h" + else: + days = int(db['age_hours'] / 24) + hours = int(db['age_hours'] % 24) + age_str = f"{days}d {hours}h" + + # Format status with color based on version state. The Age column + # already conveys file age, and a current mirror can hold an old + # file, so version state is the meaningful health signal here. + if db['is_missing']: + status_str = f"{Fore.RED}MISSING{Style.RESET_ALL}" + elif db['version_status'] == 'current': + status_str = f"{Fore.GREEN}CURRENT{Style.RESET_ALL}" + elif db['version_status'] == 'outdated': + status_str = f"{Fore.RED}BEHIND{Style.RESET_ALL}" + else: + status_str = f"{Fore.YELLOW}UNKNOWN{Style.RESET_ALL}" + + # Format size + if db['file_size_bytes'] is None: + size_str = '-' + elif db['file_size_bytes'] < 1024: + size_str = f"{db['file_size_bytes']} B" + elif db['file_size_bytes'] < 1024 * 1024: + size_str = f"{db['file_size_bytes'] / 1024:.1f} KB" + else: + size_str = f"{db['file_size_bytes'] / (1024 * 1024):.1f} MB" + + # Note: the status column needs extra padding due to color codes + click.echo(f"{name:<16} {local_ver:<8} {remote_ver:<8} {age_str:<10} {status_str:<23} {size_str:<10}") + + click.echo("") + + # Show warnings + if warnings: + click.echo(f"{Fore.YELLOW}Warnings:{Style.RESET_ALL}") + for warning in warnings: + click.echo(f" - {warning}") + click.echo("") + + click.echo(f"Summary: {summary['current_count']}/{summary['total_databases']} databases current, {len(warnings)} warnings") + click.echo("") + + # Exit with appropriate code if --check flag is used + if check: + if summary['overall_status'] == 'healthy': + sys.exit(0) + elif summary['overall_status'] == 'warning': + sys.exit(1) + else: # critical + sys.exit(2) + + +@cli.command("metrics") +@click.option("--config", "-c", type=click.Path(), required=False, default="", help="Config path. [optional]") +@click.option("--verbose", "-V", is_flag=True, default=False, help="Verbose output. [optional]") +@click.option("--serve", "-s", is_flag=True, default=False, help="Start HTTP server for Prometheus scraping. [optional]") +@click.option("--port", "-p", type=int, default=9090, help="Port for metrics server. Default: 9090. [optional]") +@click.option("--bind", "-b", type=str, default="127.0.0.1", help="Address to bind metrics server. Default: 127.0.0.1. [optional]") +@click.option("--cache-ttl", type=int, default=60, help="Seconds to cache status between scrapes in --serve mode. Default: 60. [optional]") +def db_metrics(config: str, verbose: bool, serve: bool, port: int, bind: str, cache_ttl: int): + """ + Output Prometheus metrics for monitoring. + + By default, outputs metrics to stdout for one-shot collection. + Use --serve to start a persistent HTTP server for Prometheus scraping. + """ + from cvdupdate.metrics import PrometheusMetrics + import http.server + import socketserver + import threading + import time + + m = CVDUpdate(config=config, verbose=verbose) + + if serve: + # Cache status between scrapes so each request does not trigger a live + # DNS query. Prometheus scrapes frequently, and db_status() performs + # both network and disk work. + cache_lock = threading.Lock() + status_cache = {'status': None, 'time': 0.0} + + def get_cached_status(): + now = time.time() + with cache_lock: + if status_cache['status'] is None or (now - status_cache['time']) > cache_ttl: + status_cache['status'] = m.db_status() + status_cache['time'] = now + return status_cache['status'] + + class MetricsHandler(http.server.BaseHTTPRequestHandler): + verbose_mode = verbose + + def do_GET(self): + if self.path == '/metrics' or self.path == '/': + status = get_cached_status() + metrics = PrometheusMetrics(status) + content = metrics.generate().encode('utf-8') + + self.send_response(200) + self.send_header('Content-Type', 'text/plain; version=0.0.4; charset=utf-8') + self.send_header('Content-Length', str(len(content))) + self.end_headers() + self.wfile.write(content) + elif self.path == '/health': + self.send_response(200) + self.send_header('Content-Type', 'text/plain') + self.end_headers() + self.wfile.write(b'OK') + else: + self.send_response(404) + self.end_headers() + + def log_message(self, format, *args): + if self.verbose_mode: + m.logger.debug(f'{self.address_string()} - {format % args}') + + m.logger.info(f'Starting metrics server on {bind}:{port}') + m.logger.info(f'Metrics available at http://{bind}:{port}/metrics') + + with socketserver.TCPServer((bind, port), MetricsHandler) as httpd: + try: + httpd.serve_forever() + except KeyboardInterrupt: + m.logger.info('Metrics server stopped') + else: + # One-shot output + status = m.db_status() + metrics = PrometheusMetrics(status) + click.echo(metrics.generate()) + + @cli.command("serve") @click.option("--config", "-c", type=click.Path(), required=False, default="", help="Config path. [optional]") @click.option("--verbose", "-V", is_flag=True, default=False, help="Verbose output. [optional]") diff --git a/cvdupdate/cvdupdate.py b/cvdupdate/cvdupdate.py index d419ddc..05f579c 100644 --- a/cvdupdate/cvdupdate.py +++ b/cvdupdate/cvdupdate.py @@ -320,8 +320,15 @@ def config_show(self): """ Print out the config """ + # Create a copy of config with masked password and proxy credentials + config_display = copy.deepcopy(self.config) + if 'proxy_pass' in config_display and config_display['proxy_pass']: + config_display['proxy_pass'] = '********' + if 'proxy_url' in config_display and config_display['proxy_url']: + config_display['proxy_url'] = self._sanitize_proxy_url(config_display['proxy_url']) + print(f"Config file: {self.config_path}\n") - print(f"Config:\n{json.dumps(self.config, indent=4)}\n") + print(f"Config:\n{json.dumps(config_display, indent=4)}\n") print(f"State file: {self.config['state file']}\n") print(f"State:\n{json.dumps(self.state, indent=4)}\n") @@ -513,6 +520,193 @@ def db_show(self, name) -> bool: self.logger.error(f"No such database: {name}") return found + def _calculate_age_status(self, last_modified: float) -> tuple: + ''' + Calculate age status based on last modification time. + + Args: + last_modified: Unix timestamp of last modification + + Returns: + tuple: (age_hours, age_status) + ''' + if last_modified == 0: + return (None, 'missing') + + age_seconds = time.time() - last_modified + age_hours = age_seconds / 3600 + + if age_hours < 24: + return (age_hours, 'current') + elif age_hours < 48: + return (age_hours, 'recent') + elif age_hours < 72: + return (age_hours, 'stale') + else: + return (age_hours, 'outdated') + + def db_status(self) -> dict: + ''' + Get status of all databases. + + Returns: + dict: Status report containing: + - summary: Overall health summary + - databases: Per-database status details + - warnings: List of issues requiring attention + ''' + dbs = self._index_local_databases() + warnings = [] + databases = [] + + # Try to get remote versions via DNS + dns_available = False + self.dns_version_tokens = [] + for _attempt in range(self.config.get('max retry', 3)): + if self._query_dns_txt_entry(): + dns_available = True + break + time.sleep(0.1) + + if not dns_available: + warnings.append('DNS query failed, unable to verify remote versions') + + current_count = 0 + stale_count = 0 + outdated_count = 0 + unknown_count = 0 + missing_count = 0 + cooldown_count = 0 + + for db_name in dbs: + db_info = dbs[db_name] + db_path = self.db_dir / db_name + + # Check if file exists + is_missing = not db_path.exists() + if is_missing: + missing_count += 1 + warnings.append(f'Database {db_name} is missing from disk') + + # Get local version + local_version = db_info.get('local version', 0) if db_name.endswith('.cvd') else None + + # Get remote version from DNS + remote_version = None + if dns_available and db_name.endswith('.cvd') and db_info.get('DNS field', 0) > 0: + try: + remote_version = int(self.dns_version_tokens[db_info['DNS field']]) + except (IndexError, ValueError): + pass + + # Determine version status. 'unknown' means we could not verify the + # remote version (for example, DNS was unavailable) and is reported + # distinctly from 'outdated' so callers do not treat the two alike. + is_current = False + if local_version is not None and remote_version is not None: + if local_version >= remote_version: + version_status = 'current' + is_current = True + else: + version_status = 'outdated' + warnings.append(f'Database {db_name} is behind: local v{local_version} vs remote v{remote_version}') + else: + version_status = 'unknown' + + # Calculate age + last_modified = db_info.get('last modified', 0) + age_hours, age_status = self._calculate_age_status(last_modified) + + # Tally version state. Age on its own does not mark a database + # unhealthy: some databases such as main.cvd change infrequently, so + # a current mirror can legitimately hold an old but correct file. + if not is_missing: + if version_status == 'current': + current_count += 1 + elif version_status == 'outdated': + outdated_count += 1 + else: + unknown_count += 1 + + # Only flag age staleness when the database is not confirmed current. + if age_status in ('stale', 'outdated') and version_status != 'current': + stale_count += 1 + if age_hours is not None: + warnings.append(f'Database {db_name} is {int(age_hours)} hours old and may be stale') + + # Check cooldown + on_cooldown = False + cooldown_until = None + retry_after = db_info.get('retry after', 0) + if retry_after > time.time(): + on_cooldown = True + cooldown_until = retry_after + cooldown_count += 1 + cooldown_str = datetime.datetime.fromtimestamp(retry_after).strftime('%Y-%m-%d %H:%M:%S') + warnings.append(f'Database {db_name} is on cooldown until {cooldown_str}') + + # Get file size + file_size_bytes = None + if db_path.exists(): + try: + file_size_bytes = db_path.stat().st_size + except Exception: + pass + + # Count CDIFFs + cdiff_count = len(db_info.get('CDIFFs', [])) + + # Calculate age in seconds for metrics + age_seconds = None + if last_modified > 0: + age_seconds = time.time() - last_modified + + databases.append({ + 'name': db_name, + 'local_version': local_version, + 'remote_version': remote_version, + 'is_current': is_current, + 'version_status': version_status, + 'is_missing': is_missing, + 'last_modified': last_modified, + 'age_hours': age_hours, + 'age_seconds': age_seconds, + 'age_status': age_status, + 'on_cooldown': on_cooldown, + 'cooldown_until': cooldown_until, + 'cdiff_count': cdiff_count, + 'file_size_bytes': file_size_bytes, + }) + + # Determine overall status. A missing database is critical because the + # mirror cannot serve it. Being behind on version, on cooldown, or aged + # is a warning. An unverifiable version does not warn on its own, since + # non-CVD databases have no version to check; it surfaces through the + # DNS-failure or age warnings when it actually matters. + total_databases = len(dbs) + if missing_count > 0: + overall_status = 'critical' + elif outdated_count > 0 or cooldown_count > 0 or stale_count > 0 or len(warnings) > 0: + overall_status = 'warning' + else: + overall_status = 'healthy' + + return { + 'summary': { + 'total_databases': total_databases, + 'current_count': current_count, + 'stale_count': stale_count, + 'outdated_count': outdated_count, + 'unknown_count': unknown_count, + 'missing_count': missing_count, + 'cooldown_count': cooldown_count, + 'overall_status': overall_status, + 'last_check': time.time(), + }, + 'databases': databases, + 'warnings': warnings, + } + def _query_dns_txt_entry(self) -> bool: ''' Attempt to get version from current.cvd.clamav.net DNS TXT entry @@ -581,6 +775,77 @@ def _get_nameserver_string(self) -> str: return "" + @staticmethod + def _sanitize_proxy_url(url: str) -> str: + ''' + Return the proxy URL with any embedded userinfo masked, so that + credentials are never written to logs or printed by `config show`. + ''' + if not url: + return url + + from urllib.parse import urlparse, urlunparse + + parsed = urlparse(url) + if not parsed.username and not parsed.password: + return url + + host = parsed.hostname or '' + if parsed.port: + host = f'{host}:{parsed.port}' + netloc = f'***:***@{host}' + return urlunparse((parsed.scheme, netloc, parsed.path, parsed.params, parsed.query, parsed.fragment)) + + def _get_proxy_configuration(self) -> Optional[dict]: + ''' + Get proxy configuration from environment variables or config file. + Environment variables take precedence over config file settings. + + If proxy credentials are configured, they are embedded in the proxy + URL so that the requests library sends the Proxy-Authorization header. + + Returns: + dict: Proxy configuration for requests, or None if not configured. + ''' + from urllib.parse import urlparse, urlunparse, quote + + proxy_url = os.environ.get('CVDUPDATE_PROXY_URL') + + if not proxy_url: + proxy_url = self.config.get('proxy_url') + + if not proxy_url: + return None + + if not urlparse(proxy_url).scheme: + self.logger.warning( + "Proxy URL is missing a scheme such as 'http://', so the proxy may be ignored." + ) + + proxy_user = os.environ.get('CVDUPDATE_PROXY_USER') + proxy_pass = os.environ.get('CVDUPDATE_PROXY_PASS') + + if not proxy_user: + proxy_user = self.config.get('proxy_user') + if not proxy_pass: + proxy_pass = self.config.get('proxy_pass') + + if proxy_user and proxy_pass: + parsed = urlparse(proxy_url) + if not parsed.username: + netloc = f'{quote(proxy_user, safe="")}:{quote(proxy_pass, safe="")}@{parsed.hostname}' + if parsed.port: + netloc += f':{parsed.port}' + proxy_url = urlunparse((parsed.scheme, netloc, parsed.path, parsed.params, parsed.query, parsed.fragment)) + self.logger.info(f'Using authenticated proxy: {self._sanitize_proxy_url(proxy_url)}') + else: + self.logger.info(f'Using proxy: {self._sanitize_proxy_url(proxy_url)}') + + return { + 'http': proxy_url, + 'https': proxy_url, + } + def _query_cvd_version_dns(self, db: str) -> int: ''' This is a faux query. @@ -634,6 +899,9 @@ def _query_cvd_version_http(self, db: str) -> int: ims = datetime.datetime.fromtimestamp(self.state['dbs'][db]['last modified'], tz=datetime.timezone.utc).strftime('%a, %d %b %Y %H:%M:%S GMT') + # Get proxy configuration + proxies = self._get_proxy_configuration() + retry = 0 response = None while retry < self.config['max retry']: @@ -641,7 +909,7 @@ def _query_cvd_version_http(self, db: str) -> int: 'User-Agent': f'CVDUPDATE/{self.version} ({self.state["uuid"]})', 'Range': 'bytes=0-95', 'If-Modified-Since': ims, - }) + }, proxies=proxies) if ((response.status_code == 200 or response.status_code == 206) and ('content-length' in response.headers) and @@ -706,13 +974,16 @@ def _download_db_from_url(self, db: str, url: str, last_modified: int, version=0 ''' ims: str = datetime.datetime.fromtimestamp(last_modified, tz=datetime.timezone.utc).strftime('%a, %d %b %Y %H:%M:%S GMT') + # Get proxy configuration + proxies = self._get_proxy_configuration() + retry = 0 response = None while retry < self.config['max retry']: response = requests.get(url, headers = { 'User-Agent': f'CVDUPDATE/{self.version} ({self.state["uuid"]})', 'If-Modified-Since': ims, - }) + }, proxies=proxies) if ((response.status_code == 200 or response.status_code == 206) and ('content-length' in response.headers) and @@ -812,12 +1083,15 @@ def _download_cdiff(self, db: str, file: str, db_url: str, last_modified: int, d base_url = db_url.rsplit('/', 1)[0] url = f"{base_url}/{file}" + # Get proxy configuration + proxies = self._get_proxy_configuration() + retry = 0 response = None while retry < self.config['max retry']: response = requests.get(url, headers = { 'User-Agent': f'CVDUPDATE/{self.version} ({self.state["uuid"]})', - }) + }, proxies=proxies) if ((response.status_code == 200 or response.status_code == 206) and ('content-length' in response.headers) and @@ -922,13 +1196,16 @@ def _download_sign_file_for(self, file: str, file_url: str, last_modified: int, base_url = file_url.rsplit('/', 1)[0] url = f"{base_url}/{sign_file}" + # Get proxy configuration + proxies = self._get_proxy_configuration() + retry = 0 response = None while retry < self.config['max retry']: response = requests.get(url, headers = { 'User-Agent': f'CVDUPDATE/{self.version} ({self.state["uuid"]})', 'If-Modified-Since': ims, - }) + }, proxies=proxies) if ((response.status_code == 200 or response.status_code == 206) and ('content-length' in response.headers) and @@ -1107,7 +1384,10 @@ def check(name): current_version_str = _get_version(name) current_version = version.parse(current_version_str) - response = requests.get(f"https://pypi.org/pypi/{name}/json") # Get package info + # Get proxy configuration + proxies = self._get_proxy_configuration() + + response = requests.get(f"https://pypi.org/pypi/{name}/json", proxies=proxies) response.raise_for_status() # Raise an exception for bad status codes (4xx or 5xx) latest_version_str = response.json()["info"]["version"] latest_version = version.parse(latest_version_str) diff --git a/cvdupdate/metrics.py b/cvdupdate/metrics.py new file mode 100644 index 0000000..f10e665 --- /dev/null +++ b/cvdupdate/metrics.py @@ -0,0 +1,153 @@ +""" +Prometheus metrics generation for CVD-Update. + +Copyright (C) 2021-2025 Cisco Systems, Inc. and/or its affiliates. All rights reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +""" + +import time +from typing import Dict, Any, List + + +METRIC_PREFIX = 'cvdupdate' + + +class PrometheusMetrics: + """Generate Prometheus-format metrics from CVD-Update status.""" + + def __init__(self, status: Dict[str, Any]): + """ + Initialize metrics generator with status data. + + Args: + status: Status dict from CVDUpdate.db_status() + """ + self.status = status + # Unix timestamp in seconds, per the Prometheus convention for *_timestamp. + self.timestamp = int(time.time()) + + @staticmethod + def _escape_label_value(value: str) -> str: + ''' + Escape a label value per the Prometheus text exposition format: + backslash, double quote, and newline must be escaped. + ''' + return ( + str(value) + .replace('\\', '\\\\') + .replace('"', '\\"') + .replace('\n', '\\n') + ) + + def generate(self) -> str: + """ + Generate Prometheus metrics text. + + Returns: + str: Prometheus exposition format text + """ + lines: List[str] = [] + + # Add metadata comments + lines.append('# HELP cvdupdate_database_version Local version of ClamAV database') + lines.append('# TYPE cvdupdate_database_version gauge') + + lines.append('# HELP cvdupdate_database_remote_version Remote version available') + lines.append('# TYPE cvdupdate_database_remote_version gauge') + + lines.append('# HELP cvdupdate_database_age_seconds Age of database in seconds') + lines.append('# TYPE cvdupdate_database_age_seconds gauge') + + lines.append('# HELP cvdupdate_database_current Database is current (1) or outdated (0)') + lines.append('# TYPE cvdupdate_database_current gauge') + + lines.append('# HELP cvdupdate_database_missing Database is missing (1) or present (0)') + lines.append('# TYPE cvdupdate_database_missing gauge') + + lines.append('# HELP cvdupdate_database_cooldown Database is on cooldown (1) or not (0)') + lines.append('# TYPE cvdupdate_database_cooldown gauge') + + lines.append('# HELP cvdupdate_database_size_bytes Size of database file in bytes') + lines.append('# TYPE cvdupdate_database_size_bytes gauge') + + lines.append('# HELP cvdupdate_database_cdiff_count Number of CDIFF patch files') + lines.append('# TYPE cvdupdate_database_cdiff_count gauge') + + lines.append('# HELP cvdupdate_databases_total Total number of configured databases') + lines.append('# TYPE cvdupdate_databases_total gauge') + + lines.append('# HELP cvdupdate_databases_current Number of current databases') + lines.append('# TYPE cvdupdate_databases_current gauge') + + lines.append('# HELP cvdupdate_databases_stale Number of stale databases') + lines.append('# TYPE cvdupdate_databases_stale gauge') + + lines.append('# HELP cvdupdate_databases_missing Number of missing databases') + lines.append('# TYPE cvdupdate_databases_missing gauge') + + lines.append('# HELP cvdupdate_health_status Overall health (0=critical, 1=warning, 2=healthy)') + lines.append('# TYPE cvdupdate_health_status gauge') + + lines.append('# HELP cvdupdate_last_check_timestamp Unix timestamp of last status check') + lines.append('# TYPE cvdupdate_last_check_timestamp gauge') + + # Per-database metrics + for db in self.status.get('databases', []): + name = self._escape_label_value(db['name']) + labels = f'database="{name}"' + + if db.get('local_version') is not None: + lines.append(f'cvdupdate_database_version{{{labels}}} {db["local_version"]}') + + if db.get('remote_version') is not None: + lines.append(f'cvdupdate_database_remote_version{{{labels}}} {db["remote_version"]}') + + if db.get('age_seconds') is not None: + lines.append(f'cvdupdate_database_age_seconds{{{labels}}} {db["age_seconds"]:.0f}') + + # Only emit the current gauge when the version was verified, so an + # unknown state (for example, DNS unavailable) is not reported as 0. + if db.get('version_status', 'unknown') != 'unknown': + current = 1 if db.get('is_current') else 0 + lines.append(f'cvdupdate_database_current{{{labels}}} {current}') + + missing = 1 if db.get('is_missing') else 0 + lines.append(f'cvdupdate_database_missing{{{labels}}} {missing}') + + cooldown = 1 if db.get('on_cooldown') else 0 + lines.append(f'cvdupdate_database_cooldown{{{labels}}} {cooldown}') + + if db.get('file_size_bytes') is not None: + lines.append(f'cvdupdate_database_size_bytes{{{labels}}} {db["file_size_bytes"]}') + + if db.get('cdiff_count') is not None: + lines.append(f'cvdupdate_database_cdiff_count{{{labels}}} {db["cdiff_count"]}') + + # Summary metrics + summary = self.status.get('summary', {}) + + lines.append(f'cvdupdate_databases_total {summary.get("total_databases", 0)}') + lines.append(f'cvdupdate_databases_current {summary.get("current_count", 0)}') + lines.append(f'cvdupdate_databases_stale {summary.get("stale_count", 0)}') + lines.append(f'cvdupdate_databases_missing {summary.get("missing_count", 0)}') + + # Map status to numeric value + status_map = {'critical': 0, 'warning': 1, 'healthy': 2} + health_value = status_map.get(summary.get('overall_status', 'critical'), 0) + lines.append(f'cvdupdate_health_status {health_value}') + + lines.append(f'cvdupdate_last_check_timestamp {self.timestamp}') + + return '\n'.join(lines) + '\n' + diff --git a/setup.py b/setup.py index 992e397..93ecd9b 100644 --- a/setup.py +++ b/setup.py @@ -5,7 +5,7 @@ setuptools.setup( name="cvdupdate", - version="1.2.0", + version="1.3.0", author="The ClamAV Team", author_email="clamav-bugs@external.cisco.com", copyright="Copyright (C) 2025 Cisco Systems, Inc. and/or its affiliates. All rights reserved.", @@ -29,6 +29,7 @@ "rangehttpserver", "packaging", ], + extras_require={}, classifiers=[ "Programming Language :: Python :: 3", "License :: OSI Approved :: Apache Software License", diff --git a/tests/test_cli.py b/tests/test_cli.py new file mode 100644 index 0000000..23ad1b9 --- /dev/null +++ b/tests/test_cli.py @@ -0,0 +1,124 @@ +""" +Tests that exercise the Click command-line layer end to end. + +These complement the unit tests, which call methods directly. The CLI layer +(option parsing, defaults, and validation) is where the earlier `config set` +regression lived, so it needs direct coverage. + +Copyright (C) 2021-2025 Cisco Systems, Inc. and/or its affiliates. All rights reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +""" + +import json +from unittest import mock + +import pytest +from click.testing import CliRunner + +from tests.fixtures.revert import revert_homedir +from cvdupdate.__main__ import cli + + +def _make_config(runner, tmp_path): + """Create a config that points logs and databases at the temp dir.""" + cfg = tmp_path / 'config.json' + result = runner.invoke(cli, [ + 'config', 'set', + '--config', str(cfg), + '--logdir', str(tmp_path / 'logs'), + '--dbdir', str(tmp_path / 'db'), + ]) + assert result.exit_code == 0, result.output + return cfg + + +class TestConfigSet: + """Tests for `cvd config set` option handling.""" + + def test_succeeds_without_proxy_options(self, revert_homedir, tmp_path): + """config set must work when only directories are provided.""" + runner = CliRunner() + cfg = _make_config(runner, tmp_path) + assert cfg.exists() + + def test_persists_proxy_url(self, revert_homedir, tmp_path): + """A provided proxy URL is written to the config file.""" + runner = CliRunner() + cfg = tmp_path / 'config.json' + result = runner.invoke(cli, [ + 'config', 'set', + '--config', str(cfg), + '--logdir', str(tmp_path / 'logs'), + '--dbdir', str(tmp_path / 'db'), + '--proxy-url', 'http://proxy.example.com:8080', + ]) + assert result.exit_code == 0, result.output + data = json.loads(cfg.read_text()) + assert data['proxy_url'] == 'http://proxy.example.com:8080' + + def test_proxy_cert_option_is_removed(self, revert_homedir, tmp_path): + """The proxy client-cert option no longer exists.""" + runner = CliRunner() + result = runner.invoke(cli, [ + 'config', 'set', + '--config', str(tmp_path / 'config.json'), + '--logdir', str(tmp_path / 'logs'), + '--dbdir', str(tmp_path / 'db'), + '--proxy-cert', '/does/not/exist', + ]) + assert result.exit_code != 0 + assert 'no such option' in result.output.lower() + + +class TestStatusCli: + """Tests for `cvd status`.""" + + def test_status_runs(self, revert_homedir, tmp_path): + runner = CliRunner() + cfg = _make_config(runner, tmp_path) + with mock.patch('cvdupdate.cvdupdate.CVDUpdate._query_dns_txt_entry', return_value=False): + result = runner.invoke(cli, ['status', '--config', str(cfg)]) + assert result.exit_code == 0, result.output + assert 'Database Status' in result.output + + def test_status_check_exits_critical_when_all_missing(self, revert_homedir, tmp_path): + """With no databases downloaded, --check reports critical (exit 2).""" + runner = CliRunner() + cfg = _make_config(runner, tmp_path) + with mock.patch('cvdupdate.cvdupdate.CVDUpdate._query_dns_txt_entry', return_value=False): + result = runner.invoke(cli, ['status', '--config', str(cfg), '--check']) + assert result.exit_code == 2 + + def test_status_json_with_check(self, revert_homedir, tmp_path): + """--json and --check together must not raise.""" + runner = CliRunner() + cfg = _make_config(runner, tmp_path) + with mock.patch('cvdupdate.cvdupdate.CVDUpdate._query_dns_txt_entry', return_value=False): + result = runner.invoke(cli, ['status', '--config', str(cfg), '--json', '--check']) + # Exit code reflects health, but the command must not crash. + assert result.exit_code in (0, 1, 2) + payload = json.loads(result.output) + assert 'summary' in payload + + +class TestMetricsCli: + """Tests for `cvd metrics` one-shot output.""" + + def test_metrics_runs(self, revert_homedir, tmp_path): + runner = CliRunner() + cfg = _make_config(runner, tmp_path) + with mock.patch('cvdupdate.cvdupdate.CVDUpdate._query_dns_txt_entry', return_value=False): + result = runner.invoke(cli, ['metrics', '--config', str(cfg)]) + assert result.exit_code == 0, result.output + assert 'cvdupdate_databases_total' in result.output diff --git a/tests/test_metrics.py b/tests/test_metrics.py new file mode 100644 index 0000000..7dd072d --- /dev/null +++ b/tests/test_metrics.py @@ -0,0 +1,376 @@ +""" +Tests for Prometheus metrics generation. + +Copyright (C) 2021-2025 Cisco Systems, Inc. and/or its affiliates. All rights reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +""" + +import re +from unittest import mock + +import pytest + +from tests.fixtures.revert import revert_homedir +from cvdupdate.cvdupdate import CVDUpdate +from cvdupdate.metrics import PrometheusMetrics + + +class TestPrometheusMetricsGenerate: + """Tests for PrometheusMetrics.generate method.""" + + def test_generates_valid_prometheus_format(self): + """Test that output is valid Prometheus exposition format.""" + status = { + 'summary': { + 'total_databases': 3, + 'current_count': 2, + 'stale_count': 1, + 'missing_count': 0, + 'cooldown_count': 0, + 'overall_status': 'warning', + 'last_check': 1704556800.0, + }, + 'databases': [ + { + 'name': 'main.cvd', + 'version_status': 'current', + 'local_version': 62, + 'remote_version': 62, + 'is_current': True, + 'is_missing': False, + 'age_seconds': 3600, + 'on_cooldown': False, + 'file_size_bytes': 157286400, + 'cdiff_count': 5, + }, + ], + 'warnings': [], + } + + metrics = PrometheusMetrics(status) + output = metrics.generate() + + # Should end with newline + assert output.endswith('\n') + + # Should contain HELP and TYPE comments + assert '# HELP' in output + assert '# TYPE' in output + + # Should contain metric names + assert 'cvdupdate_database_version' in output + assert 'cvdupdate_databases_total' in output + assert 'cvdupdate_health_status' in output + + def test_includes_all_expected_metrics(self): + """Test that all expected metrics are present.""" + status = { + 'summary': { + 'total_databases': 1, + 'current_count': 1, + 'stale_count': 0, + 'missing_count': 0, + 'cooldown_count': 0, + 'overall_status': 'healthy', + 'last_check': 1704556800.0, + }, + 'databases': [ + { + 'name': 'test.cvd', + 'version_status': 'current', + 'local_version': 100, + 'remote_version': 100, + 'is_current': True, + 'is_missing': False, + 'age_seconds': 1800, + 'on_cooldown': False, + 'file_size_bytes': 1024, + 'cdiff_count': 2, + }, + ], + 'warnings': [], + } + + metrics = PrometheusMetrics(status) + output = metrics.generate() + + expected_metrics = [ + 'cvdupdate_database_version', + 'cvdupdate_database_remote_version', + 'cvdupdate_database_age_seconds', + 'cvdupdate_database_current', + 'cvdupdate_database_missing', + 'cvdupdate_database_cooldown', + 'cvdupdate_database_size_bytes', + 'cvdupdate_database_cdiff_count', + 'cvdupdate_databases_total', + 'cvdupdate_databases_current', + 'cvdupdate_databases_stale', + 'cvdupdate_databases_missing', + 'cvdupdate_health_status', + 'cvdupdate_last_check_timestamp', + ] + + for metric in expected_metrics: + assert metric in output, f'Missing metric: {metric}' + + def test_labels_are_correctly_formatted(self): + """Test that labels use correct quoting.""" + status = { + 'summary': { + 'total_databases': 1, + 'current_count': 1, + 'stale_count': 0, + 'missing_count': 0, + 'cooldown_count': 0, + 'overall_status': 'healthy', + 'last_check': 1704556800.0, + }, + 'databases': [ + { + 'name': 'daily.cvd', + 'version_status': 'current', + 'local_version': 27456, + 'remote_version': 27456, + 'is_current': True, + 'is_missing': False, + 'age_seconds': 3600, + 'on_cooldown': False, + 'file_size_bytes': 60817408, + 'cdiff_count': 10, + }, + ], + 'warnings': [], + } + + metrics = PrometheusMetrics(status) + output = metrics.generate() + + # Labels should use double quotes + assert 'database="daily.cvd"' in output + + def test_health_status_maps_correctly(self): + """Test that health status maps to correct numeric values.""" + test_cases = [ + ('healthy', 2), + ('warning', 1), + ('critical', 0), + ] + + for status_str, expected_value in test_cases: + status = { + 'summary': { + 'total_databases': 1, + 'current_count': 1, + 'stale_count': 0, + 'missing_count': 0, + 'cooldown_count': 0, + 'overall_status': status_str, + 'last_check': 1704556800.0, + }, + 'databases': [], + 'warnings': [], + } + + metrics = PrometheusMetrics(status) + output = metrics.generate() + + assert f'cvdupdate_health_status {expected_value}' in output + + def test_handles_none_values_gracefully(self): + """Test that None values are handled correctly.""" + status = { + 'summary': { + 'total_databases': 1, + 'current_count': 0, + 'stale_count': 0, + 'missing_count': 1, + 'cooldown_count': 0, + 'overall_status': 'critical', + 'last_check': 1704556800.0, + }, + 'databases': [ + { + 'name': 'missing.cvd', + 'local_version': None, + 'remote_version': None, + 'is_current': False, + 'is_missing': True, + 'age_seconds': None, + 'on_cooldown': False, + 'file_size_bytes': None, + 'cdiff_count': 0, + }, + ], + 'warnings': [], + } + + metrics = PrometheusMetrics(status) + output = metrics.generate() + + # Should not contain 'None' as text + assert 'None' not in output + + # Should still have the missing metric + assert 'cvdupdate_database_missing{database="missing.cvd"} 1' in output + + +class TestMetricsIntegration: + """Integration tests for metrics with CVDUpdate.""" + + def test_metrics_from_cvdupdate_status(self, revert_homedir): + """Test generating metrics from actual CVDUpdate status.""" + c = CVDUpdate() + + with mock.patch.object(c, '_query_dns_txt_entry', return_value=False): + status = c.db_status() + + metrics = PrometheusMetrics(status) + output = metrics.generate() + + # Should be non-empty + assert len(output) > 0 + + # The missing gauge is emitted once per database, so it tracks count. + db_count = output.count('cvdupdate_database_missing{') + assert db_count == len(status['databases']) + + def test_metrics_output_matches_status_values(self, revert_homedir): + """Test that metrics output matches status dict values.""" + c = CVDUpdate() + + with mock.patch.object(c, '_query_dns_txt_entry', return_value=False): + status = c.db_status() + + metrics = PrometheusMetrics(status) + output = metrics.generate() + + # Check summary metrics match + total = status['summary']['total_databases'] + assert f'cvdupdate_databases_total {total}' in output + + current = status['summary']['current_count'] + assert f'cvdupdate_databases_current {current}' in output + + stale = status['summary']['stale_count'] + assert f'cvdupdate_databases_stale {stale}' in output + + missing = status['summary']['missing_count'] + assert f'cvdupdate_databases_missing {missing}' in output + + +class TestMetricsLabelEscaping: + """Tests for Prometheus label value escaping.""" + + def _status_with_name(self, name): + return { + 'summary': { + 'total_databases': 1, + 'current_count': 1, + 'stale_count': 0, + 'missing_count': 0, + 'cooldown_count': 0, + 'overall_status': 'healthy', + 'last_check': 1704556800.0, + }, + 'databases': [ + { + 'name': name, + 'version_status': 'current', + 'local_version': 1, + 'remote_version': 1, + 'is_current': True, + 'is_missing': False, + 'age_seconds': 1, + 'on_cooldown': False, + 'file_size_bytes': 1, + 'cdiff_count': 0, + }, + ], + 'warnings': [], + } + + def test_escapes_backslash_quote_and_newline(self): + """Label values must escape backslash, double quote, and newline.""" + output = PrometheusMetrics(self._status_with_name('a\\b"c\nd.cvd')).generate() + assert 'database="a\\\\b\\"c\\nd.cvd"' in output + # A raw newline must never appear inside a label value. + for line in output.splitlines(): + assert not (line.startswith('cvdupdate_database') and line.endswith('database="a')) + + def test_plain_name_is_unchanged(self): + """A name without special characters is emitted verbatim.""" + output = PrometheusMetrics(self._status_with_name('daily.cvd')).generate() + assert 'database="daily.cvd"' in output + + +class TestMetricsTimestampUnit: + """Tests for the last-check timestamp unit.""" + + def test_timestamp_is_seconds_not_milliseconds(self): + """The timestamp must be Unix seconds (10 digits), not milliseconds.""" + status = { + 'summary': { + 'total_databases': 0, + 'current_count': 0, + 'stale_count': 0, + 'missing_count': 0, + 'cooldown_count': 0, + 'overall_status': 'healthy', + 'last_check': 1704556800.0, + }, + 'databases': [], + 'warnings': [], + } + output = PrometheusMetrics(status).generate() + line = next(l for l in output.splitlines() if l.startswith('cvdupdate_last_check_timestamp ')) + value = int(line.split()[1]) + # Seconds are ~10 digits through the year 2286; milliseconds are ~13. + assert 10 ** 9 <= value < 10 ** 11 + + +class TestMetricsUnknownVersion: + """Tests that an unverified version is not reported as outdated.""" + + def test_current_gauge_omitted_when_unknown(self): + """The current gauge is skipped when the version could not be verified.""" + status = { + 'summary': { + 'total_databases': 1, + 'current_count': 0, + 'stale_count': 0, + 'missing_count': 0, + 'cooldown_count': 0, + 'overall_status': 'warning', + 'last_check': 1704556800.0, + }, + 'databases': [ + { + 'name': 'main.cvd', + 'version_status': 'unknown', + 'local_version': 62, + 'remote_version': None, + 'is_current': False, + 'is_missing': False, + 'age_seconds': 1, + 'on_cooldown': False, + 'file_size_bytes': 1, + 'cdiff_count': 0, + }, + ], + 'warnings': [], + } + output = PrometheusMetrics(status).generate() + assert 'cvdupdate_database_current{database="main.cvd"}' not in output + diff --git a/tests/test_proxy_auth.py b/tests/test_proxy_auth.py new file mode 100644 index 0000000..0055c85 --- /dev/null +++ b/tests/test_proxy_auth.py @@ -0,0 +1,182 @@ +""" +Tests for proxy authentication support. + +Copyright (C) 2021-2025 Cisco Systems, Inc. and/or its affiliates. All rights reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +""" + +import os +from unittest import mock + +import pytest + +from tests.fixtures.revert import revert_homedir +from cvdupdate.cvdupdate import CVDUpdate + + +class TestGetProxyConfiguration: + """Tests for _get_proxy_configuration method.""" + + def test_returns_none_when_not_configured(self, revert_homedir): + """Test that None is returned when no proxy is configured.""" + c = CVDUpdate() + result = c._get_proxy_configuration() + assert result is None + + def test_returns_proxy_from_config(self, revert_homedir): + """Test that proxy URL is returned from config.""" + c = CVDUpdate() + c.config['proxy_url'] = 'http://proxy.example.com:8080' + result = c._get_proxy_configuration() + assert result == { + 'http': 'http://proxy.example.com:8080', + 'https': 'http://proxy.example.com:8080', + } + + def test_env_var_takes_precedence(self, revert_homedir): + """Test that environment variable takes precedence over config.""" + c = CVDUpdate() + c.config['proxy_url'] = 'http://config-proxy.example.com:8080' + + with mock.patch.dict(os.environ, {'CVDUPDATE_PROXY_URL': 'http://env-proxy.example.com:3128'}): + result = c._get_proxy_configuration() + assert result == { + 'http': 'http://env-proxy.example.com:3128', + 'https': 'http://env-proxy.example.com:3128', + } + + def test_embeds_credentials_in_proxy_url(self, revert_homedir): + """Test that user/pass are embedded in the proxy URL for Proxy-Authorization.""" + c = CVDUpdate() + c.config['proxy_url'] = 'http://proxy.example.com:8080' + c.config['proxy_user'] = 'testuser' + c.config['proxy_pass'] = 'testpass' + result = c._get_proxy_configuration() + assert result == { + 'http': 'http://testuser:testpass@proxy.example.com:8080', + 'https': 'http://testuser:testpass@proxy.example.com:8080', + } + + def test_url_encodes_special_chars_in_credentials(self, revert_homedir): + """Test that special characters in credentials are URL-encoded.""" + c = CVDUpdate() + c.config['proxy_url'] = 'http://proxy.example.com:8080' + c.config['proxy_user'] = 'user@domain' + c.config['proxy_pass'] = 'p@ss:word/123' + result = c._get_proxy_configuration() + assert 'user%40domain' in result['http'] + assert 'p%40ss%3Aword%2F123' in result['http'] + assert '@proxy.example.com:8080' in result['http'] + + def test_does_not_double_embed_credentials(self, revert_homedir): + """Test that credentials already in the URL are not duplicated.""" + c = CVDUpdate() + c.config['proxy_url'] = 'http://existinguser:existingpass@proxy.example.com:8080' + c.config['proxy_user'] = 'newuser' + c.config['proxy_pass'] = 'newpass' + result = c._get_proxy_configuration() + assert 'existinguser:existingpass@' in result['http'] + assert 'newuser' not in result['http'] + + def test_env_credentials_take_precedence(self, revert_homedir): + """Test that env var credentials take precedence over config.""" + c = CVDUpdate() + c.config['proxy_url'] = 'http://proxy.example.com:8080' + c.config['proxy_user'] = 'config_user' + c.config['proxy_pass'] = 'config_pass' + + with mock.patch.dict(os.environ, { + 'CVDUPDATE_PROXY_USER': 'env_user', + 'CVDUPDATE_PROXY_PASS': 'env_pass', + }): + result = c._get_proxy_configuration() + assert 'env_user:env_pass@' in result['http'] + + def test_no_credentials_without_both_user_and_pass(self, revert_homedir): + """Test that credentials are not embedded when only user or only pass is set.""" + c = CVDUpdate() + c.config['proxy_url'] = 'http://proxy.example.com:8080' + c.config['proxy_user'] = 'testuser' + result = c._get_proxy_configuration() + assert 'testuser' not in result['http'] + assert result == { + 'http': 'http://proxy.example.com:8080', + 'https': 'http://proxy.example.com:8080', + } + + +class TestPasswordMasking: + """Tests for password masking in config show.""" + + def test_config_show_masks_password(self, revert_homedir, capsys): + """Test that password is masked in config show output.""" + c = CVDUpdate() + c.config['proxy_pass'] = 'supersecret' + c.config_show() + captured = capsys.readouterr() + assert 'supersecret' not in captured.out + assert '********' in captured.out + + def test_config_show_masks_credentials_in_proxy_url(self, revert_homedir, capsys): + """Credentials embedded directly in proxy_url must be masked too.""" + c = CVDUpdate() + c.config['proxy_url'] = 'http://user:supersecret@proxy.example.com:8080' + c.config_show() + captured = capsys.readouterr() + assert 'supersecret' not in captured.out + assert '***:***@proxy.example.com:8080' in captured.out + + +class TestSanitizeProxyUrl: + """Tests for the _sanitize_proxy_url helper.""" + + def test_masks_userinfo(self, revert_homedir): + url = 'http://user:secret@proxy.example.com:8080/path' + sanitized = CVDUpdate._sanitize_proxy_url(url) + assert 'secret' not in sanitized + assert sanitized == 'http://***:***@proxy.example.com:8080/path' + + def test_leaves_url_without_userinfo_unchanged(self, revert_homedir): + url = 'http://proxy.example.com:8080' + assert CVDUpdate._sanitize_proxy_url(url) == url + + +class TestProxyLogging: + """Tests that proxy credentials are never written to the logs.""" + + def test_log_does_not_leak_credentials_in_proxy_url(self, revert_homedir): + import logging + + c = CVDUpdate() + c.config['proxy_url'] = 'http://user:supersecret@proxy.example.com:8080' + + messages = [] + + class _Capture(logging.Handler): + def emit(self, record): + messages.append(record.getMessage()) + + handler = _Capture() + c.logger.addHandler(handler) + try: + result = c._get_proxy_configuration() + finally: + c.logger.removeHandler(handler) + + # The returned proxy URL still carries the real credentials for use. + assert 'supersecret' in result['http'] + # Nothing logged may contain the secret. + assert all('supersecret' not in msg for msg in messages) + assert any('***:***@proxy.example.com:8080' in msg for msg in messages) + diff --git a/tests/test_status.py b/tests/test_status.py new file mode 100644 index 0000000..8733cf6 --- /dev/null +++ b/tests/test_status.py @@ -0,0 +1,298 @@ +""" +Tests for database status command. + +Copyright (C) 2021-2025 Cisco Systems, Inc. and/or its affiliates. All rights reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +""" + +import json +import time +from unittest import mock + +import pytest + +from tests.fixtures.revert import revert_homedir +from cvdupdate.cvdupdate import CVDUpdate + + +class TestCalculateAgeStatus: + """Tests for _calculate_age_status method.""" + + def test_returns_missing_for_zero_timestamp(self, revert_homedir): + """Test that missing status is returned for zero timestamp.""" + c = CVDUpdate() + age_hours, age_status = c._calculate_age_status(0) + assert age_hours is None + assert age_status == 'missing' + + def test_returns_current_for_recent_files(self, revert_homedir): + """Test that current status is returned for files less than 24 hours old.""" + c = CVDUpdate() + # 12 hours ago + last_modified = time.time() - (12 * 3600) + age_hours, age_status = c._calculate_age_status(last_modified) + assert 11 < age_hours < 13 + assert age_status == 'current' + + def test_returns_recent_for_24_to_48_hours(self, revert_homedir): + """Test that recent status is returned for files 24-48 hours old.""" + c = CVDUpdate() + # 36 hours ago + last_modified = time.time() - (36 * 3600) + age_hours, age_status = c._calculate_age_status(last_modified) + assert 35 < age_hours < 37 + assert age_status == 'recent' + + def test_returns_stale_for_48_to_72_hours(self, revert_homedir): + """Test that stale status is returned for files 48-72 hours old.""" + c = CVDUpdate() + # 60 hours ago + last_modified = time.time() - (60 * 3600) + age_hours, age_status = c._calculate_age_status(last_modified) + assert 59 < age_hours < 61 + assert age_status == 'stale' + + def test_returns_outdated_for_over_72_hours(self, revert_homedir): + """Test that outdated status is returned for files over 72 hours old.""" + c = CVDUpdate() + # 96 hours ago + last_modified = time.time() - (96 * 3600) + age_hours, age_status = c._calculate_age_status(last_modified) + assert 95 < age_hours < 97 + assert age_status == 'outdated' + + +class TestDbStatus: + """Tests for db_status method.""" + + def test_returns_correct_structure(self, revert_homedir): + """Test that db_status returns the expected structure.""" + c = CVDUpdate() + + # Mock DNS query to avoid network calls + with mock.patch.object(c, '_query_dns_txt_entry', return_value=False): + status = c.db_status() + + assert 'summary' in status + assert 'databases' in status + assert 'warnings' in status + + # Check summary structure + summary = status['summary'] + assert 'total_databases' in summary + assert 'current_count' in summary + assert 'stale_count' in summary + assert 'missing_count' in summary + assert 'cooldown_count' in summary + assert 'overall_status' in summary + assert 'last_check' in summary + + # Check databases is a list + assert isinstance(status['databases'], list) + + # Check warnings is a list + assert isinstance(status['warnings'], list) + + def test_detects_missing_databases(self, revert_homedir): + """Test that missing databases are detected.""" + c = CVDUpdate() + + # Databases don't exist since we haven't downloaded them + with mock.patch.object(c, '_query_dns_txt_entry', return_value=False): + status = c.db_status() + + # All databases should be reported as missing + assert status['summary']['missing_count'] == len(status['databases']) + + def test_json_output_is_valid(self, revert_homedir): + """Test that status can be serialized to valid JSON.""" + c = CVDUpdate() + + with mock.patch.object(c, '_query_dns_txt_entry', return_value=False): + status = c.db_status() + + # Should not raise + json_str = json.dumps(status) + assert json_str + + # Should be valid JSON + parsed = json.loads(json_str) + assert parsed == status + + def test_overall_status_values(self, revert_homedir): + """Test that overall_status has valid values.""" + c = CVDUpdate() + + with mock.patch.object(c, '_query_dns_txt_entry', return_value=False): + status = c.db_status() + + assert status['summary']['overall_status'] in ['healthy', 'warning', 'critical'] + + def test_dns_failure_warning(self, revert_homedir): + """Test that DNS failure adds a warning.""" + c = CVDUpdate() + + with mock.patch.object(c, '_query_dns_txt_entry', return_value=False): + status = c.db_status() + + dns_warnings = [w for w in status['warnings'] if 'DNS' in w] + assert len(dns_warnings) > 0 + + +class TestDbStatusDatabaseFields: + """Tests for individual database fields in db_status.""" + + def test_database_has_required_fields(self, revert_homedir): + """Test that each database entry has required fields.""" + c = CVDUpdate() + + with mock.patch.object(c, '_query_dns_txt_entry', return_value=False): + status = c.db_status() + + required_fields = [ + 'name', + 'local_version', + 'remote_version', + 'is_current', + 'version_status', + 'is_missing', + 'last_modified', + 'age_hours', + 'age_seconds', + 'age_status', + 'on_cooldown', + 'cooldown_until', + 'cdiff_count', + 'file_size_bytes', + ] + + for db in status['databases']: + for field in required_fields: + assert field in db, f"Missing field: {field}" + + def test_cooldown_detection(self, revert_homedir): + """Test that cooldown is correctly detected.""" + c = CVDUpdate() + + # Set a database on cooldown + c.state['dbs']['main.cvd']['retry after'] = time.time() + 3600 + + with mock.patch.object(c, '_query_dns_txt_entry', return_value=False): + status = c.db_status() + + main_db = next(db for db in status['databases'] if db['name'] == 'main.cvd') + assert main_db['on_cooldown'] is True + assert main_db['cooldown_until'] is not None + + # Check warning was added + cooldown_warnings = [w for w in status['warnings'] if 'cooldown' in w] + assert len(cooldown_warnings) > 0 + + +class TestDbStatusVersionState: + """Tests that version state distinguishes unknown, current, and outdated.""" + + def _single_main_db(self, c, local_version, last_modified): + c.db_dir.mkdir(parents=True, exist_ok=True) + (c.db_dir / 'main.cvd').write_bytes(b'x') + c.state['dbs'] = { + 'main.cvd': { + 'url': 'https://database.clamav.net/main.cvd', + 'retry after': 0, + 'last modified': last_modified, + 'last checked': 0, + 'DNS field': 1, + 'local version': local_version, + 'CDIFFs': [], + } + } + + def test_unknown_when_dns_unavailable(self, revert_homedir): + """A present database with no remote version is unknown, not outdated.""" + c = CVDUpdate() + self._single_main_db(c, local_version=62, last_modified=time.time()) + + with mock.patch.object(c, '_query_dns_txt_entry', return_value=False): + status = c.db_status() + + main_db = next(db for db in status['databases'] if db['name'] == 'main.cvd') + assert main_db['version_status'] == 'unknown' + assert main_db['is_current'] is False + assert status['summary']['unknown_count'] == 1 + assert status['summary']['outdated_count'] == 0 + + def test_outdated_when_local_behind_remote(self, revert_homedir): + """A database behind the advertised version is outdated and warns.""" + c = CVDUpdate() + self._single_main_db(c, local_version=60, last_modified=time.time()) + + def fake_dns(): + c.dns_version_tokens = ['0', '62'] + return True + + with mock.patch.object(c, '_query_dns_txt_entry', side_effect=fake_dns): + status = c.db_status() + + main_db = next(db for db in status['databases'] if db['name'] == 'main.cvd') + assert main_db['version_status'] == 'outdated' + assert status['summary']['outdated_count'] == 1 + assert status['summary']['overall_status'] == 'warning' + assert any('behind' in w for w in status['warnings']) + + def test_current_with_old_file_is_healthy(self, revert_homedir): + """A current database is healthy even when its file is old.""" + c = CVDUpdate() + self._single_main_db(c, local_version=62, last_modified=time.time() - (200 * 3600)) + + def fake_dns(): + c.dns_version_tokens = ['0', '62'] + return True + + with mock.patch.object(c, '_query_dns_txt_entry', side_effect=fake_dns): + status = c.db_status() + + main_db = next(db for db in status['databases'] if db['name'] == 'main.cvd') + assert main_db['version_status'] == 'current' + assert status['summary']['stale_count'] == 0 + assert status['summary']['overall_status'] == 'healthy' + + def test_fresh_non_cvd_database_is_healthy(self, revert_homedir): + """A freshly downloaded non-CVD database has no version, so it is + reported as unknown but must not flag the mirror as unhealthy.""" + c = CVDUpdate() + c.db_dir.mkdir(parents=True, exist_ok=True) + (c.db_dir / 'custom.ndb').write_bytes(b'x') + c.state['dbs'] = { + 'custom.ndb': { + 'url': 'https://example.com/custom.ndb', + 'retry after': 0, + 'last modified': time.time(), + 'last checked': 0, + 'DNS field': 0, + 'local version': 0, + 'CDIFFs': [], + } + } + + def fake_dns(): + c.dns_version_tokens = ['0', '62'] + return True + + with mock.patch.object(c, '_query_dns_txt_entry', side_effect=fake_dns): + status = c.db_status() + + custom = next(db for db in status['databases'] if db['name'] == 'custom.ndb') + assert custom['version_status'] == 'unknown' + assert status['summary']['overall_status'] == 'healthy' +