support helper command list filtering

[jmartin-tech]

Summary

When working with a *_spec or configuration file it would be helpful to be able to extract the list of plugins that will be activated without having to run the full test lifecycle.

Basic example

Probes via CLI

% garak --list_probes -p dan
garak LLM vulnerability scanner v0.13.1.pre1 ( https://github.com/NVIDIA/garak ) at 2025-09-10T12:37:31.715465
probes: dan.Ablation_Dan_11_0
probes: dan.AutoDANCached

% garak --list_probes -p dan,dan.AntiDAN
garak LLM vulnerability scanner v0.13.1.pre1 ( https://github.com/NVIDIA/garak ) at 2025-09-10T12:37:31.715465
probes: dan.Ablation_Dan_11_0
probes: dan.AntiDAN
probes: dan.AutoDANCached

% garak --list_detectors -d misleading
garak LLM vulnerability scanner v0.13.1.pre1 ( https://github.com/NVIDIA/garak ) at 2025-09-10T12:37:31.715465
detectors: misleading.MustRefuteClaimModel

Detectors via CLI

% garak --list_detectors -d misleading.MustContradictNLI
garak LLM vulnerability scanner v0.13.1.pre1 ( https://github.com/NVIDIA/garak ) at 2025-09-10T12:37:31.715465
detectors: misleading.MustContradictNLI

Error edge case example

% garak --list_detectors -d misleading.Invalid
garak LLM vulnerability scanner v0.13.1.pre1 ( https://github.com/NVIDIA/garak ) at 2025-09-10T12:37:31.715465
No detectors match the provided `detector_spec`: misleading.Invalid.

Probes via configuration file

config.yaml

plugins:
  probe_spec: dan,danAntiDan

garak --list_probes --config config.yaml

Motivation

This would allow users a quicker light weight method to understand impact of *_spec values and could in the future would enable exploration of more complex *_spec syntax for selection based on things other than module &module.classname entries.

[JosephDavisC]

Hi there! I’d like to take this issue as my first contribution to garak. I think my plan is to update the --list_probes and --list_detectors commands so they can filter results based on --spec or a --config file, and include error handling for invalid cases. I’ll also add some tests and update the CLI help text accordingly. Please let me know if you have any specific guidelines or preferences before I get started. Thanks!

[leondz]

see also filtering in #955

[JosephDavisC]

I checked out the code and #955 . My CLI filtering is focused on the user-facing listing of probes and detectors, so it doesn’t conflict with the trait scan infrastructure. Everything looks good from my end, but happy to tweak if needed!