Repository navigation
Conversation
`pr.preview-deploy.yml` starts the infrastructure from `docker-compose.yml` and
then starts the backend against a hardcoded connection string:
-e DATABASE_URL=postgresql+asyncpg://autoaudit:autoaudit_dev_password@db:5432/autoaudit
`f7981302` externalised that password, so the database no longer has it and the
backend cannot connect. The last run that was not skipped, on PR Hardhat-Enterprises#306 on
2026-08-31, failed at "Wait for backend health (includes Alembic migrations)"
after the two-minute timeout; every run since has been skipped because nobody
applies the `deploy-preview` label any more.
Beyond not working, the stack holds a hosted runner for up to six hours per
preview, publishes mutable `pr-<number>` image tags to GHCR -- the only registry
push in the repository -- and exposes the runner through a Cloudflare quick
tunnel. `pr.preview-instructions.yml` and `pr.preview-teardown.yml` exist only to
advertise and stop it.
`git log -- .github/workflows/pr.preview-deploy.yml` has the original if previews
are ever revived.
Two pages described the repository as it was, not as it is, and both errors
send a contributor the wrong way.
**`docs/DevSecOps/workflow-documentation.md` named ten of fourteen workflows**
and generalised over them as if the list were complete. `ci.compliance.yml`,
`ci.gitleaks.yml`, `ops.branch-cleanup.yml` and `pr.size-warning.yml` appeared
nowhere. The page now names all fourteen, with one table of triggers and path
filters, so a missing workflow shows up as a gap rather than as silence.
Five of its statements were wrong:
- **`ci.grype.yml`: "fail-build is false so findings are surfaced for review
rather than blocking merges."** It is `fail-build: true` with
`severity-cutoff: critical`. A critical finding fails the check.
- **`ops.collector.yml` "runs the audit engine collector on the
engine-development branch".** Its push trigger is commented out and the job
carries `if: false`; it runs nothing, deliberately, and its header records the
two conditions for re-enabling it.
- **`ops.workflow-cleanup.yml` "runs on a schedule to delete old workflow run
history".** The scheduled step is `--dryRun=true`; deletion needs a manual
dispatch with `dry_run=false` and `confirm=DELETE`. Its schedule is Sundays
00:00 UTC, not the Saturdays 23:32 the table claimed. And it is not a
retention policy at all: the workflow passes `RETENTION_DAYS: 30` and its
input is described as "Delete runs older than this many days", but
`tools/workflow-cleanup/cleanup-workflows.js` never reads that variable. It
selects by how long a run took -- anything under two minutes is deleted, two
minutes or more is kept -- over a single unpaginated page of 20 runs. A real
deletion run would remove recent short runs and keep old long ones.
- **`ci.validate-alerts.yml` "does not do anything meaningful in its current
state".** It runs `promtool check rules` and fails the build on a syntax
error. What it does not do is now stated instead: it globs five filename
patterns non-recursively, so a new rule file under another name escapes it;
promtool is installed unpinned with `apt-get`; and a syntax check does not
test that a threshold fires or that an alert reads a metric the application
emits.
- **`pr.size-warning.yml`,** newly documented, does not comment on a pull
request as the obvious reading suggests. It emits a `core.warning` annotation
and a job summary.
Two things worth knowing are now written down: `ci.opa-eval.yml` evaluates
`engine/legacy/` and `aggregator.py` records a non-zero `opa` exit into the
report JSON rather than raising, so no policy verdict from it can fail a build,
and nothing in `.github/workflows/` runs `opa check`, `opa test` or `opa eval`
against the CIS or Essential Eight policies; and `ops.short-test.yml` filters on
`.github/workflows/short-test.yml` while the file is `ops.short-test.yml`,
because `155f82aa` applied the `ops.` prefix and left the filter behind.
**`engine/collectors/README.md` said `GraphClient` was "shared across all
Microsoft collectors (Entra, M365 services)".** It is not, and the gap is nearly
half the registry: resolving each `DATA_COLLECTORS` entry to the client its
`collect` method declares gives 26 `GraphClient` and 22 `PowerShellClient` out of
48. Every Exchange Online and SharePoint Online setting AutoAudit reads comes
through a cmdlet, because Graph does not expose it.
The page now leads with picking a client, shows a PowerShell worked example
beside the Graph one, and states the mechanism that actually decides:
`engine/worker/tasks.py` selects the client from the collector's registered ID
prefix, so a PowerShell collector registered under an `entra.` ID is handed a
`GraphClient` and fails. `exchange.dns.dns_security_records` is the carve-out
and is explained.
Four smaller corrections on the same page: the imports in the examples are
rooted at `collectors.`, not `engine.collectors.`; the folder tree showed an
`m365/exchange|sharepoint|teams` layout that does not exist; `_pending/`, which
holds the Teams and Purview collectors, was not mentioned; and two client
guarantees were overstated -- `GraphClient` returns its cached token without an
expiry check, and `get_all_pages` stops at `max_pages` (default 100) and returns
what it has rather than raising.
Verified:
$ ls .github/workflows/ # 14 files, all named by the page
$ python3 - (resolve DATA_COLLECTORS to each collect() client)
DATA_COLLECTORS entries: 48
GraphClient: 26
PowerShellClient: 22
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is wrong
Two pages describe the repository as it was, not as it is, and both errors send a contributor the wrong way.
docs/DevSecOps/workflow-documentation.mdnamed ten of fourteen workflowsci.compliance.yml,ci.gitleaks.yml,ops.branch-cleanup.ymlandpr.size-warning.ymlappeared nowhere on the page, which nevertheless generalised over the workflows as though the list were complete.Four of its statements were wrong:
ci.grype.yml— "fail-buildis false so findings are surfaced for review rather than blocking merges"fail-build: true,severity-cutoff: critical. A critical finding fails the check.ops.collector.yml"runs the audit engine collector on theengine-developmentbranch"if: false. It runs nothing, deliberately.ops.workflow-cleanup.yml"runs on a schedule to delete old workflow run history"--dryRun=true. Deletion needs a manual dispatch withdry_run=falseandconfirm=DELETE. Schedule is Sundays 00:00 UTC, not Saturdays 23:32.ci.validate-alerts.yml"does not do anything meaningful in its current state"promtool check rulesand fails the build on a syntax error.pr.size-warning.yml— undocumented, and the obvious reading is wrongcore.warningannotation and a job summary.The
fail-buildone is the costly error: a contributor who believes the page is surprised by a red check they were told could not block them.ops.workflow-cleanup.ymlturned out to be worse than out of date. It is not a retention policy at all: the workflow passesRETENTION_DAYS: 30and its dispatch input is described as "Delete runs older than this many days", buttools/workflow-cleanup/cleanup-workflows.jsnever reads that variable. It selects by how long a run took — under two minutes is deleted, two minutes or more is kept — across a single unpaginated page of 20 runs. A real deletion run would remove recent short runs and keep old long ones. Nothing here prunes by age, so this workflow should not be cited as retention evidence.Two facts are now written down that were not:
ci.opa-eval.ymlevaluatesengine/legacy/and uploads a PDF and a JSON artifact. No policy verdict from it can fail a build —aggregator.pyrecords a non-zeroopaexit into the report JSON rather than raising — though a crash in either script still fails the job. Nothing in.github/workflows/runsopa check,opa testoropa evalagainst the CIS or Essential Eight policies.ops.short-test.ymlfilters on.github/workflows/short-test.ymlwhile the file is.github/workflows/ops.short-test.yml. Commit155f82aaapplied theops.prefix and left the filter behind, so editing the canary no longer runs it.engine/collectors/README.mdsaidGraphClientwas "shared across all Microsoft collectors"It is not, and the gap is nearly half the registry. Resolving each
DATA_COLLECTORSentry to the client itscollectmethod declares gives 26GraphClientand 22PowerShellClientout of 48. Every Exchange Online and SharePoint Online setting AutoAudit reads comes through a cmdlet, because Graph does not expose it — so the page sent every new Exchange, Teams or SharePoint collector to the wrong client.The page now leads with picking a client, shows a PowerShell worked example beside the Graph one, and states the mechanism that actually decides:
engine/worker/tasks.pyselects the client from the collector's registered ID prefix, before the class is instantiated, so a PowerShell collector registered under anentra.ID is handed aGraphClientand fails. That rule and thecollectannotations agree for all 48 today.exchange.dns.dns_security_recordsis the carve-out and is explained.Four smaller corrections on the same page: the imports in the examples are rooted at
collectors., notengine.collectors.; the folder tree showed anm365/exchange|sharepoint|teamslayout that does not exist;_pending/— which holds the five Teams and two Purview collectors — was not mentioned; and twoGraphClientguarantees were overstated. The client returns its cached token with no expiry check, so it is caching and not refresh, andget_all_pagesstops atmax_pages(default 100) and returns what it has rather than raising.Verification
The collector split is resolved by importing
collectors.registryand reading each class'scollectsignature, not by countinggit grephits.Merge order
The inventory is stated against this branch's base, which is the preview-deploy removal PR in this series: fourteen workflow files, after the three
pr.preview-*files are gone. Merge that one first.Caveat (GRC-D03)
Demonstrates policy-decision correctness against fixtures. No live tenant has been
collected, so this is not evidence of any organisation's control posture.