Suggestion: Declare determinism and portability as architectural constraints

[mattss-madetech]

As part of investigating options for #39 I ran a grill-me session with Codex on the risks of losing portability and determinism whilst making performance improvements. My concern was that changes might either introduce additional complexity (such as parallel file processing)and/ or rely on specific features of shell tools which differ between platforms.

I got Codex to spit out a potential "contract" to be added to CLAUDE.md/AGENTS.md to guide future agentic work along these lines:

Determinism and portability contract

• Portability and deterministic output are architectural constraints. Performance improvements are secondary to those constraints.
• Detection trust takes precedence over throughput. The same repository should produce the same findings and the same output ordering across supported environments.
• Optimisations must preserve a single default scan path. Do not add alternate "faster" or experimental modes that use different detection semantics.
• Prefer sequential, behaviour-preserving optimisations first. Reduce redundant file reads and unnecessary process spawning before considering more invasive changes.
• Treat Bash parallelism as high risk. Do not introduce parallel scanning in Bash unless the architecture changes first and deterministic equivalence is proven.
• Treat the portability surface as Bash plus a documented safe subset of external tool behaviour. Avoid relying on non-portable or implementation-specific behaviour in find, grep, awk, sed, and sort, especially in hot paths.
• Bash is a soft implementation constraint, not a hard one. Future changes may move scanner-core behaviour into another runtime if that reduces cross-environment verification risk without requiring users to preinstall an additional runtime.

I then asked it to evaluate the existing codebase based on the above contract:

Findings:

• Medium: Core file ordering is not fully deterministic. src/file_list.sh:29 uses sort -f under the ambient
  locale and without a secondary tie-breaker. That means scan order, progress output, and final hit ordering
  can vary across locales and for names that compare equal when case-folded, which is directly against “same
  findings and same output ordering”. The existing test only checks a simple happy path and would not catch
  locale or case-tie drift (tests/file_list.bats:29).
• Medium: Git commit hit ordering is filesystem-dependent. src/git_commit_check.sh:24 consumes find output
  directly, with no stabilising sort before writing hits. On multi-repo scans, the order of [MALICIOUS GIT
  COMMITS] can vary by platform/filesystem traversal order. The tests verify presence and count, but not
  deterministic ordering (tests/git_commit_check.bats:65).
• Medium: The package-scan hot path relies on implementation-specific grep -A behaviour that the contract
  explicitly says to avoid. Examples are src/package_check.sh:65, src/package_check.sh:89, src/
  package_check.sh:212, and src/package_check.sh:256. BSD and GNU grep both usually support -A, but it is
  still outside the “safe subset” the repo claims to target, so this is a contract miss even if it works on
  your machine.
• Low: The contract is only weakly enforced by tests. The suite is broad and all-green, but it does not pin
  cross-environment invariants like locale-independent sort order, stable multi-repo ordering, or package
  parsing without GNU/BSD-specific options. See tests/file_list.bats:29, tests/git_commit_check.bats:65, and
  tests/version.bats:7.

Assessment

Architecturally, the repo is fairly close to its contract: the default scan path is sequential, there’s no
Bash parallelism, and the code is clearly trying to keep output ordered. I’d call it a moderate fit overall,
not a strong one.

The main gap is that “deterministic and portable” is more an intention than a proven property right now. The
default scan path still depends on ambient locale and some implementation-specific tool behaviour, so I would
not treat the current contract as fully satisfied across supported environments.

@willgibson-madetech Would appreciate any comments on this.

[willgibson-madetech]

I think it would be good to add the "Determinism and portability contract", then pick off those findings one at a time.

It's also provoked me to think about running the tests on multiple operating systems in the GitHub Actions to better validate portability.