Skip to content

Latest commit

 

History

History
320 lines (259 loc) · 16.8 KB

File metadata and controls

320 lines (259 loc) · 16.8 KB

Run tests and verification

Documentation / Development

Audience: maintainers verifying approved code or documentation changes. Status: executable local workflow, not real-service support evidence.

Run commands from the repository root. Start with the affected contract's smallest meaningful check, then broaden. Do not weaken assertions to make a failure disappear.

Prerequisites

  • Use the Go version required by go.mod, currently Go 1.27.0. Historical Go 1.26 verification does not qualify the current consuming graph; the minimum is not a claim about the latest available patch.
  • Put the selected toolchain's bin first in PATH and use GOTOOLCHAIN=local so child Go commands use it too. Race runs need a suitable CGO/race environment; ordinary CGO-disabled checks are separate below.
  • Make dependencies available. go mod download may need network on a new checkout. Build fixtures seed an isolated file proxy using the cached YAML v3.0.5 archive, then disable registry/sumdb access. They do not use a real service.
  • Service tests require separate authorization, isolated resources and cleanup; no production credentials are provided to ordinary tests.

Focus the check

This workspace-selection subtest initializes its own SDK fixture and runs alone:

go test -race -count=1 -timeout=3m ./internal/compatibility -run '^TestIndependentConsumerBuildSelectionAndBehavior$/^workspace-selection$'

Also run the complete group after changes to build/metadata fixtures:

go test -race -count=1 -timeout=3m ./internal/compatibility -run '^TestIndependentConsumerBuildSelectionAndBehavior$'

For package contracts and executable documentation:

go test -race -count=1 ./internal/resource ./internal/invocation ./internal/fault
go test -count=1 -run '^Example' ./...
go doc -all ./internal/resource

These are local fixtures, not a public framework tutorial. The conformance reference separates mechanism, native parser, synthetic SDK/module and missing service evidence.

Configuration/type and conformance acceptance regressions have explicit controls:

go test -count=1 ./internal/resource -run 'Test(Generic|PreparedAnonymous|YAMLExplicit|YAMLTag|CleanupHistory)'
go test -count=1 ./internal/conformance -run 'Test(Runtime|Facade)'

TestGenericConversionsCompileBoundary first compiles valid aliases/settings, then requires the compiler's intended cross-type conversion diagnostic in isolated fixtures. Keep illegal casts out of ordinary compiled test files. Runtime/Facade negative controls execute in deadline-bounded child processes and require the matching conformance failure, successful return from the helper and no canary disclosure. A PASS from a historical false-certification witness is not acceptance.

Standard repository checks

For the internal Viper v1 integration, exercise the real consumer, raw preparation handoff and rejecting controls before broadening to full checks:

go test -race -count=1 -timeout=3m ./internal/configsource/viper/v1
go test ./internal/configsource/viper/v1 -run '^$' -fuzz '^FuzzLoad$' -fuzztime=10s -parallel=2
go test ./internal/configsource/viper/v1 -run '^$' -fuzz '^FuzzQuery$' -fuzztime=10s -parallel=2
go test ./internal/configsource/viper/v1 -run '^$' -fuzz '^FuzzFormats$' -fuzztime=10s -parallel=2
GOMAXPROCS=4 go test ./internal/configsource/viper/v1 -run '^$' -bench '^BenchmarkLoading$' -benchmem -benchtime=10x -count=5
go run ./internal/configsource/viper/v1/testdata/consumer

The benchmark compares equal useful results and input/I/O conditions, not an unbounded native reader with a bounded integration. Report the additional SDK/preflight parsing and raw-copy costs for preparation separately from native query costs. Percentiles are bounded in-process samples (up to 128 per run), not production tail-latency guarantees. See the native profile and bounds.

Load/query tests own the corresponding core behavior. Integration tests exercise real files, strict preparation and the consuming executable; platform-specific and benchmark files remain separate where their execution conditions differ. Coverage reports help locate missing paths, not certify all inputs. Do not replace the real SDK merely to force an unreachable defensive branch to reach 100 percent.

The authoritative CI commands are in checks.yml.

For the Nacos v2 integration, use the selected toolchain's bin first in PATH:

go test -race -count=3 -timeout=3m ./internal/configsource/nacos/v2
go run ./internal/configsource/nacos/v2/testdata/consumer
go test ./internal/configsource/nacos/v2 -run '^$' -fuzz '^FuzzOptions$' -fuzztime=10s -parallel=2
go test ./internal/configsource/nacos/v2 -run '^$' -fuzz '^FuzzProtocol$' -fuzztime=10s -parallel=2
go test ./internal/configsource/nacos/v2 -run '^$' -fuzz '^FuzzSearchPage$' -fuzztime=10s -parallel=2
GOMAXPROCS=4 go test ./internal/configsource/nacos/v2 -run '^$' -bench '^BenchmarkRead$' -benchmem -benchtime=10x -count=3

The benchmark compares equal owned native-session lifetimes and raw results, not a warm SDK cache against a cold connection. The consumer reports the actual SDK in its own binary; its loopback service is not deployment acceptance.

The opt-in service gate writes and deletes generated test keys. Run it only with explicitly authorized isolated resources and credentials:

FATHOMRY_NACOS_TEST_CONFIG=/path/to/private-nacos-fixture.json \
  go test -tags=nacos_service -race -count=1 -timeout=4m \
  ./internal/configsource/nacos/v2 -run '^TestNacosServiceReadWriteWatchAndCleanup$'

FATHOMRY_NACOS_TEST_CONFIG=/path/to/private-nacos-fixture.json \
  go test -tags=nacos_service -race -count=1 -timeout=3m \
  ./internal/configsource/nacos/v2 -run '^TestNacosManagementService$'

The bounded mode-0600 JSON fixture contains http_url, grpc_address, namespace, username, password, admin_username, admin_password, optional root_ca_pem, allow_insecure and explicit allow_writes. Use a read-only reader and a separately authorized fixture publisher. The gate verifies generated-key absence before writes, deletes those keys and checks absence during cleanup. It does not alter roles, deploy services, raise platform limits or restart remote nodes. Its connection-interruption test drops only this client's connection. Periodic reconciliation is set beyond the bounded push wait to avoid false push acceptance. The management gate uses the explicitly authorized administrator for a unique fixture, exercises native Publish/CAS/Delete and metadata/tag search, and confirms cleanup. A generic stale-CAS server error must remain an unknown mutation outcome; the gate separately reads back the content rather than parsing error text. Compiling with -run '^$' runs no real-service test. See the Nacos profile for current version, native limitations, upstream-upgrade TODO and unexecuted service modes.

For PostgreSQL #23/#28, first run the native protocol and ownership suite:

go test -race -count=3 -timeout=3m ./internal/database/pgx/v5
go test ./internal/database/pgx/v5 -run '^$' -fuzz '^FuzzOptionsV1$' -fuzztime=10s -parallel=2
go test ./internal/database/pgx/v5 -run '^$' -fuzz '^FuzzStatement$' -fuzztime=10s -parallel=2
go test ./internal/database/pgx/v5 -run '^$' -fuzz '^FuzzProtocolFence$' -fuzztime=10s -parallel=2
GOMAXPROCS=4 go test ./internal/database/pgx/v5 -run '^$' -bench '^BenchmarkBoundedQuery$' -benchmem -benchtime=20x -count=3
GOMAXPROCS=4 go test ./internal/database/pgx/v5 -run '^$' -bench '^BenchmarkTransactionalPreparation$' -benchmem -benchtime=100ms -count=3
go run ./internal/database/pgx/v5/testdata/consumer

The consumer executes configuration and local ownership; it explicitly reports that no query was run. Protocol peers do not establish PostgreSQL transaction atomicity. The package contract describes the shared-core/facade benchmark and remaining physical-resource limits.

The following service gate creates and drops a random dedicated test database. It needs explicit authorization, a non-superuser role with the required isolated database privileges, and independently trusted TLS roots/server identity:

FATHOMRY_POSTGRES_TEST_CONFIG=/path/to/private-postgres-fixture.json \
  go test -tags=postgres_service -race -count=1 -timeout=3m \
  ./internal/database/pgx/v5 -run '^TestPostgreSQLService$'

The mode-0600, at-most-128-KiB JSON fixture contains address, port, user, password, root_ca_pem, server_name, expected_version_number, and explicit allow_create_test_database. It contacts the maintenance database only for metadata and its owned database's lifecycle. Business databases/tables and server configuration must not be modified. The loopback fault proxy verifies TLS to the real server and drops real COMMIT/CREATE responses; independent read-back supplies the effect oracle, and fresh maintenance connections reconcile fixture cleanup. Only the proxy's loopback client leg is plaintext. Never put a private fixture, packet capture or connection diagnostic in the repository. Compiling the tagged test with -run '^$' is not real-service acceptance.

The extended #28 gate also exercises ordinary SQL/DDL/MERGE, repeated native preparation, ordinary/prepared text equivalence, savepoint recovery and actual server-scope removal, same-session default restoration, chained transaction termination, explicit Ping, independently observed pool overlap and unattended expiry. Reserved fixtures use the gh28_ prefix. Both ordinary and lost-CREATE paths must retain reconciliation for an uncertain creation without adopting a preexisting resource. Savepoint failure cleanup can finalize its owning parent; test-owned asynchronous queries must cancel/join before database teardown.

For MySQL #25, run the native component/ownership and actual consumer checks:

go test -race -count=10 -timeout=5m ./internal/database/mysql/v1
go run ./internal/database/mysql/v1/testdata/consumer
go test ./internal/database/mysql/v1 -run '^$' -fuzz '^FuzzOptionsV1$' -fuzztime=10s -parallel=2
go test ./internal/database/mysql/v1 -run '^$' -fuzz '^FuzzStatement$' -fuzztime=10s -parallel=2
go test ./internal/database/mysql/v1 -run '^$' -fuzz '^FuzzWirePacket$' -fuzztime=10s -parallel=2
GOMAXPROCS=4 go test ./internal/database/mysql/v1 -run '^$' -bench '^BenchmarkPreparationReuse$' -benchmem -benchtime=100ms -count=3

The benchmark changes only native preparation reuse: the transaction, workload, incoming/result bounds, copied values and independent evidence remain identical. Its local protocol peer and bounded latency samples do not establish real-server throughput. The MySQL contract specifies the source-pinned transport and native cleanup/transaction obligations.

The explicit service gates fail if their private fixtures are missing; they do not silently skip. The write gate creates and drops an exclusive random database and needs authorization for that lifecycle and filtered metadata inspection:

FATHOMRY_MYSQL_TEST_CONFIG=/path/to/private-mysql-fixture.json \
  go test -tags=mysql_service -race -count=1 -timeout=3m \
  ./internal/database/mysql/v1 -run '^TestMySQLService$'
FATHOMRY_MYSQL_TLS_TEST_CONFIG=/path/to/private-mysql-tls-fixture.json \
  go test -tags=mysql_service -race -count=1 -timeout=3m \
  ./internal/database/mysql/v1 -run '^TestMySQLTLSService$'

Each mode-0600, at-most-128-KiB JSON fixture contains network, address, port, database, user, password, plaintext, root_ca_pem, server_name, alternative server_certificate_sha256, expected_version and allow_create_test_database. The write gate requires that last flag explicitly true. It can use verified TLS over an authorized Unix socket without changing accounts or grants; TCP TLS writes require their own authorized fixture profile. It protects preexisting resources, independently reads committed/rolled-back effects, observes server entry/eventual exit for cancellation, and verifies cleanup with fresh connections. Acknowledged ownership is registered before evidence assertions, independently of later cleanup errors. Failure-path tests close retained statements/transactions; concurrency checks observe actual overlapping server work on two owned connections. Only an acknowledged newly created fixture is eligible for deletion; an unknown CREATE result must be reconciled by its owner, not adopted or dropped automatically. It does not modify accounts/grants or server configuration. The TLS gate is read-only and requires verified TLS, not plaintext. Record the two actual transport and account profiles separately; neither compiling tagged tests nor an unexecuted service gate is MySQL acceptance. Never commit fixture credentials or full DSNs.

For a normal implementation change, run:

go mod tidy -diff
go mod verify
test -z "$(gofmt -l .)"
go vet ./...
go test -race -count=1 -timeout=10m ./...
go build ./...
git diff --check
bash .agents/skills/fathomry-development/scripts/new-issue_test.sh

The scaffold check is offline and does not publish issues. Signing, PR checks and merge remain separate gates in contribution policy.

Repeated concurrency and compatibility checks

Choose repeats and bounds proportionate to the changed behavior:

go test -race -count=20 -timeout=5m ./internal/resource ./internal/invocation ./internal/fault ./internal/conformance
go test -race -count=3 -timeout=3m ./internal/compatibility
go test -race -shuffle=20260910 -cpu=1,4 -count=3 -timeout=5m ./...
CGO_ENABLED=0 go test -count=1 -timeout=10m ./...

Whole-test shuffling does not establish nested subtest independence; select important subtests explicitly. Independent-module checks do not force child -race. The CGO-disabled suite must execute its import/build checks, not silently skip them.

Bounded fuzz checks

Use the existing targets, not a new test framework:

go test ./internal/resource -run '^$' -fuzz '^FuzzPrepare$' -fuzztime=10s -parallel=2
go test ./internal/compatibility -run '^$' -fuzz '^FuzzBuildMetadataPrivacy$' -fuzztime=10s -parallel=2
go test ./internal/conformance -run '^$' -fuzz '^FuzzPrivateLiteralDiagnostics$' -fuzztime=10s -parallel=2
go test ./internal/fault -run '^$' -fuzz '^FuzzTechnicalFaultContext$' -fuzztime=10s -parallel=2

Record the corpus, executions, failures and limits when relevant. Bounded success is neither exhaustive proof nor a throughput comparison. For ownership/result changes, use independent native/workload oracles and deliberately broken controls. Compilation failure or timeout is not the intended contract rejection unless the test explicitly owns that behavior.

Record what was proved

Record exact commit/worktree, toolchain/platform, commands and observed outcomes, including failed-before/fixed-after evidence. Payloads/errors/native allocations retain their stated ownership and bounds. The finite microbenchmark measures mechanism overhead, not SDK throughput or a native-memory cap. Sustained fixtures check declared high-water limits, not every physical resource.

The build test executes SDK behavior and inspects that same binary. Fathomry intentionally contributes no package to that particular probe; a separate in-module probe covers framework-as-main. The rebuilt failure and settings contracts have independently compiled consumers. Settings default tests run in fresh processes, without a production reset hook; parent-process coverage alone does not include those child counters. Do not substitute inspector metadata or manufacture passing support records from startup facts.

Actual SDK termination, session/account isolation, service effects, native buffers and workload/SLO limits still need concrete integration evidence. No Temporal command, converter or history/payload format is introduced by these foundations; future changes to those boundaries need appropriate replay evidence. Keep raw logs in local issue literature, not the product manual.