Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 23 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ name: ci
# on Enterprise (500 concurrent jobs), so the split no longer works around a
# cap; it keeps per-push runner time and queue time down. So:
#
# - Tier A runs on every pull_request push: fmt, clippy, typos,
# - Tier A runs on every pull_request push: fmt, clippy, typos, shellcheck,
# third-party-license, the single non-experimental leg of each compat
# suite, `test` on postgres:17 only (compose.yaml's default), and ci-gate.
# - Tier B runs once per merge-queue entry (`merge_group`) and on push:
Expand Down Expand Up @@ -96,8 +96,8 @@ env:
jobs:
# Whether a pull_request run needs the Rust jobs at all. A PR that only
# touches CI config this workflow doesn't read, or prose, skips every job
# below except typos and ci-gate (see .github/scripts/ci-changes.sh for the
# list). merge_group and push always get rust=true. The classifier is
# below except typos, shellcheck and ci-gate (see
# .github/scripts/ci-changes.sh for the list). merge_group and push always get rust=true. The classifier is
# checked out from the PR's base, so a PR can't widen its own skip list.
# The same job is in dependencies.yml and codeql.yml.
changes:
Expand Down Expand Up @@ -509,6 +509,24 @@ jobs:
- uses: crate-ci/typos@00f422f3b19c57bc6338715ebfe3316d38768461 # v1.50.3
with:
config: typos.toml
# actionlint already runs shellcheck on each workflow `run:` block. This
# job checks the shell scripts in the repository.
shellcheck:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4.0.0
with:
just-version: 1.58.0
# We install a pinned version instead of using the runner image's copy,
# so that a runner image update cannot fail this check on unchanged code.
- uses: taiki-e/install-action@83ac0ad63c0167e6f06796fab0fce28db1bf3db0 # v2.87.22
with:
tool: shellcheck@0.11.0
- run: just shellcheck
third-party-license:
if: needs.changes.outputs.rust == 'true'
needs: [changes]
Expand Down Expand Up @@ -947,6 +965,7 @@ jobs:
- compat-franz-go
- cargo-publish-dry-run
- typos
- shellcheck
- third-party-license
- src
- package
Expand Down Expand Up @@ -985,7 +1004,7 @@ jobs:
# PR gate would stay green with no Tier A signal at all. The list names
# the Tier B exemptions, so a newly added job defaults to must-not-skip.
# Skipped when `changes` found no Rust-relevant path, since then
# every Tier A job but typos skips by design.
# every Tier A job but typos and shellcheck skips by design.
- name: Tier A ran on pull_request
if: github.event_name == 'pull_request' && needs.changes.outputs.rust == 'true'
env:
Expand Down
3 changes: 2 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ just test-doc # cargo test --workspace --doc --all-features
just doc # rustdoc, warnings denied, private items too; pass --open to browse
just clippy # cargo clippy --workspace --all-features --all-targets -- -D warnings
just fmt # cargo fmt --all --check
just shellcheck # shellcheck every tracked *.sh / *.bash script
just check # cargo check --workspace --all-features --all-targets
just smoke <engine> # Kafka CLI smoke suite (nisshi-smoke-test) against postgres, sqlite, memory or s3; starts and removes its own broker and services
just ci # (re)starts the docker compose services (postgres, minio, lakehouse) that integration tests depend on - safe to rerun if services are in a bad state
Expand Down Expand Up @@ -146,7 +147,7 @@ Lake features: `parquet`, `iceberg`, `delta` - enable writing schema-backed topi

GitHub Actions (`.github/workflows/ci.yml`) runs in two tiers, gated by `ci-gate`, the single required check that fans in every other job:

- **Tier A, every pull_request push:** `fmt`, `clippy` (which also runs `just doc`), `typos`, `third-party-license`, `test` (postgres:17 only), one non-experimental leg each of `compat-librdkafka` / `compat-franz-go`.
- **Tier A, every pull_request push:** `fmt`, `clippy` (which also runs `just doc`), `typos`, `shellcheck`, `third-party-license`, `test` (postgres:17 only), one non-experimental leg each of `compat-librdkafka` / `compat-franz-go`.
- **Tier B, once per merge-queue entry (`merge_group`) and on push to `main`:** the `build-storage` matrix (one build per storage engine, `turso` included) and `build-storage-lake` (one build per lake format on `dynostore`), `test` on postgres:16/17/18, the experimental compat legs, `cargo-publish-dry-run`, `src`, `release`, `package`, `smoke` (the `nisshi-smoke-test` suite: Kafka CLI tools from Kafka 3.9 and 4.3 against the packaged image on postgres, memory and s3, and against a source build on sqlite until #796 is fixed, on x86 and arm), `smoke-oldest-client` (the same suite with the Kafka 3.7 CLI tools, the oldest client CI tests, on every engine on x86 only); `smoke-report` puts every leg of both in one pass/fail grid in the step summary.

Merging goes through a merge queue: "Merge when ready" queues the PR, the queue re-runs CI on it against the current tip of `main`, and squash-merges it if everything is green: the PR title becomes the commit's subject and the PR description its body. Tier B is skipped on same-repo PRs only while the `MERGE_QUEUE` repository variable is `on`; with it unset, they run everything. Fork PRs can't read the variable, so Tier B is always skipped on them and runs in the queue. The other required checks come from `codeql.yml`, `workflow-lint.yml`, `dependencies.yml` and `pr-title.yml` (the PR title must follow Conventional Commits; see `CONTRIBUTING.md`).
Expand Down
3 changes: 3 additions & 0 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,9 @@ clippy:
fmt:
cargo fmt --all --check

shellcheck:
git ls-files -z '*.sh' '*.bash' | xargs -0 --no-run-if-empty shellcheck

miri:
cargo +nightly miri test --no-fail-fast --all-features

Expand Down
Loading