Skip to content

packaging: pass the elements to jq by pipes instead of command line - #12226

Merged
edsiper merged 1 commit into
masterfrom
lecaros-release-script
Aug 5, 2026
Merged

packaging: pass the elements to jq by pipes instead of command line#12226
edsiper merged 1 commit into
masterfrom
lecaros-release-script

Conversation

@lecaros

@lecaros lecaros commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Pass the elements to jq by pipe instead of command line and reduce the versions to display in the index


Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • New Features

    • Added configurable minimum major-version filtering for package catalog entries, including version metadata generation.
    • Improved numeric validation/normalization of the minimum setting and ensured consistent eligibility filtering across platforms.
    • Support for including all major versions when the minimum is set to 0.
  • Tests

    • Expanded catalog/index tests with legacy fixtures, negative/non-numeric validation, and a 500-entry versions.json stress test.

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 80113d0b-e17e-4f2e-9268-545db69df84c

📥 Commits

Reviewing files that changed from the base of the PR and between cf50b7e and c4e6a37.

📒 Files selected for processing (3)
  • packaging/build-catalog.awk
  • packaging/generate-packages-index.sh
  • packaging/testing/test-packages-index.sh
🚧 Files skipped from review as they are similar to previous changes (3)
  • packaging/testing/test-packages-index.sh
  • packaging/build-catalog.awk
  • packaging/generate-packages-index.sh

📝 Walkthrough

Walkthrough

The package catalog now applies a configurable minimum major version to paths, artifacts, and emitted versions. The package index script passes this setting to the AWK builder. Tests cover legacy versions, invalid input, and large catalogs.

Changes

Catalog filtering and index generation

Layer / File(s) Summary
Catalog eligibility rules
packaging/build-catalog.awk
Adds shared version and path eligibility functions. Schema, Windows, macOS, Linux, filter-mode, and version output paths use the minimum major version.
Package index integration
packaging/generate-packages-index.sh
Adds and documents MIN_CATALOG_MAJOR, validates it, filters package-tree entries, and streams version rows into jq for versions.json.
Filtering and large-catalog tests
packaging/testing/test-packages-index.sh
Adds legacy 2.1.0 fixtures, validates default and permissive filtering, rejects invalid minimum-major values, and checks a 500-entry catalog.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant PackageIndex as generate-packages-index.sh
  participant CatalogBuilder as build-catalog.awk
  participant JSONBuilder as jq
  PackageIndex->>CatalogBuilder: pass MIN_CATALOG_MAJOR
  CatalogBuilder-->>PackageIndex: filtered package paths and versions
  PackageIndex->>JSONBuilder: stream version rows
  JSONBuilder-->>PackageIndex: build versions.json
Loading

Possibly related PRs

Suggested reviewers: celalettin1286

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title focuses on a specific implementation detail (piping to jq) that represents one aspect of the changes, but the changeset encompasses broader modifications including version filtering, validation, testing, and catalog eligibility logic.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch lecaros-release-script

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
packaging/build-catalog.awk (1)

291-295: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Apply the configured catalog filter to Linux artifacts.

With MIN_CATALOG_MAJOR=0, Line 291 accepts 2.1.0 and Line 295 calls note_linux. note_linux then drops the RPM because version_includes_linux accepts only 4.x and 5.x. This also drops Linux artifacts for future major versions such as 6.x. The documented permissive mode does not include all catalog artifacts.

  • packaging/build-catalog.awk#L291-L295: Remove the fixed 4.x/5.x gate from Linux artifact registration. Use the configured version_in_catalog result.
  • packaging/testing/test-packages-index.sh#L281-L286: Assert that fluent-bit-2.1.0-1.x86_64.rpm is present in the 2.1.0 Linux artifacts in versions.json.

As per coding guidelines, validate both success and failure paths, including invalid payloads, boundary sizes, and null or missing fields.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packaging/build-catalog.awk` around lines 291 - 295, The Linux artifact
registration in note_linux must rely solely on version_in_catalog(v), removing
the fixed 4.x/5.x filtering so configured permissive mode includes all catalog
versions; update packaging/build-catalog.awk lines 291-295 accordingly. Add a
test in packaging/testing/test-packages-index.sh lines 281-286 asserting
fluent-bit-2.1.0-1.x86_64.rpm appears among the 2.1.0 Linux artifacts in
versions.json.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packaging/build-catalog.awk`:
- Line 3: Update the initialization around min_catalog_major so
MIN_CATALOG_MAJOR is validated as a present, nonnegative integer before numeric
coercion; reject nonnumeric and negative values instead of allowing them to
become the permissive 0 setting. Add coverage for valid values, nonnumeric and
negative inputs, and relevant missing/null or boundary cases, verifying both
accepted and rejected paths.

In `@packaging/generate-packages-index.sh`:
- Around line 208-217: Update the build-catalog.awk invocation in
generate-packages-index.sh to remove the failure suppression and ensure a
nonzero awk exit status terminates the script before the filtered tree is used.
Preserve the existing empty-output handling only for successful filtering, and
validate both successful output and filter failures, including invalid or
missing catalog data and boundary-sized inputs.

---

Outside diff comments:
In `@packaging/build-catalog.awk`:
- Around line 291-295: The Linux artifact registration in note_linux must rely
solely on version_in_catalog(v), removing the fixed 4.x/5.x filtering so
configured permissive mode includes all catalog versions; update
packaging/build-catalog.awk lines 291-295 accordingly. Add a test in
packaging/testing/test-packages-index.sh lines 281-286 asserting
fluent-bit-2.1.0-1.x86_64.rpm appears among the 2.1.0 Linux artifacts in
versions.json.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 13dd62e4-153b-4abb-8b07-26f19a973e80

📥 Commits

Reviewing files that changed from the base of the PR and between cf50b7e and 78ef159.

📒 Files selected for processing (3)
  • packaging/build-catalog.awk
  • packaging/generate-packages-index.sh
  • packaging/testing/test-packages-index.sh

Comment thread packaging/build-catalog.awk
Comment thread packaging/generate-packages-index.sh
Signed-off-by: lecaros <lecaros@chronosphere.io>
@lecaros
lecaros force-pushed the lecaros-release-script branch from 78ef159 to c4e6a37 Compare August 5, 2026 15:27
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@lecaros
lecaros requested a review from cosmo0920 August 5, 2026 15:42
@edsiper
edsiper merged commit ec7ee5d into master Aug 5, 2026
21 checks passed
@edsiper
edsiper deleted the lecaros-release-script branch August 5, 2026 17:02
@edsiper edsiper added this to the Fluent Bit v5.1 milestone Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants