KAFKA-19174 Gradle version upgrade 8 -->> 9 #19513

dejan2609 · 2025-04-18T14:15:19Z

@reviewers:

if you want to save some time please skip straight to this comment:
⬇️
#19513 (comment)

Related JIRA ticket:

https://issues.apache.org/jira/browse/KAFKA-19174

List of changes:

prerequisite Jira ticket:
KAFKA-19591
- at the moment Kafka trunk branch is using Gradle version 8.14.1
- gradlew is still referencing an older template file version
  (unixStartScript.txt from a Gradle 8.7.0)
- it means that gradlew is missing all recent changes for a template
  file unixStartScript.txt
Gradle version will be upgraded: 8.14.1 -->> 9.0.0
Gradle Shadow plugin version will be upgraded: 8.3.6 -->> 9.0.2
build logic will be refactored in order to accommodate Gradle 9
breaking changes
(optional): dozen of Gradle plugin versions will be upgraded also
another JIRA ticket solved:
KAFKA-16801

Related links:

Note: new Gradle version brings up some breaking changes (as
always 😃); Kafka build with Gradle 9 has same issues as other
projects:

github-actions · 2025-04-26T03:18:52Z

A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the triage label
should be removed to prevent this automation from happening again.

github-actions · 2025-07-25T03:52:02Z

This PR is being marked as stale since it has not had any activity in 90 days. If you
would like to keep this PR alive, please leave a comment asking for a review. If the PR has
merge conflicts, update it with the latest from the base branch.

If you are having difficulty finding a reviewer, please reach out on the [mailing list](https://kafka.apache.org/contact).

If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed.

github-actions · 2025-07-27T03:38:36Z

A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the triage label
should be removed to prevent this automation from happening again.

dejan2609 · 2025-08-01T08:46:13Z

Update: project compilation works as expected (when powered by Gradle 9) but binary release creation fails.
More work has to be done in order to accommodate to Gradle breaking changes.

github-actions · 2025-08-03T03:40:51Z

A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the triage label
should be removed to prevent this automation from happening again.

chia7712 · 2025-08-06T16:50:37Z

@dejan2609 any update?

github-actions · 2025-08-08T03:37:34Z

A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the triage label
should be removed to prevent this automation from happening again.

dejan2609 · 2025-08-08T06:28:01Z

@chia7712
Will provide an update during the weekend (there are two problems with Gradle 9 build, will explain in more details and provide some insights).

chia7712 · 2025-08-08T07:32:37Z

@dejan2609 thanks!

dejan2609 · 2025-08-08T16:08:41Z

@chia7712 feel free to check this related PR: #20327

dejan2609 · 2025-08-08T16:19:52Z

Pending: I will also explain second problem (specific to Gradle 9, that is).

github-actions · 2025-08-10T03:37:35Z

A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the triage label
should be removed to prevent this automation from happening again.

dejan2609 · 2025-09-01T06:15:48Z

I just accidentally deleted remote branch (from my command line via git push origin :<KAFKA-ticketNumber>).
Will recover it instantly.

dejan2609 · 2025-09-01T06:16:47Z

Done, branch recovered 😌

dejan2609 · 2025-09-01T07:19:33Z

FYI @chia7712

Just to sum it up here:

Gradle version upgrade (8 to 9) is completed (see commits) ✅ - I did a lots of testing and everything looks fine (when compiled locally, that is)
but alas: Github Actions build with Gradle 9 has some issues related to Develocity ❌; see here for more details: KAFKA-19654 [WIP] test issues with GitHub actions builds <<--->> Develocity instance (for both trunk/Gradle 8 and attempted upgrade to Gradle 9) #20450 (and please consider providing workflow runs for that PR so I can test some scenarios to bypass these issues)

In a broader context:

GitHub Actions <<--->> Develocity issues are blocking Gradle upgrade from 8 to 9 (KAFKA-19654)
Gradle 9.1 is required to run Java 25 builds (KAFKA-19664)
- https://github.com/gradle/gradle/releases/tag/v9.1.0-RC2
- https://docs.gradle.org/9.1.0-rc-2/userguide/compatibility.html#java_runtime

github-actions · 2025-09-02T03:24:00Z

A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the triage label
should be removed to prevent this automation from happening again.

github-actions · 2025-09-04T03:15:51Z

A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the triage label
should be removed to prevent this automation from happening again.

…rsion (i.e. file `unixStartScript.txt`) was being referenced

details: - gradle: 8.14.1 -->> 9.0.0 - https://github.com/gradle/gradle/releases/tag/v9.0.0 - https://gradle.org/whats-new/gradle-9 - https://docs.gradle.org/9.0.0/userguide/upgrading_major_version_9.html#changes_major_9 - shadow plugin: 8.3.6 -->> 9.0.2 - https://github.com/GradleUp/shadow/releases/tag/9.0.0 - https://github.com/GradleUp/shadow/releases/tag/9.0.1 - https://github.com/GradleUp/shadow/releases/tag/9.0.2 refactoring (performed in order to accomodate Gradle 9 breaking changes): - tasks `siteDocsTar` and `releaseTarGz` are moved out of `core` (and into a new module: `distribution`); GA workflows and Python scripts are changed accordingly - `preserveFileTimestamps` Tar taks property value explicitly set to `true` (default values: Gradle 8: `true`, Gradle 9: 'false') - files and folders permissions are introduced - git placeholder file (.gitkeep) is added in order to preserve `distribution` folder (Apache Rat check for that file is skipped) - `wrapper.gradle` changes: no need to check number of dots for Gradle versions >=9 anymore - redundant 'gradleVersion' ext property is removed (in favor of 'versions.gradle' usage) - folder pattern 'bin/' is added into .gitignore

… dependencies/plugins and GitHub Actions details: - `setup-gradle` GitHub Action: 4.3.0 -->> 4.4.2 (note: this version was tested against Gradle 9: https://github.com/gradle/actions/releases/tag/v4.4.2) - grgit: 4.1.1 -->> 5.3.0 - gradle versions plugin: 0.48.0 -->> 0.52.0 - owasp dependency check : 8.2.1 -->> 12.1.3 - spotbugs: 6.2.3 -->> 6.2.5 - spotless: 6.25.0 -->> 7.2.1 - Zinc Scala compiler: 1.9.2 -->> 1.10.8 - Scala fmt: 3.7.14 -->> 3.9.9 - Scala Coverage: - scoverage: 2.0.11 -->> 2.3.0 - Scala version for scoverage: 2.13.9 -->> 2.13.16 (i.e. `org.scoverage:scalac-scoverage-plugin_2.13.16:2.3.0` will be used) - scoverage gradle plugin: 8.0.3 -->> 8.1

… a `upgrade-system-tests*` submodules related link: https://docs.gradle.org/9.0.0/userguide/upgrading_major_version_9.html#test_task_fails_when_no_tests_are_discovered

github-actions · 2025-09-06T03:15:11Z

A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the triage label
should be removed to prevent this automation from happening again.

chia7712

@dejan2609 thanks for this patch. It is coool that we could use gradle 9.

chia7712 · 2025-09-06T15:28:26Z

.github/actions/setup-gradle/action.yml

@@ -42,7 +42,7 @@ runs:
        distribution: temurin
        java-version: ${{ inputs.java-version }}
    - name: Setup Gradle
-      uses: gradle/actions/setup-gradle@94baf225fe0a508e581a564467443d0e2379123b # v4.3.0


Could you please use the git commit hash instead, to follow the Apache policy (see below)?

You MUST pin all external actions to the specific git hash (SHA1) of the action that has been reviewed for use by the project. For instance, you MUST pin foobar/baz-action@8843d7f92416211de9ebb963ff4ce28125932878.

reference: https://infra.apache.org/github-actions-policy.html

chia7712 · 2025-09-06T15:29:54Z

.github/workflows/build.yml

@@ -164,7 +164,7 @@ jobs:
        # Check if there are any empty files under ./site-docs/generated, If any empty files are found, print an error 
        # message and list the empty files
        run: |
-          tar zxvf core/build/distributions/kafka_2.13-$(./gradlew properties | grep version: | awk '{print $NF}' | head -n 1)-site-docs.tgz
+          tar zxvf distribution/build/distributions/kafka_2.13-$(./gradlew properties | grep version: | awk '{print $NF}' | head -n 1)-site-docs.tgz


Changing the path may break downstream CI or workflows. Could you please provide more details about this change?

chia7712 · 2025-09-06T15:30:32Z

gradle/dependencies.gradle

@@ -119,13 +119,13 @@ versions += [
  // When updating the scalafmt version please also update the version field in checkstyle/.scalafmt.conf. scalafmt now
  // has the version field as mandatory in its configuration, see
  // https://github.com/scalameta/scalafmt/releases/tag/v3.1.0.
-  scalafmt: "3.7.14",
-  scoverage: "2.0.11",
+  scalafmt: "3.9.9",


Are those changes necessary?

dejan2609 · 2025-09-07T08:14:12Z

Thank you for a review @chia7712.
Good points, will address them and update commits ASAP.

github-actions bot added triage PRs from the community build Gradle build or GitHub Actions small Small PRs labels Apr 18, 2025

dejan2609 changed the title ~~[KAFKA-19174] Gradle version upgrade 8 -->> 9~~ [Work in progress] [KAFKA-19174] Gradle version upgrade 8 -->> 9 Apr 18, 2025

github-actions bot added the needs-attention label Apr 26, 2025

github-actions bot added stale Stale PRs and removed needs-attention stale Stale PRs labels Jul 25, 2025

github-actions bot added the needs-attention label Jul 27, 2025

dejan2609 force-pushed the KAFKA-19174 branch from db25d8e to 98ebfe2 Compare August 1, 2025 08:41

github-actions bot removed the needs-attention label Aug 2, 2025

github-actions bot added the needs-attention label Aug 3, 2025

github-actions bot removed the needs-attention label Aug 7, 2025

github-actions bot added the needs-attention label Aug 8, 2025

dejan2609 mentioned this pull request Aug 8, 2025

KAFKA-19591 solving issues with gradlew content: an old template version was being referenced #20327

Closed

dejan2609 force-pushed the KAFKA-19174 branch from 98ebfe2 to b2dd7ee Compare August 8, 2025 16:46

github-actions bot removed the needs-attention label Aug 9, 2025

github-actions bot added the needs-attention label Aug 10, 2025

github-actions bot removed the needs-attention label Sep 1, 2025

dejan2609 closed this Sep 1, 2025

dejan2609 deleted the KAFKA-19174 branch September 1, 2025 06:13

dejan2609 restored the KAFKA-19174 branch September 1, 2025 06:16

dejan2609 reopened this Sep 1, 2025

dejan2609 mentioned this pull request Sep 1, 2025

KAFKA-19654 [WIP] test issues with GitHub actions builds <<--->> Develocity instance (for both trunk/Gradle 8 and attempted upgrade to Gradle 9) #20450

Open

dejan2609 changed the title ~~KAFKA-19174 KAFKA-19591 Gradle version upgrade 8 -->> 9~~ KAFKA-19174 Gradle version upgrade 8 -->> 9 Sep 1, 2025

github-actions bot added needs-attention and removed needs-attention labels Sep 2, 2025

dejan2609 force-pushed the KAFKA-19174 branch from f9fcc36 to c3975b7 Compare September 3, 2025 11:42

github-actions bot added the needs-attention label Sep 4, 2025

dejan2609 force-pushed the KAFKA-19174 branch from c3975b7 to 1796129 Compare September 4, 2025 16:22

github-actions bot removed the needs-attention label Sep 5, 2025

dejan2609 added 4 commits September 5, 2025 21:35

KAFKA-19591 solving issues with gradlew content: an old template ve…

5fe4c95

…rsion (i.e. file `unixStartScript.txt`) was being referenced

KAFKA-16801 do not fail Gradle 9 build when no test are discovered in…

a8e9978

… a `upgrade-system-tests*` submodules related link: https://docs.gradle.org/9.0.0/userguide/upgrading_major_version_9.html#test_task_fails_when_no_tests_are_discovered

dejan2609 force-pushed the KAFKA-19174 branch from 1796129 to a8e9978 Compare September 5, 2025 19:36

github-actions bot added the needs-attention label Sep 6, 2025

chia7712 reviewed Sep 6, 2025

View reviewed changes

chia7712 mentioned this pull request Sep 6, 2025

MINOR: update gradle from 8.14.1 to 8.14.3 #20495

Merged

github-actions bot removed needs-attention triage PRs from the community labels Sep 7, 2025

KAFKA-19174 Gradle version upgrade 8 -->> 9 #19513

Are you sure you want to change the base?

KAFKA-19174 Gradle version upgrade 8 -->> 9 #19513

Conversation

dejan2609 commented Apr 18, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

@reviewers:

Related JIRA ticket:

Uh oh!

github-actions bot commented Apr 26, 2025

Uh oh!

github-actions bot commented Jul 25, 2025

Uh oh!

github-actions bot commented Jul 27, 2025

Uh oh!

dejan2609 commented Aug 1, 2025

Uh oh!

github-actions bot commented Aug 3, 2025

Uh oh!

chia7712 commented Aug 6, 2025

Uh oh!

github-actions bot commented Aug 8, 2025

Uh oh!

dejan2609 commented Aug 8, 2025

Uh oh!

chia7712 commented Aug 8, 2025

Uh oh!

dejan2609 commented Aug 8, 2025

Uh oh!

dejan2609 commented Aug 8, 2025

Uh oh!

github-actions bot commented Aug 10, 2025

Uh oh!

dejan2609 commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dejan2609 commented Sep 1, 2025

Uh oh!

dejan2609 commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 2, 2025

Uh oh!

github-actions bot commented Sep 4, 2025

Uh oh!

github-actions bot commented Sep 6, 2025

Uh oh!

chia7712 left a comment

Choose a reason for hiding this comment

Uh oh!

chia7712 Sep 6, 2025

Choose a reason for hiding this comment

Uh oh!

chia7712 Sep 6, 2025

Choose a reason for hiding this comment

Uh oh!

chia7712 Sep 6, 2025

Choose a reason for hiding this comment

Uh oh!

dejan2609 commented Sep 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

dejan2609 commented Apr 18, 2025 •

edited by github-actions bot

Loading

dejan2609 commented Sep 1, 2025 •

edited

Loading

dejan2609 commented Sep 1, 2025 •

edited

Loading

dejan2609 commented Sep 7, 2025 •

edited

Loading