Support building Proxy Native via GraalVM CE for JDK 25

[linghengqian]

For #29052 .

Changes proposed in this pull request:

• The current PR has some preceding PRs, located at Make ProxyConfigurationLoader compatible with classless runtime environments #38721 and Update the outdated HiveServer2 doc regarding third-party dependencies #38722 .
• Support building Proxy Native via GraalVM CE for JDK 25. These are the key steps for using https://github.com/graalvm/native-build-tools/releases/tag/1.0.0 .
• Increase the timeout for detecting the ShardingSphere proxy in nativetest to 5 minutes. Also see https://github.com/netty/netty/wiki/Java-24-and-sun.misc.Unsafe . Netty no longer utilizes Unsafe on JDK 25; consequently, ShardingSphere Proxy has slowed down, exposing issues that had previously gone unnoticed.

[INFO] Running org.apache.shardingsphere.test.natived.proxy.databases.MySQLTest
[INFO] MultiKeyMap stripe configuration: 32 locks for 16 cores
WARNING: A restricted method in java.lang.System has been called
WARNING: java.lang.System::loadLibrary has been called by io.netty.util.internal.NativeLibraryUtil in an unnamed module (file:/home/lingh/.m2/repository/io/netty/netty-common/4.2.9.Final/netty-common-4.2.9.Final.jar)
WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for callers in this module
WARNING: Restricted methods will be blocked in a future release unless native access is enabled

WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
WARNING: sun.misc.Unsafe::objectFieldOffset has been called by com.github.benmanes.caffeine.cache.UnsafeAccess (file:/home/lingh/.m2/repository/com/github/ben-manes/caffeine/caffeine/2.9.3/caffeine-2.9.3.jar)
WARNING: Please consider reporting this to the maintainers of class com.github.benmanes.caffeine.cache.UnsafeAccess
WARNING: sun.misc.Unsafe::objectFieldOffset will be removed in a future release
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 356.9 s -- in org.apache.shardingsphere.test.natived.proxy.databases.MySQLTest
[INFO] Running org.apache.shardingsphere.test.natived.proxy.databases.PostgresTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 180.0 s -- in org.apache.shardingsphere.test.natived.proxy.databases.PostgresTest

• Refactor org.apache.shardingsphere.test.natived.commons.util.ProxyTestingServer to use fix from Reset proxy backend executor across repeated ShardingSphereProxy lifecycles #38664 . Also fixes Completely prevent GraalVM Tracing Agent from collecting unreasonable GRM for unit tests related to org.apache.shardingsphere.test.natived.proxy.features.** #33206 .
• Prevent test/native/native-image-filter/extra-filter.json from grabbing the GraalVM Reachability Metadata of java.lang.**. Compared with GraalVM CE For JDK 24.0.2, GraalVM CE For JDK 25.0.2 has constraints on more JDK own classes. If our grab part of the GRM of java.lang.**, the native image will fail to build.
  • As for why GraalVM CE for JDK 24.0.2 did not trigger this problem, but GraalVM CE for JDK 25.0.2 did, I think only Oracle employees know the answer.🤷‍♂️🤦‍♂️
• Fixes org.apache.commons:commons-compress dependency conflict with org.apache.shardingsphere:shardingsphere-test-native. This is related to [repo] Dependencies version update ClickHouse/clickhouse-java#2696 .
• Fixes a bug where org.apache.shardingsphere.test.natived.jdbc.modes.cluster.ZookeeperTest would still make requests to the ZooKeeper server via the ContextManager after the ZooKeeper server was shut down. This was actually an effect of a recent change introduced in the ShardingSphere master branch, which now attempts to access the ZooKeeper server when the ContextManager is shut down.
• Apply the changes from [GR-72012] Available memory should never exceed total memory. oracle/graal#12720 to remove "-DjvmArgs=-XX:MaxRAMPercentage=85.0" from the CI.

---

Before committing this PR, I'm sure that I have checked the following options:

• My code follows the code of conduct of this project.
• I have self-reviewed the commit code.
• I have (or in comment I request) added corresponding labels for the pull request.
• I have passed maven check locally : ./mvnw clean install -B -T1C -Dmaven.javadoc.skip -Dmaven.jacoco.skip -e.
• I have made corresponding changes to the documentation.
• I have added corresponding unit tests for my changes.
• I have updated the Release Notes of the current development version. For more details, see Update Release Note

[terrymanu]

Decision

Merge Verdict: Not Mergeable

Reviewed Scope: latest PR head 5874874; .github/workflows, distribution/proxy-native, pom.xml, proxy/backend/core, test/native, docs, and reachability metadata. No SQL parser changes were detected. Need Expert Review: GraalVM/native-image and dependency compatibility.

Not Reviewed Scope: I did not run the full native-image suite locally; I relied on GitHub CI/API evidence and local diff inspection.

Positive Feedback

The direction is reasonable: the PR targets the real native-image path, updates the GraalVM build/test version, and the latest GraalVM rerun shows Ubuntu and Windows success for JDK 25.0.2.

Major Issues

[P0] Style gate is currently failing.

git diff --check fails with trailing whitespace in ProxyTestingServer.java. This violates the project gate in CODE_OF_CONDUCT.md (line 17), which requires coding standards and build checks to pass. Please run Spotless and recheck.

[P1] Shared config lookup precedence changed without a counterexample test.

ProxyConfigurationLoader#getResourceFile now prefers an existing filesystem path before classpath lookup. That is likely the right root-cause direction for #33206, but it changes shared Proxy startup/config-loading semantics used by normal Proxy bootstrap, e2e embedded Proxy, and native tests. I did not find a new test covering same-name/shadowing or fallback behavior. Please add direct validation for filesystem-first behavior, classpath fallback, and an adjacent counterexample where a classpath resource remains selected when no filesystem path exists.

Newly Introduced Issues

[P1] ZookeeperTest can leak the data source on failure paths.

The PR moves ResourceUtils.closeJdbcDataSource(logicDataSource) from @AfterEach into the happy path inside the TestingServer block in ZookeeperTest.java. If initEnvironment, processSuccess, or cleanEnvironment throws, the datasource is not closed before ZooKeeper is closed, which can reproduce the same shutdown-order risk this change is trying to avoid. Please use try/finally so close always happens before the TestingServer is closed.

Unrelated Changes

Please roll back or split the HiveServer2 doc change unless it is tied to this PR with evidence. The PR removes the commons-text exclusion in the user docs, but test/native/pom.xml still excludes commons-text; the PR body discusses commons-compress, not commons-text.

Verification

Commands run:

git fetch apache +5874874... +579de83...: exit 0

git diff --name-status apache/pr-38682-base..apache/pr-38682: exit 0

git diff --check apache/pr-38682-base..apache/pr-38682: exit 2, trailing whitespace found

git diff -G 'computeIfAbsent|ConcurrentHashMap' ...: exit 0, no hot-path computeIfAbsent change found

GitHub API checks: latest GraalVM rerun success on Ubuntu/Windows; earlier Windows run failed but was superseded by a later success

Next Step

Please fix the whitespace, add lookup-precedence/counterexample tests for ProxyConfigurationLoader, make ZookeeperTest cleanup exception-safe, and roll back or justify the unrelated HiveServer2 documentation change. Then this PR is worth another focused review.

[linghengqian]

Well, from my personal perspective, I can separate some things out.

[P0] Style gate is currently failing.

This seems to be an illusion caused by the agent you're using; neither CI nor I noticed the problem. As far as I remember, all files should have a blank line at the bottom.

[P1] Shared config lookup precedence changed without a counterexample test.

Adding unit tests to the current PR isn't very convenient for viewing, given GitHub's less-than-ergonomic web UI. I've split it into #38721. #38721 is the previous PR of this PR.

[P1] ZookeeperTest can leak the data source on failure paths.

This was intentional. If this native test fails while other native tests succeed, GraalVM Reachability Metadata should not be generated or the native test should be considered successful. GraalVM Native Build Tools simply did not consider this scenario of allowing failed unit tests in the unit test suite.

Please roll back or split the HiveServer2 doc change unless it is tied to this PR with evidence. The PR removes the commons-text exclusion in the user docs, but test/native/pom.xml still excludes commons-text; the PR body discusses commons-compress, not commons-text.

I split it at #38722 . This is just a dependency issue I'm verifying while working on the current PR.

[terrymanu]

Decision

• Merge Verdict: Not Mergeable
• Reviewed Scope: Latest PR head 587487419d1ca85903318004f43d0b2ccb377fb1 against apache/master; workflows, proxy-native Dockerfiles, pom.xml, GraalVM filter/metadata files at diff level, ProxyConfigurationLoader, native test utility/tests, and changed docs.
• Not Reviewed Scope: Full line-by-line validation of every generated reachability metadata deletion; local GraalVM native-image build; full dependency vulnerability/license scan. No SQL parser files were touched.
• Need Expert Review: Native image/GraalVM metadata maintainer review is still needed; dependency/supply-chain review is recommended for the ClickHouse JDBC version bump.

Positive Feedback

• The direction of avoiding classpath lookup for existing filesystem paths is aligned with the reported native-image resource-matching problem.
• Latest GitHub check-run scan for the head commit showed 83 latest unique check names with no non-success result.

Major Issues

• [Required] Make ZooKeeper datasource cleanup exception-safe (test/native/src/test/java/org/apache/shardingsphere/test/natived/jdbc/modes/cluster/ZookeeperTest.java:64)

  • Symptom: The datasource is closed only after initEnvironment(), processSuccess(), and cleanEnvironment() all succeed; afterEach() now only clears the system property.
  • Risk: If any statement before the close fails, TestingServer is closed while logicDataSource / ContextManager may remain open, so the original "requests to ZooKeeper after server shutdown" path is still reachable on the failure path.
  • Action: Please close logicDataSource in a finally block inside the TestingServer scope so it always closes before the ZooKeeper server, and keep or expand validation for the failure path.

• [Required] Add counterexample coverage for filesystem-first config lookup (proxy/backend/core/src/main/java/org/apache/shardingsphere/proxy/backend/config/ProxyConfigurationLoader.java:99)

  • Symptom: getResourceFile now returns an existing filesystem path before checking classpath and changes URL conversion to new File(url.toURI()), but this PR does not change ProxyConfigurationLoaderTest.
  • Risk: This is a lookup-precedence change shared by Proxy startup and native tests; without a same-name/shadowing or classpath-fallback counterexample, regressions in classpath config loading or encoded filesystem paths can slip through.
  • Action: Please add tests covering filesystem path wins when it exists, classpath fallback when no local file exists, and compatible server.yaml / config-*.yaml fallback under the new order.

• [Required] Fix formatting gate failure (test/native/src/test/java/org/apache/shardingsphere/test/natived/commons/util/ProxyTestingServer.java:78)

  • Symptom: git diff --check $(git merge-base apache/master apache/pr/38682) apache/pr/38682 fails with trailing whitespace at this line.
  • Risk: This violates the style/Spotless gate and can block the required quality checks even though functional checks pass.
  • Action: Please run ./mvnw spotless:apply -Pcheck -T1C, then recheck style.

Unrelated Changes

• HiveServer2 optional-plugin doc cleanup belongs outside this PR (docs/document/content/user-manual/shardingsphere-jdbc/optional-plugins/hiveserver2/_index.en.md:55)
  • Symptom: The PR removes the commons-text exclusion from HiveServer2 docs, while the main goal is Proxy Native on JDK 25 and the PR description says this was split to #38722.
  • Risk: Merging the current head brings a separate documentation/dependency-guidance change together with the native-image upgrade, blurring review and rollback scope.
  • Action: Please roll this back from this PR, or rebase this PR after the dedicated doc PR is merged so the latest diff contains only the intended JDK 25/native-image changes.

Next Steps

• Make the ZooKeeper native test cleanup exception-safe.
• Add direct ProxyConfigurationLoader regression tests for the new lookup precedence and fallback paths.
• Remove or rebase out the HiveServer2 doc-only change.
• Fix the trailing whitespace and rerun Spotless/checkstyle plus the relevant native/GraalVM CI path.

[linghengqian]

Waiting for #38721 to be reviewed.

[terrymanu]

Summary

• Merge Decision: Not Mergeable
• Reason: The latest head still has a failure-path cleanup gap and is missing a required release note for user-visible native-image/dependency changes.
• Expert Review Needed: Native-image/reachability metadata review is still needed after the blockers are fixed because this PR changes large generated metadata files and JDK 25 build behavior.

Positive Feedback

• The latest scope removed the previous ProxyConfigurationLoader and HiveServer2-doc blockers from this PR.
• The generated JSON files parse successfully, and no SQL parser files are touched.

Issues

• [P2] Add release note for user-visible native-image and ClickHouse changes (pom.xml:134)

  • Problem: The PR changes the ClickHouse JDBC version to 0.9.8, updates GraalVM native-image user docs to 25.0.2, changes Proxy Native platform guidance, and updates Proxy Native Docker/workflow build versions, but RELEASE-NOTES.md is not in the latest GitHub file list.
  • Impact: Users and operators will not see the upgrade impact for the GraalVM CE baseline, Proxy Native platform/build behavior, and ClickHouse optional dependency version.
  • Required Change: Please add a RELEASE-NOTES.md entry, likely under Enhancements, describing the user-visible outcome and affected areas, or explicitly delegate the release note to an accepted umbrella PR.

• [P2] Make ZooKeeper native-test cleanup failure-path safe (test/native/src/test/java/org/apache/shardingsphere/test/natived/jdbc/modes/cluster/ZookeeperTest.java:64)

  • Problem: afterEach() calls ResourceUtils.closeJdbcDataSource(logicDataSource) before testingServer.close() without a null guard or finally; if data source creation fails or data source close throws, the ZooKeeper TestingServer close is skipped.
  • Impact: The previous shutdown-order issue is fixed on the normal path, but setup/cleanup failure paths can still leak the ZooKeeper test server and affect later native-test or metadata-generation runs.
  • Required Change: Please make cleanup close both resources reliably while preserving data-source-before-ZooKeeper ordering when the data source exists, for example with null-safe cleanup and a finally for testingServer.close().

Multi-Round Comparison

• Fixed: The shared ProxyConfigurationLoader lookup-precedence blocker is no longer in this PR diff after the preceding merged change.
• Fixed: The HiveServer2 unrelated doc cleanup is no longer in the latest PR diff.
• Partially fixed: ZookeeperTest now closes the data source before ZooKeeper on the normal path, but the failure path remains unresolved and is tracked above.
• Verification note: git diff --check still reports trailing whitespace in ProxyTestingServer.java:78 and ZookeeperTest.java:54; I did not treat this as a standalone blocker without running the repository Spotless/Checkstyle gates.

Review Details

• Reviewed Scope: Latest PR head 855b6a34315af444996ec8abdc3bbd261ee7df35; local merge-base 21c96bb9a02f8ad2a6e65475ff4886a6aa22e676; local triple-dot file list matched GitHub /pulls/38682/files exactly. Reviewed workflows, Proxy Native Dockerfiles, GraalVM native-image docs, ClickHouse docs/version, reachability metadata diffs, ProxyTestingServer, ZookeeperTest, MySQL/PostgreSQL proxy native timeout changes, and native metadata filters.
• Not Reviewed Scope: Full native-image build/nativeTest execution, full line-by-line semantic validation of every generated metadata deletion, full ClickHouse dependency vulnerability/license scan, and GitHub Actions/CI status.
• Verification: git fetch apache master:refs/remotes/apache/master exit 0; forced PR ref refresh exit 0; GitHub/local file-list diff exit 0; JSON syntax validation for changed JSON files exit 0; git diff -G 'computeIfAbsent|ConcurrentHashMap' exit 0 with no matching diff; parser-file search exit 1 with no parser files; git diff --check exit 2 with trailing-whitespace diagnostics only; no Maven/native build was run locally.
• Release Note / User Docs: User docs were reviewed at diff level; official GraalVM docs support the filter rule ordering and GraalVM 25.0.2 macOS x64 removal (https://www.graalvm.org/latest/reference-manual/native-image/metadata/AutomaticMetadataCollection/, https://www.graalvm.org/release-notes/JDK_25/). Release note is required and missing.

[linghengqian]

[P2] Add release note for user-visible native-image and ClickHouse changes (pom.xml:134)

• Is it really necessary to add a release note? The clickhouse jdbc driver is just a test dependency in the master branch. There are no changes to the sql parser in the integration test and unit test. And we do not prevent the use of old versions of graalvm ce to compile shardingsphere proxy native, refer to https://shardingsphere.apache.org/document/current/en/user-manual/shardingsphere-jdbc/graalvm-native-image/

Users can still use old versions of Oracle GraalVM such as 21.0.8-graal on SDKMAN! to build ShardingSphere’s GraalVM Native Image product. But this will cause the failure of building GraalVM Native Image when integrating some third-party dependencies.

[P2] Make ZooKeeper native-test cleanup failure-path safe (test/native/src/test/java/org/apache/shardingsphere/test/natived/jdbc/modes/cluster/ZookeeperTest.java:64)

Closing the logical data source of shardingsphere after closing the zookeeper server will destroy the unit test. This conflicts with the processing of shardingsphere jdbc, because closing the logical data source of shardingsphere requires connecting to the zookeeper server.

Java's finally keyword has no meaning here.