Skip to content

fix: [Task Plugin] Fix AliyunServerlessSpark plugin dependency conflicts and improve exception handling#18179

Open
includetts wants to merge 4 commits intoapache:devfrom
includetts:fix/aliyun-serverless-spark-deps
Open

fix: [Task Plugin] Fix AliyunServerlessSpark plugin dependency conflicts and improve exception handling#18179
includetts wants to merge 4 commits intoapache:devfrom
includetts:fix/aliyun-serverless-spark-deps

Conversation

@includetts
Copy link
Copy Markdown

Was this PR generated or assisted by AI?

NO

Purpose of the pull request

Fix two critical dependency version bugs that cause NoSuchMethodError and NoSuchFieldError at runtime when using the AliyunServerlessSpark task plugin, and improve exception handling to preserve root cause information for debugging.

Closes #18177 Closes #18178

Brief change log

  • Upgrade emr_serverless_spark20230808 from 1.0.0 to 2.4.1 in datasource plugin to match task plugin version
    • Upgrade credentials-java from 0.3.0 to 1.0.1 in both datasource and task plugins to satisfy tea-openapi:0.3.8 requirement
    • Propagate original exception message and cause in 3 catch blocks (buildClient, startJobRun, cancelJobRun) in AliyunServerlessSparkTask.java

Verify this pull request

Manually verified the change by testing locally:

  • Built both shade jars and confirmed Client.getTemplate() method exists in datasource plugin shade jar
    • Confirmed CredentialModel.providerName field present in both shade jars (credentials-java 1.0.1)
    • Confirmed exception cause chain is preserved via bytecode inspection

Pull Request Notice

Pull Request Notice

…datasource plugin emr SDK to 2.4.1 and credentials-java to 1.0.1

Updated dependencies and their versions in pom.xml for the Aliyun Serverless Spark datasource plugin.
@boring-cyborg
Copy link
Copy Markdown

boring-cyborg bot commented Apr 17, 2026

Thanks for opening this pull request! Please check out our contributing guidelines. (https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md)

Copy link
Copy Markdown
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow the pull request notice to use the correct title. And avoid unnecessary changes.

@SbloodyS SbloodyS added the first time contributor First-time contributor label Apr 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend first time contributor First-time contributor

Projects

None yet

2 participants