🐛 bug(55827):[source-kafka]: added AWS MSK IAM Auth jar to classpath an… #55828
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@nickcopley-sa is attempting to deploy a commit to the Airbyte Growth Team on Vercel. A member of the Team first needs to authorize it. |
nickcopley-sa
had a problem deploying
to
community-ci
GitHub Actions
Error
nickcopley-sa
had a problem deploying
to
community-ci-auto
GitHub Actions
Failure
nickcopley-sa
had a problem deploying
to
community-ci
GitHub Actions
Error
…d config required. Also found after it connected JSON format messages are broken so I fixed it in order to test it.
nickcopley-sa
force-pushed
the
nickcopley-sa-55827-kafka-msk
branch
from
c4dd664 to
82776c9
Compare
nickcopley-sa
had a problem deploying
to
community-ci-auto
GitHub Actions
Failure
nickcopley-sa
had a problem deploying
to
community-ci
GitHub Actions
Error
nickcopley-sa
had a problem deploying
to
community-ci
GitHub Actions
Error
nickcopley-sa
had a problem deploying
to
community-ci
GitHub Actions
Error
nickcopley-sa
had a problem deploying
to
community-ci
GitHub Actions
Error
nickcopley-sa
had a problem deploying
to
community-ci-auto
GitHub Actions
Failure
marcosmarxm
temporarily deployed
to
community-ci
GitHub Actions
Inactive
marcosmarxm
temporarily deployed
to
community-ci-auto
GitHub Actions
Inactive
marcosmarxm
temporarily deployed
to
community-ci
GitHub Actions
Inactive
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
Valgard
pushed a commit
to mayflower/airbyte
that referenced
this pull request
Mar 25, 2025
…an… (airbytehq#55828) Co-authored-by: Marcos Marx <[email protected]>
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change is to address issue (#55827).
What
AWS MSK uses SASL for IAM based access.
How
SASL OAUTHBEARER support was added in 0.3.0 however MSK uses its own handler and therefore this needs adding to the classpath. (https://github.com/aws/aws-msk-iam-auth). I have then added a option in the dropdown to configure MSK IAM to ease configuration for people.
When testing it connected fine but the messages were all blank. I found it was this issue (#29464) It seems the JSON formatter has been broken a while. Airbyte expects to get data in a JSON data node keyed with "value". However current the record value (the actual data) is added to the airbyte record one level too high. I've added a new "value" node and left the fields at the higher level too.
Review guide
airbyte-integrations/connectors/source-kafka/build.gradle
airbyte-integrations/connectors/source-kafka/src/main/resources/spec.json
airbyte-integrations/connectors/source-kafka/src/main/java/io/airbyte/integrations/source/kafka/format/AbstractFormat.java
airbyte-integrations/connectors/source-kafka/src/main/java/io/airbyte/integrations/source/kafka/format/JsonFormat.java
User Impact
I've bumped it to 0.4.0 from 0.3.0 given its small new feature.
Can this PR be safely reverted and rolled back?