Skip to content

Add dbt-deltastream documentation #8014

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Nov 4, 2025
Merged

Add dbt-deltastream documentation #8014

merged 7 commits into from
Nov 4, 2025

Conversation

@Kayrnt
Copy link
Contributor

@Kayrnt Kayrnt commented Oct 9, 2025 •
edited
Loading

What are you changing in this pull request and why?

This PR adds comprehensive documentation for DeltaStream adapter configurations in dbt.
The PR covers:

  • Add documentation pages for the dbt deltastream adapter (setup & resource config)
  • Add pages in the sidebar
  • Update community adapter page

Related Links:

To learn more about the writing conventions used in the dbt Labs docs, see the Content style guide.

Checklist

ADDING OR REMOVING PAGES (if so, uncomment):

  • Add/remove page in website/sidebars.js
  • Provide a unique filename for new pages

@Kayrnt Kayrnt requested a review from a team as a code owner October 9, 2025 22:41
@welcome
Copy link

welcome bot commented Oct 9, 2025

Hello!👋 Thanks for contributing to the dbt product documentation and opening this pull request! ✨
We use Markdown and some HTML to write the dbt product documentation. When writing content, you can use our style guide and content types to understand our writing standards and how we organize information in the dbt product docs.
We'll review your contribution and respond as soon as we can. 😄

@vercel
Copy link

vercel bot commented Oct 9, 2025

@Kayrnt is attempting to deploy a commit to the dbt-labs Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions github-actions bot added the content Improvements or additions to content label Oct 9, 2025
@runleonarun runleonarun added the new contributor Label for first-time contributors label Oct 9, 2025
@mirnawong1
Copy link
Contributor

mirnawong1 commented Oct 13, 2025

hey @Kayrnt , thanks for creating this pr! some of the team is at our coalesce analytics engineering event this week and we're review this as soon as we can! cc @amychen1776

@amychen1776
Copy link
Contributor

amychen1776 commented Oct 28, 2025

Hello @Kayrnt ! Could you give us a bit more context on the changes?

@Kayrnt
Copy link
Contributor Author

Kayrnt commented Oct 28, 2025 •
edited
Loading

Hello @amychen1776, sure!
The PR aims to add to the docs.getdbt.com documentation website the dbt-deltastream adapter.

dbt-deltastream is the official dbt adapter for DeltaStream, a unified streaming and batch processing engine built on Apache Flink.

It's similar to already existing adapter such as ones for Decodable, Flink or Materialize (I'm not adding them but you're likely aware there's few dbt adapters that are not shared in the official doc).

Do you expect some more details I could share?

@vercel
Copy link

vercel bot commented Nov 4, 2025 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Updated (UTC)
docs-getdbt-com Ready Ready Preview Nov 4, 2025 9:53am

mirnawong1
mirnawong1 approved these changes Nov 4, 2025
View reviewed changes
website/docs/reference/resource-configs/deltastream-configs.md Outdated
dbt run-operation terminate_all_queries
```

### Restart a Query
Copy link
Contributor

@mirnawong1 mirnawong1 Nov 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Restart a Query
### Restart a query

website/docs/reference/resource-configs/deltastream-configs.md Outdated
dbt run-operation restart_query --args '{query_id: "<QUERY_ID>"}'
```

## Application Macro
Copy link
Contributor

@mirnawong1 mirnawong1 Nov 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Application Macro
## Application macro

website/docs/reference/resource-configs/deltastream-configs.md Outdated

## Application Macro

### Execute Multiple Statements as a Unit
Copy link
Contributor

@mirnawong1 mirnawong1 Nov 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Execute Multiple Statements as a Unit
### Execute multiple statements as a unit

website/docs/reference/resource-configs/deltastream-configs.md Outdated

## Troubleshooting

### Function Source Readiness
Copy link
Contributor

@mirnawong1 mirnawong1 Nov 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Function Source Readiness
### Function source readiness

website/docs/reference/resource-configs/deltastream-configs.md Outdated
3. **Dependency Order**: Ensure function sources are created before dependent functions
4. **Manual Retry**: If automatic retry fails, wait a few minutes and retry the operation

### File Attachment Issues
Copy link
Contributor

@mirnawong1 mirnawong1 Nov 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### File Attachment Issues
### File attachment issues

mirnawong1
mirnawong1 reviewed Nov 4, 2025
View reviewed changes
mirnawong1
mirnawong1 reviewed Nov 4, 2025
View reviewed changes
mirnawong1
mirnawong1 reviewed Nov 4, 2025
View reviewed changes
mirnawong1
mirnawong1 reviewed Nov 4, 2025
View reviewed changes
mirnawong1
mirnawong1 reviewed Nov 4, 2025
View reviewed changes
mirnawong1
mirnawong1 reviewed Nov 4, 2025
View reviewed changes
mirnawong1
mirnawong1 reviewed Nov 4, 2025
View reviewed changes
mirnawong1
mirnawong1 reviewed Nov 4, 2025
View reviewed changes
mirnawong1
mirnawong1 reviewed Nov 4, 2025
View reviewed changes
@mirnawong1 mirnawong1 enabled auto-merge November 4, 2025 09:51
@mirnawong1
Copy link
Contributor

mirnawong1 commented Nov 4, 2025

hey @Kayrnt , thanks for opening this up and for your patience! this looks great and merging after Amy's approval! i made some small editorial updates for consistency: https://docs-getdbt-com-git-fork-kayrnt-deltastream-setup-dbt-labs.vercel.app/reference/resource-configs/deltastream-configs

@mirnawong1 mirnawong1 merged commit 370a6fd into dbt-labs:current Nov 4, 2025
2 checks passed
@Kayrnt
Copy link
Contributor Author

Kayrnt commented Nov 4, 2025

Hey @mirnawong1,
Thanks a lot for the thorough review. I really appreciate it!
Totally agree about the header casing. It often ends up weird after AI-assisted edits (even though I tried to follow the repo’s guidelines, it looks like they weren’t fully picked up).
Slightly unrelated, but I think adding an AGENTS.md with some expectations and conventions could help avoid these small, repetitive fixes in the future.
All your changes look good to me! I didn’t think about the (the content is mostly copied from the adapter’s own README).
Thanks again for taking the time to go through this so carefully 🙏
Let me know if you need anything else from me!

@mirnawong1
Copy link
Contributor

mirnawong1 commented Nov 4, 2025

hey @Kayrnt ! we do have a style guide available to explain this but i LOVE this idea you're suggesting! i'm bringing this up to our docs team internally so we can tackle this and add to the repo . i can thne point our linter to it! thanks so much again and great contributions! ✨

@Kayrnt
Copy link
Contributor Author

Kayrnt commented Nov 4, 2025

Yes, I had a look but I was too optimistic on the LLM being to be able to apply all the guidelines by passing it as a reviewer to my changes. Yet I hope it gets better and follow these as the models get better on "large" contexts.
If there's indeed a way to have a Markdown linter that would do the changes (or pinpoint errors), it would probably add a lot of value and reduce the overhead for you down the line 👍
Happy to contribute and feel free to add this pending doc change as well in your backlog 😉

@mirnawong1
Copy link
Contributor

mirnawong1 commented Nov 4, 2025 •
edited
Loading

thanks so much and i'll take a look at that PR 🥇 !

fyi we're in the processing of reviewing this pr to add an agents.md file : #8134

@Kayrnt
Copy link
Contributor Author

Kayrnt commented Nov 4, 2025

Great, it will be very helpful 💯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mirnawong1 mirnawong1 mirnawong1 approved these changes

+1 more reviewer

@amychen1776 amychen1776 amychen1776 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

content Improvements or additions to content new contributor Label for first-time contributors

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Kayrnt @mirnawong1 @amychen1776 @runleonarun