Skip to content

Update loader status #1315

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Update loader status #1315

wants to merge 1 commit into from

Conversation

stanch
Copy link
Collaborator

@stanch stanch commented Jul 10, 2025

No description provided.

@stanch stanch requested review from istreeter and mscwilson July 10, 2025 11:18
Copy link

netlify bot commented Jul 10, 2025

Deploy Preview for snowplow-docs ready!

Name Link
🔨 Latest commit ccc7e97
🔍 Latest deploy log https://app.netlify.com/projects/snowplow-docs/deploys/686fa19fd5b8fd00081b35e5
😎 Deploy Preview https://deploy-preview-1315--snowplow-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Copy link
Contributor

@istreeter istreeter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thank you for removing the postgres loader.

| GCS | Delta | Databricks | [Lake Loader](/docs/api-reference/loaders-storage-targets/lake-loader/index.md) | Early release |
| GCS | JSON | BigQuery | [GCS Loader](/docs/api-reference/loaders-storage-targets/google-cloud-storage-loader/index.md) | Only recommended for [failed events](/docs/fundamentals/failed-events/index.md) |
| GCS | Delta | Databricks | [Lake Loader](/docs/api-reference/loaders-storage-targets/lake-loader/index.md) | Production-ready |
| GCS | JSON | BigQuery | [GCS Loader](/docs/api-reference/loaders-storage-targets/google-cloud-storage-loader/index.md) | Only recommended for [raw failed events](/docs/fundamentals/failed-events/index.md) |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh are they called raw failed events now?


:::tip

Please note that currently the S3 _Delta_ loader is not compatible with Databricks. The loader uses [DynamoDB tables for mutually exclusive writes to S3](https://docs.delta.io/latest/delta-storage.html#multi-cluster-setup), a feature of Delta. Databricks, however, does not support this (as of July 2025). This means that it’s not possible to alter the data via Databricks (e.g. to run `OPTIMIZE` or to delete PII).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not possible to alter the data

I paused on this line, because the word "alter" makes me think of DDL statements, like ALTER TABLE ADD COLUMN. But I don't have a suggestion for a better word. And you said "alter the data" so that's ok.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants