Consolidate source + remove tests by fivetran-data-model-bot · Pull Request #67 · fivetran/dbt_salesforce

fivetran-data-model-bot · 2025-08-13T00:34:03Z

Summary

This PR is to combine the source and transform packages and remove tests not compatible with dbt Fusion 1.10.6+.

Please check the below automated updates were successful:

MANUAL UPDATES

If applicable, merge in the Q2 FY26 Automatic Package Updates PR.
If there are 2 docs.md files, consolidate the definitions the file in the main folder and remove the 2nd file.
Scan the README for non-standard references to the source package and remove or update.
Copy the passthrough vars or other non-standard vars from the source dbt_project.yml to the transform dbt_project.yml.
Check BK run logs for at least 2 warehouses
- No further deprecation warnings
- No union data "table not found" type warnings
Make sure below automatic updates were applied correctly.

Source + Transform Consolidation

Moved models/macros to models/staging/ and macros/staging/.
Updated all var('') references within models to use ref('stg_salesforce__').
Updated vars in dbt_project.yml from refs to sources.
Removed the source package from packages.yml.
Major version bump to v1.0.0 (or higher if applicable)

dbt Fusion Compatibility

Removed dbt_utils.unique_combination_of_columns test
Accepted values tests were removed where applicable (Shopify, NetSuite, QuickBooks).

YAML & Config Updates

Moved loaded_at_field under config: in src.yml, grouped with freshness
Added +schema and +materialization for the staging models in dbt_project.yml.

README

Removed references to the source package in:
- Overview
- Installation instructions
- Dependencies list
- Schema override examples

* Q2 FY26: Apply automated update. * Q2 FY26: Update auto-release workflow only. --------- Co-authored-by: fivetran-catfritz <111930712+fivetran-catfritz@users.noreply.github.com> Co-authored-by: Jamie Rodriguez <65564846+fivetran-jamie@users.noreply.github.com>

…com/fivetran/dbt_salesforce into MagicBot/consolidate-source-package

fivetran-joemarkiewicz

@fivetran-jamie a few requests to update before approval

fivetran-joemarkiewicz · 2025-08-14T19:38:53Z

    +materialized: table
    intermediate:
      +materialized: ephemeral


This config doesn't look right? There's no intermediate folder in this location.

This may be a FR for us to explore more in the future

Yeah I noticed that too and noted it to make a FR later. Also all of the models get written to the target schema - would we want to explore changing that later as well?

@fivetran-joemarkiewicz should i create a FR here

Yes, that would be great. I wouldn't want to change this just now, but we should in a future update.

Created FR #68 for us to evaluate in a future update.

fivetran-joemarkiewicz · 2025-08-14T19:43:17Z

+      - name: user
+        identifier: "{{ var('salesforce_user_identifier', 'user')}}"
+        config:
+          loaded_at_field: _fivetran_synced


Is this meant to be here?

@fivetran-joemarkiewicz We should remove this with the global config in place

As these are redundant, we ended up removing in this release.

fivetran-joemarkiewicz · 2025-08-14T19:43:27Z

+        identifier: "{{ var('salesforce_account_history_identifier', 'account')}}"
+        description: Represents historical records of individual accounts, which are organizations or people involved with your business (such as customers, competitors, and partners).
+        config:
+          loaded_at_field: _fivetran_synced


Is this meant to be here?

see below comment

As these are redundant, we ended up removing in this release.

fivetran-joemarkiewicz · 2025-08-14T19:43:36Z

+        identifier: "{{ var('salesforce_contact_history_identifier', 'contact')}}"
+        description: Represents the historical record of contacts, which are people associated with an account.
+        config:
+          loaded_at_field: _fivetran_synced


Is this meant to be here?

It was in the original source yml file, and it is valid according to dbt's docs

But I don't think it's necessary, definitely open to removing! I initially left it just to introduce as few changes as possible at the source/staging layer, but it is inconsistent with our other packages.

@fivetran-joemarkiewicz I think we can remove these with the global config in place, all the history models have this config, so it's redundant now.

Yeah let's remove them now that they're not necessary given the global config.

As these are redundant, we ended up removing in this release.

fivetran-joemarkiewicz · 2025-08-14T19:43:46Z

+        identifier: "{{ var('salesforce_opportunity_history_identifier', 'opportunity')}}"
+        description: Represents historical records of opportunities, which are sales or pending deals.
+        config:
+          loaded_at_field: _fivetran_synced


Is this meant to be here?

see reply above

As these are redundant, we ended up removing in this release.

Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com>

fivetran-joemarkiewicz

LGTM

fivetran-avinash

@fivetran-joemarkiewicz A few comments and suggestions before approval.

fivetran-avinash · 2025-08-18T17:48:23Z

+      - name: user
+        identifier: "{{ var('salesforce_user_identifier', 'user')}}"
+        config:
+          loaded_at_field: _fivetran_synced


@fivetran-joemarkiewicz We should remove this with the global config in place

fivetran-avinash · 2025-08-18T17:52:01Z

+        identifier: "{{ var('salesforce_contact_history_identifier', 'contact')}}"
+        description: Represents the historical record of contacts, which are people associated with an account.
+        config:
+          loaded_at_field: _fivetran_synced


@fivetran-joemarkiewicz I think we can remove these with the global config in place, all the history models have this config, so it's redundant now.

fivetran-avinash · 2025-08-18T18:07:26Z

    +materialized: table
    intermediate:
      +materialized: ephemeral


@fivetran-joemarkiewicz should i create a FR here

fivetran-avinash · 2025-08-18T18:11:24Z

+        tmp:
+          +materialized: view


There are no tmp models (or folder) in salesforce_history, should we remove this?

Good call, removing as it's not used.

fivetran-avinash · 2025-08-18T18:17:28Z

Should we remove the +materialized: table in line 8 since it's setting everything in Buildkite as a table by default; although everything appears to be a table right now, it could mess up any future configurations we set up in the base dbt_project.yml

Yeah that's a good callout. Removed the config to avoid any confusion in future updates.

fivetran-avinash · 2025-08-18T18:17:53Z

+    salesforce:
+      staging:
+        +materialized: table
+        tmp:


There are no tmp models (or folder) in salesforce, should we remove this?

Yes, removed this config as it's not being used.

Co-authored-by: Avinash Kunnath <108772760+fivetran-avinash@users.noreply.github.com>

fivetran-avinash

@fivetran-joemarkiewicz lgtm

Consolidate dbt source into transform

8108b6d

fivetran-data-model-bot added the next-release Include in the next release. label Aug 13, 2025

fivetran-data-model-bot and others added 2 commits August 13, 2025 19:07

manual

fec0eae

fivetran-jamie self-assigned this Aug 13, 2025

fivetran-jamie added docs:ready Triggers the docs generator workflow. and removed docs:ready Triggers the docs generator workflow. labels Aug 14, 2025

fivetran-jamie added 3 commits August 14, 2025 08:38

reorganize subfolders

ba87972

cofnirm schemas are good

b5187bd

revert

3d82ad5

fivetran-jamie added the docs:ready Triggers the docs generator workflow. label Aug 14, 2025

debugging

8e495b7

fivetran-jamie added docs:ready Triggers the docs generator workflow. and removed docs:ready Triggers the docs generator workflow. labels Aug 14, 2025

try this

490301b

fivetran-jamie added docs:ready Triggers the docs generator workflow. and removed docs:ready Triggers the docs generator workflow. labels Aug 14, 2025

Update generate-docs.yml

61d71c0

fivetran-catfritz added docs:ready Triggers the docs generator workflow. and removed docs:ready Triggers the docs generator workflow. labels Aug 14, 2025

github-actions Bot and others added 3 commits August 14, 2025 14:36

Generate dbt docs via GitHub Actions

b1ddae2

revert some stuff

57f38ae

Merge branch 'MagicBot/consolidate-source-package' of https://github.…

dbfdcbc

…com/fivetran/dbt_salesforce into MagicBot/consolidate-source-package

fivetran-jamie added docs:ready Triggers the docs generator workflow. and removed docs:ready Triggers the docs generator workflow. labels Aug 14, 2025

github-actions Bot and others added 3 commits August 14, 2025 15:24

Generate dbt docs via GitHub Actions

4439381

readme

898092d

readme again

99f10bf

fivetran-joemarkiewicz linked an issue Aug 14, 2025 that may be closed by this pull request

[Documentation] Move badges below H1 heading #58

Closed

4 tasks

fivetran-joemarkiewicz self-requested a review August 14, 2025 19:35

fivetran-joemarkiewicz requested changes Aug 14, 2025

View reviewed changes

Apply suggestions from code review

1445ad0

Co-authored-by: Joe Markiewicz <74217849+fivetran-joemarkiewicz@users.noreply.github.com>

fivetran-joemarkiewicz approved these changes Aug 18, 2025

View reviewed changes

fivetran-avinash requested changes Aug 18, 2025

View reviewed changes

fivetran-joemarkiewicz and others added 5 commits August 18, 2025 14:47

Update dbt_project.yml

260edfe

Co-authored-by: Avinash Kunnath <108772760+fivetran-avinash@users.noreply.github.com>

Update CHANGELOG.md

6294378

Co-authored-by: Avinash Kunnath <108772760+fivetran-avinash@users.noreply.github.com>

Update CHANGELOG.md

96cc6d9

Co-authored-by: Avinash Kunnath <108772760+fivetran-avinash@users.noreply.github.com>

Update dbt_project.yml

45239e9

Update dbt_project.yml

f4b1752

fivetran-joemarkiewicz removed the docs:ready Triggers the docs generator workflow. label Aug 18, 2025

fivetran-joemarkiewicz added 2 commits August 18, 2025 14:52

Update src_salesforce_history.yml

edb31bd

Update src_salesforce.yml

80ad2e1

fivetran-joemarkiewicz added the docs:ready Triggers the docs generator workflow. label Aug 18, 2025

Generate dbt docs via GitHub Actions

6c8e79d

fivetran-avinash approved these changes Aug 18, 2025

View reviewed changes

Update consistency_columns.sql

7701e2e

fivetran-joemarkiewicz merged commit 5a68ce8 into main Aug 18, 2025
9 checks passed

fivetran-joemarkiewicz deleted the MagicBot/consolidate-source-package branch August 18, 2025 23:14

Uh oh!

Conversation

fivetran-data-model-bot commented Aug 13, 2025 • edited by fivetran-jamie Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

MANUAL UPDATES

Source + Transform Consolidation

dbt Fusion Compatibility

YAML & Config Updates

README

Uh oh!

fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

Uh oh!

fivetran-avinash left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fivetran-data-model-bot commented Aug 13, 2025 •

edited by fivetran-jamie

Loading