Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -74,3 +74,4 @@ venv.bak/

# Project-specific
CLAUDE.md
worktrees/
10 changes: 9 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
# dbt_asana v1.4.0

[PR #56](https://github.com/fivetran/dbt_asana/pull/56) includes the following updates:

## Under the Hood
- Migrates the `union_connections`, `apply_source_relation`, and `partition_by_source_relation` macros to the `dbt_fivetran_utils` package.
- Adds the `fivetran_using_source_casing` variable for case-sensitive destination support. When enabled, downstream transformations respect source casing to ensure consistent results. See the [Additional Configurations](https://github.com/fivetran/dbt_asana/#source-casing-for-case-sensitive-destinations) section of the README for details.

# dbt_asana v1.3.0

[PR #50](https://github.com/fivetran/dbt_asana/pull/50) includes the following updates:
Expand Down Expand Up @@ -194,4 +202,4 @@ PR [#19](https://github.com/fivetran/dbt_asana/pull/19) introduced the following
- Please note, if you are installing a version of `dbt_utils` in your `packages.yml` that is not in the range above then you will encounter a package dependency error.

# dbt_asana v0.1.0 -> v0.4.0
Refer to the relevant release notes on the Github repository for specific details for the previous releases. Thank you!
Refer to the relevant release notes on the Github repository for specific details for the previous releases. Thank you!
53 changes: 13 additions & 40 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,19 +64,17 @@ Include the following asana package version in your `packages.yml` file.
```yaml
packages:
- package: fivetran/asana
version: [">=1.3.0", "<1.4.0"] # we recommend using ranges to capture non-breaking changes automatically
version: [">=1.4.0", "<1.5.0"] # we recommend using ranges to capture non-breaking changes automatically
```
> All required sources and staging models are now bundled into this transformation package. Do not include `fivetran/asana_source` in your `packages.yml` since this package has been deprecated.

### Define database and schema variables

#### Option A: Single connection
By default, this package runs using your [destination](https://docs.getdbt.com/docs/running-a-dbt-project/using-the-command-line-interface/configure-your-profile) and the `asana` schema. If this is not where your Asana data is (for example, if your Asana schema is named `asana_fivetran`), add the following configuration to your root `dbt_project.yml` file:
By default, this package runs using your destination and the `asana` schema. If this is not where your Asana data is (for example, if your Asana schema is named `asana_fivetran`), add the following configuration to your root `dbt_project.yml` file:

```yml
vars:
asana:
asana_database: your_database_name
asana_database: your_destination_name
asana_schema: your_schema_name
```

Expand All @@ -100,42 +98,9 @@ vars:
name: connection_2_source_name
```

##### Recommended: Incorporate unioned sources into DAG
> *If you are running the package through [Fivetran Transformations for dbt Core™](https://fivetran.com/docs/transformations/dbt#transformationsfordbtcore), the below step is necessary in order to synchronize model runs with your Asana connections. Alternatively, you may choose to run the package through Fivetran [Quickstart](https://fivetran.com/docs/transformations/quickstart), which would create separate sets of models for each Asana source rather than one set of unioned models.*

By default, this package defines one single-connection source, called `asana`, which will be disabled if you are unioning multiple connections. This means that your DAG will not include your Asana sources, though the package will run successfully.
#### Optional: Incorporate unioned sources into DAG

To properly incorporate all of your Asana connections into your project's DAG:
1. Define each of your sources in a `.yml` file in the `models` directory of your project. Utilize the following template for the `source`-level configurations, and, **most importantly**, copy and paste the table and column-level definitions from the package's `src_asana.yml` [file](https://github.com/fivetran/dbt_asana/blob/main/models/staging/src_asana.yml).

```yml
# a .yml file in your root project

version: 2

sources:
- name: <name> # ex: Should match name in asana_sources
schema: <schema_name>
database: <database_name>
loader: fivetran
config:
loaded_at_field: _fivetran_synced
freshness: # feel free to adjust to your liking
warn_after: {count: 72, period: hour}
error_after: {count: 168, period: hour}

tables: # copy and paste from asana/models/staging/src_asana.yml - see https://support.atlassian.com/bitbucket-cloud/docs/yaml-anchors/ for how to use anchors to only do so once
```

> **Note**: If there are source tables you do not have (see [Enabling/Disabling Models](https://github.com/fivetran/dbt_asana?tab=readme-ov-file#enablingdisabling-models)), you may still include them, as long as you have set the right variables to `False`.

2. Set the `has_defined_sources` variable (scoped to the `asana` package) to `True`, like such:
```yml
# dbt_project.yml
vars:
asana:
has_defined_sources: true
```
If you use [Fivetran Transformations for dbt Core™](https://fivetran.com/docs/transformations/dbt#transformationsfordbtcore) and are unioning multiple Asana connections, you can define your sources in a property `.yml` file, [using this as a template](https://github.com/fivetran/dbt_asana/blob/main/models/staging/src_asana.yml). Set the variable `has_defined_sources: true` under the Asana namespace in your `dbt_project.yml`. Otherwise, your Asana connections won't appear in your DAG. See the `union_connections` macro [documentation](https://github.com/fivetran/dbt_fivetran_utils/tree/releases/v0.4.latest#optional-union-connections-defined-sources-configuration) for full configuration details.

### Enabling/Disabling Models

Expand Down Expand Up @@ -182,6 +147,14 @@ vars:
asana_<default_source_table_name>_identifier: your_table_name
```

#### Source casing for case-sensitive destinations
By default, the package applies case-insensitive comparisons when resolving `source_relation` values. If your destination is case-sensitive and you want downstream transformations to respect the exact casing of your source database and schema names, set the following variable:

```yml
vars:
fivetran_using_source_casing: true
```

</details>

### (Optional) Orchestrate your models with Fivetran Transformations for dbt Core™
Expand Down
2 changes: 1 addition & 1 deletion dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
config-version: 2
name: 'asana'
version: '1.3.0'
version: '1.4.0'
require-dbt-version: [">=1.3.0", "<3.0.0"]
models:
asana:
Expand Down
2 changes: 1 addition & 1 deletion docs/catalog.json

Large diffs are not rendered by default.

253 changes: 3 additions & 250 deletions docs/index.html

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/manifest.json

Large diffs are not rendered by default.

6 changes: 6 additions & 0 deletions integration_tests/ci/test_scenarios.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,9 @@ test_scenarios:
asana_tag_identifier: missing
asana_task_tag_identifier: missing
include_incremental: false

- name: "MDLS"
vars:
fivetran_using_source_casing: true
include_incremental: false
include_dbt_seed: true
4 changes: 2 additions & 2 deletions integration_tests/dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
config-version: 2
name: 'asana_integration_tests'
version: '1.3.0'
version: '1.4.0'
profile: 'integration_tests'
vars:
asana_schema: asana_integrations_tests_03
Expand All @@ -26,7 +26,7 @@ models:
seeds:
+docs:
show: False
+quote_columns: "{{ true if target.type == 'redshift' else false }}"
+quote_columns: "{{ target.type == 'redshift' or var('fivetran_using_source_casing', false) }}"
asana_integration_tests:
+column_types:
_fivetran_synced: timestamp
Expand Down
15 changes: 0 additions & 15 deletions macros/union/apply_source_relation.sql

This file was deleted.

15 changes: 0 additions & 15 deletions macros/union/partition_by_source_relation.sql

This file was deleted.

92 changes: 0 additions & 92 deletions macros/union/union_connections.sql

This file was deleted.

Loading
Loading