Feature/stripe mrr arr models

.quickstart/quickstart.yml

@@ -33,5 +33,6 @@ public_models: [
   "stripe__balance_change_from_activity_itemized_3",
   "stripe__ending_balance_reconciliation_itemized_4",
   "stripe__payout_itemized_3",
-  "stripe__line_item_enhanced"
+  "stripe__line_item_enhanced",
+  "stripe__subscription_item_mrr_report"
 ]

CHANGELOG.md

@@ -1,3 +1,36 @@
+# dbt_stripe v1.3.0-a2
+[PR #138](https://github.com/fivetran/dbt_stripe/pull/138) includes the following updates:
+
+## Under the Hood
+- Adds `stripe__subscription_item_mrr_report` model to quickstart.yml public models list.
+
+# dbt_stripe v1.3.0-a1
+[PR #138](https://github.com/fivetran/dbt_stripe/pull/138) includes the following updates:
+
+## Schema/Data Change
+**1 total change • 0 possible breaking changes**
+
+| Data Model(s) | Change type | Old | New | Notes |
+| ---------- | ----------- | -------- | -------- | ----- |
+| [`stripe__subscription_item_mrr_report`](https://fivetran.github.io/dbt_stripe/#!/model/model.stripe.stripe__subscription_item_mrr_report) | New End Model | | | Each record represents a subscription item for a given month with MRR metrics and movement classification. Tracks MRR changes over time, classifying each month as new, expansion, contraction, churned, reactivation, or unchanged. |
+
+## Feature Update
+- Adds new analyses folder with compiled SQL for advanced revenue reporting:
+  - `stripe__arr_snapshot_analysis`: Generates a high-level ARR snapshot report for the entire business for revenue forecasting.
+  - `stripe__customer_mrr_analysis`: Generates an MRR report at the customer level for retention reporting and cohort analysis.
+  - These analysis files reference the `stripe__subscription_item_mrr_report` model and can be compiled using `dbt compile` and executed directly in your data warehouse.
+
+## Bug Fix
+- Fixes a circular reference in `stg_stripe__price_plan` where the model incorrectly references itself instead of `stg_stripe__price_plan_tmp`, causing compilation errors.
+
+## Documentation
+- Adds comprehensive column documentation for `stripe__subscription_item_mrr_report` in `stripe.yml`.
+- Adds README in the analysis folder with instructions on how to compile and use the analysis SQL.
+
+## Under the Hood
+- Adds consistency test for `stripe__subscription_item_mrr_report` model.
+- Updates `integration_tests/seeds/price_data.csv` with additional test data.
+
 # dbt_stripe v1.2.0
 
 [PR #137](https://github.com/fivetran/dbt_stripe/pull/137) includes the following updates:

README.md

@@ -33,6 +33,7 @@ The following table provides a detailed list of all tables materialized within t
 | [`stripe__invoice_line_item_details`](https://fivetran.github.io/dbt_stripe/#!/model/model.stripe.stripe__invoice_line_item_details) | Includes line items with charge, customer, subscription, and pricing details.<br><br>**Example Analytics Questions:**<br>• Which products or services contribute most to total invoiced revenue?<br>• Are there any products consistently discounted or refunded? |
 | [`stripe__daily_overview`](https://fivetran.github.io/dbt_stripe/#!/model/model.stripe.stripe__daily_overview) | Summarizes daily and rolling Stripe transaction totals by type.<br><br>**Example Analytics Questions:**<br>• What is the trend in daily net payments and refunds?<br>• What is the MRR trend over the last 6 months? |
 | [`stripe__subscription_details`](https://fivetran.github.io/dbt_stripe/#!/model/model.stripe.stripe__subscription_details) | Contains subscription records with customer and payment metrics.<br><br>**Example Analytics Questions:**<br>• How many active subscriptions are there by plan or product?<br>• What is the average customer subscription length before cancellation? |
+| [`stripe__subscription_item_mrr_report`](https://fivetran.github.io/dbt_stripe/#!/model/model.stripe.stripe__subscription_item_mrr_report) | Tracks monthly recurring revenue (MRR) at the subscription item level with movement classification.<br><br>**Example Analytics Questions:**<br>• What is the total MRR and how is it trending month over month?<br>• What portion of MRR is from expansion vs new customers vs churn? |
 | [`stripe__customer_overview`](https://fivetran.github.io/dbt_stripe/#!/model/model.stripe.stripe__customer_overview) | Shows customer-level metrics with transaction details and associations.<br><br>**Example Analytics Questions:**<br>• Who are the top 10 customers by total lifetime value?<br>• How many customers made a payment in the last 90 days? |
 | [`stripe__activity_itemized_2`](https://fivetran.github.io/dbt_stripe/#!/model/model.stripe.stripe__activity_itemized_2) | Lists balance transactions with invoice, fee, refund, and customer data.<br><br>**Example Analytics Questions:**<br>• What are the exact transaction-level fees for each invoice or customer?<br>• How much are we paying in interchange and platform fees per transaction? |
 | [`stripe__balance_change_from_activity_itemized_3`](https://fivetran.github.io/dbt_stripe/#!/model/model.stripe.stripe__balance_change_from_activity_itemized_3) | Reconciles Stripe balance changes like a detailed bank statement.<br><br>**Example Analytics Questions:**<br>• What was the source of each Stripe balance change over the last month?<br>• How accurate is my accounting ledger compared to Stripe's balance records? |
@@ -76,7 +77,7 @@ Include the following stripe package version in your `packages.yml` file:
 ```yaml
 packages:
   - package: fivetran/stripe
-    version: [">=1.2.0", "<1.3.0"]
+    version: "1.3.0-a2"
 ```
 > All required sources and staging models are now bundled into this transformation package. Do not include `fivetran/stripe_source` in your `packages.yml` since this package has been deprecated.
 

analyses/README.md

@@ -0,0 +1,23 @@
+# Stripe Analysis
+> Note: The compiled sql within the analysis folder references the final model [stripe__subscription_item_mrr_report](https://github.com/fivetran/dbt_stripe/blob/master/models/stripe_financial_reports/stripe__subscription_item_mrr_report.sql). As such, prior to compiling the provided sql to roll MRR up to the customer level or analyze ARR metrics, you must first execute `dbt run`.
+
+
+## Analysis SQL
+| **sql**                | **description**                                                                                                                                |
+| ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- |
+| [stripe__arr_snapshot_analysis](https://github.com/fivetran/dbt_stripe/blob/master/analysis/stripe__arr_snapshot_analysis.sql) | The output of the compiled sql will generate a high-level ARR snapshot report for the entire business that can be used for revenue forecasting. The analysis provides year-end ARR metrics by aggregating MRR data from the last month of each calendar year and multiplying by 12. The SQL references the `stripe__subscription_item_mrr_report` model and aggregates metrics by currency. Subscriptions must be enabled for this report to compile correctly. |
+| [stripe__customer_mrr_analysis](https://github.com/fivetran/dbt_stripe/blob/master/analysis/stripe__customer_mrr_analysis.sql) | The output of the compiled sql will generate an MRR report at the customer level of granularity that can be used for retention reporting and cohort analysis. The analysis aggregates monthly recurring revenue by customer and month, providing insights into customer-level revenue trends over time. The SQL references the `stripe__subscription_item_mrr_report` model. This report can also be generated at the overall business level by removing the customer_id field from the aggregation. |
+
+## SQL Compile Instructions
+Leveraging the above sql is made possible by the [analysis functionality of dbt](https://docs.getdbt.com/docs/building-a-dbt-project/analyses/). In order to
+compile the sql, you will perform the following steps:
+- Execute `dbt run` to create the package models.
+- Execute `dbt compile` to generate the target specific sql.
+- Navigate to your project's `/target/compiled/stripe/analysis` directory.
+- Copy the desired analysis code (`stripe__arr_snapshot_analysis` or `stripe__customer_mrr_analysis`) and run in your data warehouse.
+- Confirm the revenue metrics match your expected subscription and billing patterns.
+- Analyze the ARR snapshots and customer-level MRR trends to identify growth opportunities, retention patterns, and revenue forecasting insights.
+
+## Contributions
+Don't see a compiled sql statement you would have liked to be included? Notice any bugs when compiling
+and running the analysis sql? If so, we highly encourage and welcome contributions to this package! If interested, the best first step is [opening a feature request](https://github.com/fivetran/dbt_stripe/issues/new?template=feature-request.yml).

analyses/stripe__arr_snapshot_analysis.sql

@@ -0,0 +1,49 @@
+{{ config(enabled=var('stripe__using_subscriptions', True)) }}
+
+with mrr_by_item as (
+    select *
+    from {{ ref('stripe__subscription_item_mrr_report') }}
+
+),
+
+monthly_rollup as (
+    select
+        source_relation,
+        subscription_month as recurring_rev_month,
+        subscription_year as recurring_rev_year,
+        currency,
+        sum(current_month_mrr) as total_mrr
+    from mrr_by_item
+     {{ dbt_utils.group_by(4) }}
+
+),
+
+snapshots as (
+    select
+        source_relation,
+        recurring_rev_month,
+        recurring_rev_year,
+        currency,
+        total_mrr,
+        total_mrr * 12 as total_arr,
+        row_number() over (
+            partition by source_relation, recurring_rev_year, currency
+            order by recurring_rev_month desc
+        ) as month_number
+    from monthly_rollup
+
+),
+
+final as (
+    select
+        source_relation,
+        recurring_rev_year,
+        currency,
+        round(total_arr, 2) as total_arr
+    from snapshots
+    where month_number = 1       -- last month in that year
+
+)
+
+select *
+from final

analyses/stripe__customer_mrr_analysis.sql

@@ -0,0 +1,37 @@
+{{ config(enabled=var('stripe__using_subscriptions', True)) }}
+
+with item_mrr as (
+    select *
+    from {{ ref('stripe__subscription_item_mrr_report') }}
+),
+
+customer_mrr as (
+    select
+        source_relation,
+        customer_id,
+        subscription_year,
+        subscription_month,
+        currency,
+        sum(current_month_mrr)  as current_month_mrr,
+        sum(previous_month_mrr) as previous_month_mrr
+    from item_mrr
+    group by
+      1,2,3,4,5
+)
+
+select
+    *,
+    case
+        when previous_month_mrr is null 
+             and current_month_mrr > 0 then 'new'
+        when current_month_mrr > previous_month_mrr then 'expansion'
+        when previous_month_mrr > current_month_mrr
+             and current_month_mrr > 0 then 'contraction'
+        when (current_month_mrr = 0 or current_month_mrr is null)
+             and previous_month_mrr > 0 then 'churned'
+        when previous_month_mrr = 0
+             and current_month_mrr > 0 then 'reactivation'
+        when current_month_mrr = previous_month_mrr then 'unchanged'
+        else 'unknown'
+    end as customer_mrr_type
+from customer_mrr

dbt_project.yml

@@ -1,6 +1,6 @@
 config-version: 2
 name: 'stripe'
-version: '1.2.0'
+version: '1.3.0'
 require-dbt-version: [">=1.3.0", "<3.0.0"]
 
 models:

integration_tests/dbt_project.yml

@@ -1,7 +1,7 @@
 config-version: 2
 
 name: 'stripe_integration_tests'
-version: '1.2.0'
+version: '1.3.0'
 
 profile: 'integration_tests'
 
@@ -69,6 +69,11 @@ seeds:
       +column_types:
         destination_bank_account_id: "{{ 'varchar(500)' if target.type in ('redshift','postgres') else 'string'}}"
         destination_card_id: "{{ 'varchar(500)' if target.type in ('redshift','postgres') else 'string'}}"
+    price_data:
+      +column_types:
+        recurring_interval: "{{ 'varchar(100)' if target.type in ('redshift','postgres') else 'string'}}"
+        recurring_aggregate_usage: "{{ 'varchar(100)' if target.type in ('redshift','postgres') else 'string'}}"
+        recurring_usage_type: "{{ 'varchar(100)' if target.type in ('redshift','postgres') else 'string'}}"
     discount_data:
       +enabled: "{{ true if target.type not in ('snowflake', 'postgres') else false }}"
     discount_data_snowflake:

integration_tests/seeds/price_data.csv

@@ -1,2 +1,5 @@
 id,_fivetran_synced,active,billing_scheme,created,currency,invoice_item_id,is_deleted,livemode,lookup_key,metadata,nickname,product_id,recurring_aggregate_usage,recurring_interval,recurring_interval_count,recurring_usage_type,tiers_mode,transform_quantity_divide_by,transform_quantity_round,type,unit_amount,unit_amount_decimal
-price_1,2022-09-22 23:49:02,TRUE,per_unit,2020-05-02 19:21:44,usd,ii_FYFJSuzUa5YRk,FALSE,TRUE,,{},,prod_HCqb8lm6kfYk,,,,,,,,one_time,100000,100000
+price_1,2022-09-22 23:49:02,TRUE,per_unit,2020-05-02 19:21:44,usd,ii_FYFJSuzUa5YRk,FALSE,TRUE,,{},,prod_HCqb8lm6kfYk,,,,,,,,one_time,100000,100000
+price_2,2024-01-15 10:30:00,TRUE,per_unit,2023-06-01 12:00:00,usd,,FALSE,TRUE,,{},Basic Plan,prod_001,,month,1,licensed,,,,recurring,2999,2999
+price_3,2024-01-15 10:30:00,TRUE,per_unit,2023-07-15 14:30:00,usd,,FALSE,TRUE,,{},Annual Plan,prod_002,,year,1,licensed,,,,recurring,29990,29990
+price_4,2024-01-15 10:30:00,TRUE,per_unit,2023-08-20 09:15:00,eur,,FALSE,TRUE,,{},Quarterly Plan,prod_003,,month,3,licensed,,,,recurring,8000,8000

integration_tests/tests/consistency/consistency_subscription_item_mrr_report.sql

@@ -0,0 +1,53 @@
+{{ config(
+    tags="fivetran_validations",
+    enabled=var('fivetran_validation_tests_enabled', false)
+) }}
+
+-- this test ensures the subscription_item_mrr_report end model matches the prior version
+-- aggregated on the subscription_month, subscription_year, and source_relation grain
+with
+{% for prod_or_dev in ('prod', 'dev') %}
+    {% set cols = adapter.get_columns_in_relation(ref('stripe__subscription_item_mrr_report')) %}
+    {{ prod_or_dev }} as (
+        select
+            subscription_month,
+            subscription_year,
+            source_relation
+            {% for col in cols if col.name not in ["subscription_item_id", "subscription_id", "customer_id", "product_id", "subscription_status", "currency", "subscription_month", "subscription_year", "source_relation", "item_month_number", "mrr_type"] %}
+                , floor(sum({{ col.name }})) as summed_{{ col.name }} -- floor and sum is to keep consistency between dev and prod aggs
+            {% endfor %}
+        from {{ target.schema }}_stripe_{{ prod_or_dev }}.stripe__subscription_item_mrr_report
+        group by 1,2,3
+    ),
+{% endfor %}
+
+prod_not_in_dev as (
+    -- rows from prod not found in dev
+    select * from prod
+    except distinct
+    select * from dev
+),
+
+dev_not_in_prod as (
+    -- rows from dev not found in prod
+    select * from dev
+    except distinct
+    select * from prod
+),
+
+final as (
+    select
+        *,
+        'from prod' as source
+    from prod_not_in_dev
+
+    union all -- union since we only care if rows are produced
+
+    select
+        *,
+        'from dev' as source
+    from dev_not_in_prod
+)
+
+select *
+from final

models/staging/stg_stripe__price_plan.sql

@@ -2,7 +2,7 @@
 
 with price_plan as (
 
-    select * 
+    select *
     from {{ ref('stg_stripe__price_plan_tmp') }}
 ),
 
@@ -26,12 +26,12 @@ fields as (
 
 final as (
 
-    select 
+    select
         id as price_plan_id,
         is_active,
         {{ stripe.convert_values('unit_amount') }},
         currency,
-        recurring_interval,
+        cast(recurring_interval as {{ dbt.type_string() }}) as recurring_interval,
         recurring_interval_count,
         recurring_usage_type,
         recurring_aggregate_usage,

models/stripe.yml

@@ -936,7 +936,7 @@ models:
       - name: payout_id
         description: The Stripe object to which this transaction is related.
       - name: effective_at
-        description: For automatic payouts, this is the date we expect funds to arrive in your bank account. For manual payouts, this is the date the payout was initiated. In both cases, it’s the date the paid-out funds are deducted from your Stripe balance. All dates in the requested timezone, or UTC if not provided.
+        description: For automatic payouts, this is the date we expect funds to arrive in your bank account. For manual payouts, this is the date the payout was initiated. In both cases, it's the date the paid-out funds are deducted from your Stripe balance. All dates in the requested timezone, or UTC if not provided.
       - name: currency
         description: Three-letter ISO code for the currency in which gross, fee and net are defined.
       - name: gross
@@ -956,12 +956,42 @@ models:
       - name: payout_status
         description: Current status of the payout (paid, pending, in_transit, canceled or failed). A payout will be pending until it is submitted to the bank, at which point it becomes in_transit. It will then change to paid if the transaction goes through. If it does not go through successfully, its status will change to failed or canceled.
       - name: payout_reversed_at
-        description: Typically this field will be empty. However, if the payout’s status is canceled or failed, this field will reflect the time at which it entered that status. Times in the requested timezone, or UTC if not provided.
+        description: Typically this field will be empty. However, if the payout's status is canceled or failed, this field will reflect the time at which it entered that status. Times in the requested timezone, or UTC if not provided.
       - name: payout_type
         description: Can be bank_account or card.
       - name: payout_description
         description: An arbitrary string attached to the payout. Often useful for displaying to users.
       - name: payout_destination_id
         description: ID of the bank account or card the payout was sent to.
       - name: source_relation
-        description: "{{ doc('source_relation') }}"
+        description: "{{ doc('source_relation') }}"
+
+  - name: stripe__subscription_item_mrr_report
+    description: Each record represents a subscription item for a given month with monthly recurring revenue (MRR) metrics and movement classification. Tracks MRR changes over time, classifying each month as new, expansion, contraction, churned, reactivation, or unchanged.
+    columns:
+      - name: source_relation
+        description: "{{ doc('source_relation') }}"
+      - name: subscription_item_id
+        description: Unique identifier for the subscription item.
+      - name: subscription_id
+        description: Unique identifier for the subscription.
+      - name: customer_id
+        description: ID of the customer who owns the subscription.
+      - name: product_id
+        description: ID of the product associated with this subscription item.
+      - name: subscription_status
+        description: Current status of the subscription. Possible values are incomplete, incomplete_expired, trialing, active, past_due, canceled, or unpaid.
+      - name: currency
+        description: Three-letter ISO currency code, in lowercase.
+      - name: subscription_year
+        description: Year of the subscription period.
+      - name: subscription_month
+        description: Month of the subscription period.
+      - name: current_month_mrr
+        description: Monthly recurring revenue for this subscription item in the current month. "{{ doc('convert_values') }}"
+      - name: previous_month_mrr
+        description: Monthly recurring revenue for this subscription item in the previous month. "{{ doc('convert_values') }}"
+      - name: item_month_number
+        description: Sequential month number for this subscription item, starting from 1 for the first month.
+      - name: mrr_type
+        description: Classification of the MRR change for this month. Possible values are new (first month with MRR), expansion (MRR increased), contraction (MRR decreased but still positive), churned (MRR went to zero), reactivation (MRR returned after being zero), unchanged (MRR stayed the same), or unknown.