bugfix/restore_stripe_mrr_history

CHANGELOG.md

@@ -1,3 +1,14 @@
+# dbt_stripe v1.9.0
+
+[PR #153](https://github.com/fivetran/dbt_stripe/pull/153) includes the following updates:
+
+## Schema/Data Change
+**1 total change • 1 possible breaking change**
+
+| Data Model(s) | Change type | Old | New | Notes |
+| ------------- | ----------- | --- | --- | ----- |
+| `stripe__subscription_item_mrr_report` (possible breaking change) | Changed model | Date spine starts at `current_period_start`, every month uses the current quantity, and the model runs with only `stripe__using_subscriptions`. | Date spine anchors to each item's `created_at`; historical months use the invoiced quantity (from `invoice_line_item`), the current month the live quantity. Now also requires `stripe__using_invoices`. | See the [DECISIONLOG](https://github.com/fivetran/dbt_stripe/blob/main/DECISIONLOG.md#mrr-history-anchored-to-each-subscription-items-creation-date). |
+
 # dbt_stripe v1.8.0
 
 [PR #152](https://github.com/fivetran/dbt_stripe/pull/152) includes the following updates:

DECISIONLOG.md

@@ -1,3 +1,25 @@
+## MRR History Anchored to Each Subscription Item's Creation Date
+
+The `stripe__subscription_item_mrr_report` model anchors each item's MRR timeline to that item's `created_at`, reporting MRR from when the item was added through the current month. It previously anchored to `current_period_start`, which Stripe overwrites every billing cycle, so all months before the current period were dropped.
+
+Because `subscription_item` retains only the current state of each item, we reconstruct historical quantity from `invoice_line_item`, which records the quantity actually billed for each period. For each month, we use the quantity from the most recent invoice line whose period overlaps that month (proration adjustment lines excluded); the current month uses the live `subscription_item` quantity. This lets the report surface real `expansion` and `contraction` between months rather than back-projecting today's quantity across all history.
+
+Because this model depends on `invoice_line_item` for historical quantity, it requires `stripe__using_invoices` (enabled by default) and does not build when that variable is disabled. Stripe generates an invoice each billing period, so gaps are not expected, but when a month has no invoice line we carry the most recent prior invoiced quantity forward rather than snapping to today's quantity (which would invent phantom changes in the gap). A month falls back to the current `subscription_item` quantity only when no earlier invoice exists (months before the first invoice). Prices are not back-filled — historical months apply the item's current price.
+
+## MRR Timeline Uses an End-of-Month Snapshot
+
+The `stripe__subscription_item_mrr_report` model keeps a month for an item only when the subscription is still active at month-end. We enforce this by truncating `current_period_end` to its month in the timeline's upper bound, so a month is kept only while its first day is before the start of the `current_period_end` month.
+
+The effect is that a subscription paid through the middle of a month is not on the books at that month's end and drops out entirely. For example, a monthly subscription whose last paid period runs May 15 to June 15 keeps May (a full month) but drops June, because it has lapsed by June 30.
+
+## Stripe Subscription and Item Start-Date Fields
+
+We anchor to `subscription_item.created_at` because MRR is reported at the item grain. The relevant fields are not interchangeable:
+
+- **`subscription.start_date_at`** — Stripe's authoritative subscription start, backdate-aware, but at the subscription grain.
+- **`subscription.created_at`** — when the subscription object was created; usually equals `start_date`, but diverges on backdating or imports.
+- **`subscription_item.created_at`** — when the item was added. Stripe allows items to be added or removed mid-cycle (add-ons, seat changes, plan upgrades) and prorates by default, so this can fall later than the subscription's start. Anchoring per item attributes MRR only to the months the item existed.
+
 ## MRR Date Spine Capped at the Current Month
 
 The `stripe__subscription_item_mrr_report` model caps its date spine at the end of the current month, regardless of how far into the future a subscription's `current_period_end` extends.

README.md

@@ -45,7 +45,7 @@ By default, this package materializes the following final tables:
 | [stripe__ending_balance_reconciliation_itemized_4](https://fivetran.github.io/dbt_stripe/#!/model/model.stripe.stripe__ending_balance_reconciliation_itemized_4) | Matches bank payouts with unsettled Stripe transactions.<br><br>**Example Analytics Questions:**<br><ul><li>Which transactions remain unsettled as of the last payout?</li><li>Do all automatic payouts reconcile fully with balance changes?</li></ul> |
 | [stripe__payout_itemized_3](https://fivetran.github.io/dbt_stripe/#!/model/model.stripe.stripe__payout_itemized_3) | Details expected and actual payout amounts and statuses.<br><br>**Example Analytics Questions:**<br><ul><li>When should I expect my next payout, and for how much?</li><li>Are there any delayed or failed payouts that need follow-up?</li></ul> |
 | [stripe__line_item_enhanced](https://fivetran.github.io/dbt_stripe/#!/model/model.stripe.stripe__line_item_enhanced) | Provides unified reporting across billing platforms on product, customer, and revenue metrics. See the [Fivetran Billing Model Streamlit App](https://fivetran-billing-model.streamlit.app/) for more details.<br><br>**Example Analytics Questions:**<br><ul><li>What are the top revenue-generating products or SKUs?</li><li>What is the average revenue per user (ARPU) by subscription plan?</li></ul> |
-| [stripe__subscription_item_mrr_report](https://fivetran.github.io/dbt_stripe/#!/model/model.stripe.stripe__subscription_item_mrr_report) | Shows both contracted and billed MRR (monthly recurring revenue) with discounts applied at the subscription item level. Tracks MRR changes over time, classifying each month as new, expansion, contraction, churned, reactivation, or unchanged.<br><br>**Example Analytics Questions:**<br><ul><li>What percentage of subscription customers are churning each month as compared to new?</li><li>How much subscription revenue was lost last year due to discounts?</li></ul> |
+| [stripe__subscription_item_mrr_report](https://fivetran.github.io/dbt_stripe/#!/model/model.stripe.stripe__subscription_item_mrr_report) | Shows both contracted and billed MRR (monthly recurring revenue) with discounts applied at the subscription item level. Tracks MRR changes over time, classifying each month as new, expansion, contraction, churned, reactivation, or unchanged. Relies on `invoice_line_item` for historical quantity, so it requires `stripe__using_invoices` (enabled by default) and does not build when that variable is disabled.<br><br>**Example Analytics Questions:**<br><ul><li>What percentage of subscription customers are churning each month as compared to new?</li><li>How much subscription revenue was lost last year due to discounts?</li></ul> |
 
 ¹ Each Quickstart transformation job run materializes these models if all components of this data model are enabled. This count includes all staging, intermediate, and final models materialized as `view`, `table`, or `incremental`.
 
@@ -80,7 +80,7 @@ Include the following stripe package version in your `packages.yml` file:
 ```yaml
 packages:
   - package: fivetran/stripe
-    version: [">=1.8.0", "<1.9.0"]
+    version: [">=1.9.0", "<1.10.0"]
 ```
 > All required sources and staging models are now bundled into this transformation package. Do not include `fivetran/stripe_source` in your `packages.yml` since this package has been deprecated.
 

dbt_project.yml

@@ -1,7 +1,7 @@
 config-version: 2
 name: 'stripe'
 
-version: '1.8.0'
+version: '1.9.0'
 require-dbt-version: [">=1.3.0", "<3.0.0"]
 
 models:

integration_tests/dbt_project.yml

@@ -2,7 +2,7 @@ config-version: 2
 
 name: 'stripe_integration_tests'
 
-version: '1.8.0'
+version: '1.9.0'
 
 profile: 'integration_tests'
 

models/stripe__subscription_item_mrr_report.sql

@@ -1,4 +1,4 @@
-{{ config(enabled=var('stripe__using_subscriptions', True)) }}
+{{ config(enabled=var('stripe__using_subscriptions', True) and var('stripe__using_invoices', True)) }}
 
 {% if execute and flags.WHICH in ('run', 'build') %}
 
@@ -8,7 +8,7 @@
         cast(
           {{ dbt.date_trunc(
               'month',
-              "coalesce(subscription_item.current_period_start, subscription.current_period_start)" 
+              "coalesce(subscription_item.created_at, subscription.created_at)"
           ) }} as date
         )
       ),
@@ -65,6 +65,16 @@ price_plan as (
 
 ),
 
+-- Actual billed quantity per period; provides quantity history subscription_item lacks. Prorations excluded.
+invoice_line_item as (
+
+    select *
+    from {{ ref('stg_stripe__invoice_line_item') }}
+    where subscription_item_id is not null
+        and not coalesce(proration, false)
+
+),
+
 {% if var('stripe__using_coupons', True) and var('stripe__using_subscription_discounts', True) %}
 subscription_discount as (
 
@@ -95,6 +105,36 @@ date_dimensions as (
 
 ),
 
+-- Map each billed quantity to every month its period overlaps, keeping the latest line per month.
+invoiced_quantity_by_month as (
+
+    select
+        invoice_line_item.source_relation,
+        invoice_line_item.subscription_item_id,
+        date_dimensions.subscription_month,
+        invoice_line_item.quantity,
+        row_number() over (
+            partition by
+                invoice_line_item.source_relation,
+                invoice_line_item.subscription_item_id,
+                date_dimensions.subscription_month
+            order by invoice_line_item.period_start desc
+        ) as rn
+    from invoice_line_item
+    inner join date_dimensions
+        on date_dimensions.subscription_month >= cast({{ dbt.date_trunc('month', 'invoice_line_item.period_start') }} as date)
+        and date_dimensions.subscription_month < cast({{ dbt.date_trunc('month', 'invoice_line_item.period_end') }} as date)
+
+),
+
+invoiced_quantity_deduped as (
+
+    select source_relation, subscription_item_id, subscription_month, quantity
+    from invoiced_quantity_by_month
+    where rn = 1
+
+),
+
 base as (
 
     select
@@ -103,9 +143,10 @@ base as (
         subscription_item.subscription_id,
         subscription.customer_id,
         subscription.status as subscription_status,
+        coalesce(subscription_item.created_at, subscription.created_at) as item_created_at,
         coalesce(subscription_item.current_period_start, subscription.current_period_start) as current_period_start,
         coalesce(subscription_item.current_period_end, subscription.current_period_end) as current_period_end,
-        subscription_item.quantity,
+        subscription_item.quantity as current_quantity,
         price_plan.product_id,
         price_plan.price_plan_id,
         price_plan.recurring_interval,
@@ -129,8 +170,10 @@ normalized as (
         subscription_id,
         customer_id,
         subscription_status,
+        item_created_at,
         current_period_start,
         current_period_end,
+        current_quantity,
         product_id,
         price_plan_id,
         recurring_interval,
@@ -175,7 +218,7 @@ subscription_item_periods as (
         price_plan_id,
         subscription_status,
         currency,
-        min(cast({{ dbt.date_trunc('month', 'current_period_start') }} as date)) as first_active_month,
+        min(cast({{ dbt.date_trunc('month', 'item_created_at') }} as date)) as first_active_month,
         cast({{ dbt.dateadd('month', 3, 'max(cast(' ~ dbt.date_trunc('month', 'current_period_end') ~ ' as date))') }} as date) as last_month_to_track
     from normalized
     {{ dbt_utils.group_by(8) }}
@@ -203,6 +246,47 @@ all_item_months as (
 
 ),
 
+-- attach the invoiced quantity to each month, then carry the most recent invoiced
+-- quantity forward across any months missing an invoice (gap months)
+item_month_invoiced as (
+
+    select
+        all_item_months.*,
+        invoiced_quantity_deduped.quantity as invoiced_quantity
+    from all_item_months
+    left join invoiced_quantity_deduped
+        on all_item_months.source_relation = invoiced_quantity_deduped.source_relation
+        and all_item_months.subscription_item_id = invoiced_quantity_deduped.subscription_item_id
+        and all_item_months.subscription_month = invoiced_quantity_deduped.subscription_month
+
+),
+
+-- increment a group id each time an invoiced quantity appears; gap months inherit the prior group
+item_month_grouped as (
+
+    select
+        *,
+        sum(case when invoiced_quantity is not null then 1 else 0 end) over (
+            partition by source_relation, subscription_item_id
+            order by subscription_month
+            rows between unbounded preceding and current row
+        ) as invoiced_group
+    from item_month_invoiced
+
+),
+
+-- within each group the single invoiced value carries forward to the gap months
+item_month_carried as (
+
+    select
+        *,
+        max(invoiced_quantity) over (
+            partition by source_relation, subscription_item_id, invoiced_group
+        ) as carried_quantity
+    from item_month_grouped
+
+),
+
 -- Join back to normalized to determine if subscription was active in each month
 item_months as (
 
@@ -217,13 +301,22 @@ item_months as (
         all_item_months.currency,
         all_item_months.subscription_year,
         all_item_months.subscription_month,
-        coalesce(normalized.mrr, 0) as mrr
-    from all_item_months
+        {% set effective_quantity %}
+            case
+                when all_item_months.subscription_month = cast({{ last_month }} as date)
+                    then coalesce(normalized.current_quantity, 1)
+                else coalesce(all_item_months.carried_quantity, normalized.current_quantity, 1)
+            end
+        {% endset %}
+        -- current month uses live quantity, history uses the carried-forward invoiced quantity
+        coalesce(normalized.mrr, 0)
+            * {{ dbt_utils.safe_divide(effective_quantity, "coalesce(normalized.current_quantity, 1)") }} as mrr
+    from item_month_carried as all_item_months
     left join normalized
         on all_item_months.source_relation = normalized.source_relation
         and all_item_months.subscription_item_id = normalized.subscription_item_id
         and all_item_months.price_plan_id = normalized.price_plan_id
-        and all_item_months.subscription_month >= cast({{ dbt.date_trunc('month', 'normalized.current_period_start') }} as date)
+        and all_item_months.subscription_month >= cast({{ dbt.date_trunc('month', 'normalized.item_created_at') }} as date)
         and all_item_months.subscription_month < cast({{ dbt.date_trunc('month', 'normalized.current_period_end') }} as date)
 
 ),