new guardian subscription model #3381

johnduffell · 2026-02-03T16:32:20Z

precursor to #3323

Background

It's common in our lambdas to need to know what kind of subscription we are working on. Conventionally we loop through the ratePlans and charges array, looking for the product*id values that match what we are looking for (that are read or hard coded from the catalog).
We then need to check the lastChangeType and effective dates accordingly to make sure we're not looking at something that's a historic rate plan.

This works reasonably well for identifying a couple of products, but it leads to repetition between lambdas and also inconsistency.

At the moment we are looking to expand product switching across multiple products, and soon we will be trying to fetch all products for MMA account overview via the lambdas, meaning we need to be able to deserialise all types of subscription consistently.

The changes overview

This PR introduces an augmented "guardian subscription" model on top of the basic ZuoraSubscription, to reduce the amount of searching through zuora subscriptions we have to do in various lambdas.

At the moment it stays close to the ZuoraSubscription model, and augments it, rather than changing to more guardian and product specific terminology. That means we stick with contractEffectiveDate and effectiveStartDate rather than trying to use trialEndDate and firstDeliveryDate etc. I think it's good to keep it relatively consistent but there's no reason it can't diverge in future either generally or on a per product basis.

See the readme in the package: https://github.com/guardian/support-service-lambdas/blob/jd/gu-subscription-model/handlers/product-switch-api/src/guardianSubscription/README.md

How it works

First it processes the Zuora catalog:

group products by product id
group products by product rate plan id
group the charges in each product by product rate plan charge id

This makes it ready to join with a subscription.

Next it processes a standard ZuoraSubscription from the getSubscription call

Group similar to above for ease of joining
i. rateplans by product id and product rate plan id (nested)
i. group the charges by product rate plan charge id
Attach to the catalog - for each rate plan in the subscription: if it's in the product-catalog...
i. determine the productKey and productRatePlanKey and add them to the rate plan
i. fetch the product and productRatePlan objects and add them to the rate plan
i. fetch each chargeKey, and replace ratePlanCharges array to be an object keyed off the chargeKey
i. the ratePlan goes under ratePlans property (replacing the original)
...if it's NOT in the product-catalog, e.g. Discounts
i. fetch the product and rateplan from the zuora catalog and add them to the rate plan
i. fetch each charge name from the zuora catalog and key the ratePlanCharges off that.
i. attach the catalog charge object to the associated charge.
i. move the ratePlan under productsNotInCatalog property (from ratePlans)
Filter out irrelevant rateplans/charges
i. go through the rate plans removing those that are lastChangeType=Remove
i. If the subscription is Active - go through the charges removing any that are after their effectiveEndDate
i. If the subscription is Cancelled - remove any that have an effectiveEndDate before the termEndDate (i.e. keep whatever that was active at the very end)
Turn the array of rateplans into a single one (throw error if there are multiple)

Now we have a "guardian" subscription

Example "guardian" subscription

{
  id: "71a16bd99529b12a804b21f9f399554d",
  accountNumber: "A01075962",
  subscriptionNumber: "A-S01234567",
  status: "Active",
  contractEffectiveDate: "2025-12-15T00:00:00.000Z",
  serviceActivationDate: "2025-12-15T00:00:00.000Z",
  customerAcceptanceDate: "2025-12-15T00:00:00.000Z",
  subscriptionStartDate: "2025-12-15T00:00:00.000Z",
  subscriptionEndDate: "2026-12-15T00:00:00.000Z",
  lastBookingDate: "2025-12-15T00:00:00.000Z",
  termStartDate: "2025-12-15T00:00:00.000Z",
  termEndDate: "2026-12-15T00:00:00.000Z",
  ratePlan: {
    id: "71a16bd99529b12a804b21f9f3fc5555",
    productId: "8ad09fc281de1ce70181de3b23b2363d",
    productName: "Supporter Plus",
    productRatePlanId: "8ad08cbd8586721c01858804e3275376",
    ratePlanName: "Supporter Plus V2 - Monthly",
    productKey: "SupporterPlus",                  ****** this is the key from product-catalog
    product: {                                    ****** this is the information from product-catalog
      active: true,
      billingSystem: "zuora",
      customerFacingName: "All-access digital",   ****** this is probably the most useful bit
      isDeliveryProduct: false
    },
    productRatePlanKey: "Monthly",                ****** this is the second level key from product-catalog
    productRatePlan: {                            ****** this is the information from product-catalog
      billingPeriod: "Month",
      id: "8ad08cbd8586721c01858804e3275376",
      pricing: {
        AUD: 20,
        CAD: 15,
        EUR: 12,
        GBP: 12,
        NZD: 20,
        USD: 15
      },
      termLengthInMonths: 12,
      termType: "Recurring"
    },
    ratePlanCharges: {
      Subscription: {                             ****** the charges are keyed off their names in the product-catalog
        id: "71a16bd99529b12a804b21f9f399554e",
        productRatePlanChargeId: "8ad08cbd8586721c01858804e3715378",
        number: "C-01847004",
        name: "Subscription",
        type: "Recurring",
        model: "FlatFee",
        currency: "EUR",
        effectiveStartDate: "2025-12-15T00:00:00.000Z",
        effectiveEndDate: "2026-12-15T00:00:00.000Z",
        billingPeriod: "Month",
        processedThroughDate: "2026-01-15T00:00:00.000Z",
        chargedThroughDate: "2026-02-15T00:00:00.000Z",
        upToPeriodsType: null,
        upToPeriods: null,
        price: 12,
        discountPercentage: null,
        billingPeriodAlignment: "AlignToCharge"
      },
      Contribution: {
        id: "71a16bd99529b12a804b21f9f399554f",
        productRatePlanChargeId: "8ad09ea0858682bb0185880ac57f4c4c",
        number: "C-01847005",
        name: "Contribution",
        type: "Recurring",
        model: "FlatFee",
        currency: "EUR",
        effectiveStartDate: "2025-12-15T00:00:00.000Z",
        effectiveEndDate: "2026-12-15T00:00:00.000Z",
        billingPeriod: "Month",
        processedThroughDate: "2026-01-15T00:00:00.000Z",
        chargedThroughDate: "2026-02-15T00:00:00.000Z",
        upToPeriodsType: null,
        upToPeriods: null,
        price: 0,
        discountPercentage: null,
        billingPeriodAlignment: "AlignToCharge"
      }
    }
  }
}

This will be used in the above PR to enable generic product switching.

Testing

I added some unit tests, but it's hard to think of all the different states of subscriptions that enter the system.

Since this has to be able to parse all subscriptions that call product-switch-api, I thought the best solution was to test it against them.

As such I implemented a novel form of testing where it searches the cloudwatch logs for zuora subscription responses, parses out the JSON body, and feeds it through the parsers, checking that it doesn't fail.

This did find a few edge cases such as backdated cancelled subscriptions, and subscriptions with a current introductory discount, directly followed by a cancellation save discount, which I have fixed.

Next steps

This is not yet in use - it will be in use for all product switches once the follow on PR is shipped.

We could in future add this as an wrapper similar to withMMAIdentityCheck[1], if we want to reduce clutter in the business logic of the lambdas. Then the handlers would get a valid guardian subscription object for free.
[1] https://github.com/search?q=repo%3Aguardian%2Fsupport-service-lambdas%20withMMAIdentityCheck&type=code

handlers/product-switch-api/runManual/realSubscriptions/downloadRealSubscriptions.ts

johnduffell · 2026-02-03T17:20:03Z

handlers/product-switch-api/runManual/realSubscriptions/config.ts

@@ -0,0 +1,7 @@
+/**


runManual/realSusbcriptions/ is the package of code to extract and redact PROD subscriptions from the logs, and let us test that they can all be parsed by the subscription parser.

johnduffell · 2026-02-03T17:22:12Z

handlers/product-switch-api/runManual/realSubscriptions/downloadRealSubscriptions.ts

@@ -0,0 +1,242 @@
+import { existsSync, mkdirSync, writeFileSync } from 'fs';


this script downloads and redacts the subscription data, it's converted from a (shorter) scala version by copilot with a bit of hand tidying up so it's a little bit rough.
I feel like it's less important because it's self contained manually run code, but I'm happy to tidy it up to production standard if necessary.

It takes less than a minute to run, mostly limited by making the AWS calls.

johnduffell · 2026-02-03T17:24:04Z

handlers/product-switch-api/runManual/realSubscriptions/testRealSubscriptions.test.ts

+			const zuoraSubscription = zuoraSubscriptionSchema.parse(subscriptionData);
+			const guardianSubscription =
+				guardianSubscriptionParser.toGuardianSubscription(zuoraSubscription);
+			const filter =
+				SubscriptionFilter.activeNonEndedSubscriptionFilter(callDate);
+			const filteredSubscription =
+				filter.filterSubscription(guardianSubscription);
+			const flattenedSubscription =
+				getSinglePlanFlattenedSubscriptionOrThrow(filteredSubscription);


at the moment it just checks they can be parsed, it doesn't check any of the values make sense, which is about all we can do given we don't know much about the underlying subscriptions.
I think this is a pretty good start though.

johnduffell · 2026-02-03T17:25:30Z

...ers/product-switch-api/src/guardianSubscription/getSinglePlanFlattenedSubscriptionOrThrow.ts

+		ratePlan,
+		discountRatePlans,
+	} satisfies GuardianSubscription;
+}


if it says the below file is a "Large diff" then open it up anyway, as it's an important file.

johnduffell · 2026-02-03T17:26:30Z

handlers/product-switch-api/src/productSwitchEmail.ts

-	const billingPeriodMonths: number = {
+	const billingPeriodMap: Record<string, number> = {
 		Month: 1,
-		Quarter: 3,


quarter was never supported, it was just for the type checker. Now OneTime also exists, so I'm breaking the link (this will be fixed in the next PR to use a better type)

johnduffell · 2026-02-03T17:28:25Z

...ers/product-switch-api/test/changePlan/guardianSubscription/guardianSubscriptionNarrowing.ts

@@ -0,0 +1,28 @@
+import type { GuardianRatePlan } from '../../../src/guardianSubscription/guardianSubscriptionParser';


this is not an actual test, it's just making sure that the type checker narrows the type if we check the productKey and/or productRatePlanKey so we can do things like ratePlan.ratePlanCharges.Contribution.price to get the contribution amount without finds or getIfDefineds

johnduffell · 2026-02-03T17:29:06Z

handlers/product-switch-api/test/frequencySwitch.test.ts

 	test('throws when subscription status is not Active', async () => {
 		const now = dayjs();
 		const subscription = makeSubscriptionWithSingleCharge('Month', 10);
-		subscription.status = 'Suspended';


suspended doesn't exist, I think this must have been meant to be cancelled

johnduffell · 2026-02-03T17:30:00Z

handlers/stripe-disputes/test/services/cancelSubscriptionService.test.ts

-	it('should skip cancellation for inactive subscription (Expired)', async () => {
-		const mockSubscription = createMockSubscription('Expired');
-
-		const result = await cancelSubscriptionService(
-			mockLogger,
-			mockZuoraClient,
-			mockSubscription,
-		);
-
-		expect(result).toBe(false);
-		expect(mockLogger.log).toHaveBeenCalledWith(
-			'Subscription already inactive (Expired), skipping cancellation',
-		);
-		expect(cancelSubscription).not.toHaveBeenCalled();
-	});
-
-	it('should skip cancellation for inactive subscription (Suspended)', async () => {
-		const mockSubscription = createMockSubscription('Suspended');
-
-		const result = await cancelSubscriptionService(
-			mockLogger,
-			mockZuoraClient,
-			mockSubscription,
-		);
-
-		expect(result).toBe(false);
-		expect(mockLogger.log).toHaveBeenCalledWith(
-			'Subscription already inactive (Suspended), skipping cancellation',
-		);
-		expect(cancelSubscription).not.toHaveBeenCalled();
-	});


expired exists but wouldn't come back for a "get subscription by sub name" call, and Suspended doesn't exist. There's a test above for Cancelled anyway.

johnduffell · 2026-02-03T17:31:06Z

modules/product-catalog/src/productCatalog.ts

@@ -107,17 +107,49 @@ export type Product<P extends ProductKey> = ProductCatalog[P];



the types in this file had to be slightly complicated to make narrowing work correctly.

I'm not totally sure of the ins and outs, but it does seem to work.

johnduffell · 2026-02-03T17:32:50Z

modules/product-catalog/src/productCatalog.ts

+	validateOrThrow<P extends ProductKey>(
+		targetGuardianProductName: P,
+		productRatePlanKey: string,
+	): GuardianCatalogKeys<P> {
+		const ratePlans = this.catalogData[targetGuardianProductName].ratePlans;
+		if (!this.hasRatePlan(productRatePlanKey, ratePlans)) {
+			throw new Error(
+				`Unsupported rate plan key: ${String(
+					productRatePlanKey,
+				)} for product ${targetGuardianProductName}`,
+			);
+		}


if you pass a product and rate plan key, it checks that they are valid in combination and return an object to hold both in a type safe way, this is needed so it can check that the requested product switch actually exists at run time (as we can't do it at compile time with implicits!)

johnduffell · 2026-02-03T17:34:07Z

modules/zuora/src/types/objects/account.ts

 export const metricsSchema = z.object({
 	totalInvoiceBalance: z.number(),
-	currency: z.string(),
+	currency: z.enum(CurrencyValues),


I checked bigquery and there only are the 6 well known currencies in zuora, so this should be safe

johnduffell · 2026-02-03T17:34:55Z

modules/zuora/src/types/objects/subscription.ts

 	accountNumber: z.string(),
 	subscriptionNumber: z.string(),
-	status: z.string(),
+	status: z.enum(['Active', 'Cancelled']),


this is slightly more risky as we have a lot of "expired" subscription versions, however if you fetch by sub name, you always get the latest i.e. non expired version.

handlers/product-switch-api/src/guardianSubscription/group/buildZuoraProductIdToKey.ts

handlers/product-switch-api/runManual/realSusbcriptions/README.md

...lers/product-switch-api/src/guardianSubscription/group/groupSubscriptionByZuoraCatalogIds.ts

rupertbates · 2026-02-04T11:14:04Z

...lers/product-switch-api/src/guardianSubscription/group/groupSubscriptionByZuoraCatalogIds.ts

+		ratePlans: ZuoraSubscription['ratePlans'],
+	): IndexedZuoraSubscriptionRatePlansByProduct {
+		const doubleGroupedRatePlans: IndexedZuoraSubscriptionRatePlansByProduct =
+			mapValues(this.byProductAndRatePlan(ratePlans), (x) =>


Is it possible to make this more readable? Maybe use meaningful variable names? My brains shuts down as soon as I look at it!

I've pushed an updated version, actually by grouping charges first, then grouping by rateplan/product ids, it saves us having to dig down to do the charges afterwards.

...lers/product-switch-api/src/guardianSubscription/group/groupSubscriptionByZuoraCatalogIds.ts

# Conflicts: # modules/arrayFunctions.ts # modules/objectFunctions.ts

handlers/product-switch-api/src/guardianSubscription/subscriptionFilter.ts

handlers/product-switch-api/src/productSwitchEmail.ts

...rs/product-switch-api/src/guardianSubscription/reprocessRatePlans/guardianRatePlanBuilder.ts

rupertbates · 2026-02-09T13:59:58Z

modules/product-catalog/src/productCatalog.ts

 		const ratePlans: ProductCatalog[P]['ratePlans'] =
 			this.catalogData[productKey].ratePlans;
-		return ratePlans[productRatePlanKey];
+		return ratePlans[productRatePlanKey] as ProductRatePlan<P, PRP>;


This used to typecheck but now we're having to say as ProductRatePlan why is that?

Ah yes I didn't like that but I couldn't find a way around it.

it's because I had to change the type of ProductRatePlan from what you'd expect ProductCatalog[P]['ratePlans'][PRP] to a conditional type that proves that PRP is a valid child of P:

P extends ProductKey ? PRP extends keyof ProductCatalog[P]['ratePlans'] ? ProductCatalog[P]['ratePlans'][PRP] : never : never

If I don't do that, then if you have a ProductRatePlan without an exact type, it just has no accessible properties and collapses to never. So you can't get at the charges etc. without casting at that point. Hence why I did it as early as possible.

This (by my understanding) is if P is Contribution|SubscriptionCard and PRP is Annual|Weekend, it finds that it could be an invalid combination and collapses it to never. Doing that funny trick lets it remove all the invalid combinations from the union before it gets to that point. However it can't prove that the two types are compatible.

I think typescript basically falls down as it turns everything into unions that don't have any context, then gets in a mess.

I think in the old memsub we used implicits for keeping the context, I did wonder if we can wrap the rate plan key inside the product key somehow to replicate that, but I couldn't think how it would work.

Yes I got to pretty much the same conclusion after chatting to copilot for a bit 👍

modules/arrayFunctions.ts

modules/nullAndUndefined.ts

modules/mapFunctions.ts

modules/objectFunctions.ts

handlers/product-switch-api/src/guardianSubscription/subscriptionFilter.ts

...lers/product-switch-api/src/guardianSubscription/group/groupSubscriptionByZuoraCatalogIds.ts

modules/objectFunctions.ts

…te plan clearer

johnduffell added the feature Departmental tracking: work on a new feature label Feb 3, 2026

github-advanced-security bot found potential problems Feb 3, 2026

View reviewed changes

handlers/product-switch-api/runManual/realSubscriptions/downloadRealSubscriptions.ts Fixed Show fixed Hide fixed

new guardian subscription model

54954c5

johnduffell force-pushed the jd/gu-subscription-model branch from 0dba696 to 54954c5 Compare February 3, 2026 17:18

johnduffell commented Feb 3, 2026

View reviewed changes

johnduffell marked this pull request as ready for review February 3, 2026 17:36

add readme for the guardian subscription model

9e2a549

johnduffell commented Feb 3, 2026

View reviewed changes

handlers/product-switch-api/src/guardianSubscription/group/buildZuoraProductIdToKey.ts Outdated Show resolved Hide resolved

break down guardian subscription builder into smaller files

1526617

johnduffell mentioned this pull request Feb 4, 2026

Make T1 to T2 switch more generic and add T2 to T3 switch for internal users #3323

Draft

johnduffell added 2 commits February 4, 2026 08:41

readme tidyup

9d085e2

better json/js highlighting in readme

faa4ef4

rupertbates reviewed Feb 4, 2026

View reviewed changes

handlers/product-switch-api/runManual/realSusbcriptions/README.md Outdated Show resolved Hide resolved

rupertbates reviewed Feb 4, 2026

View reviewed changes

...lers/product-switch-api/src/guardianSubscription/group/groupSubscriptionByZuoraCatalogIds.ts Outdated Show resolved Hide resolved

rupertbates reviewed Feb 4, 2026

View reviewed changes

...lers/product-switch-api/src/guardianSubscription/group/groupSubscriptionByZuoraCatalogIds.ts Outdated Show resolved Hide resolved

rupertbates reviewed Feb 4, 2026

View reviewed changes

...lers/product-switch-api/src/guardianSubscription/group/groupSubscriptionByZuoraCatalogIds.ts Outdated Show resolved Hide resolved

rupertbates reviewed Feb 4, 2026

View reviewed changes

...lers/product-switch-api/src/guardianSubscription/group/groupSubscriptionByZuoraCatalogIds.ts Outdated Show resolved Hide resolved

Merge branch 'main' into jd/gu-subscription-model

3c0dbaa

# Conflicts: # modules/arrayFunctions.ts # modules/objectFunctions.ts

rupertbates reviewed Feb 6, 2026

View reviewed changes

handlers/product-switch-api/src/guardianSubscription/subscriptionFilter.ts Outdated Show resolved Hide resolved

rupertbates reviewed Feb 6, 2026

View reviewed changes

handlers/product-switch-api/src/productSwitchEmail.ts Outdated Show resolved Hide resolved

johnduffell added 5 commits February 6, 2026 22:30

improve type safety of product*ids and tidy up types

8894494

simplify rate plan builders

03d68dc

parse zuora catalog correctly in tests

eb63237

pr feedback mainly to tidy up subscription filter

3c10dbe

Merge branch 'main' into jd/gu-subscription-model

6d92101

johnduffell mentioned this pull request Feb 9, 2026

all rest siblings to be unused in eslint #3393

Merged

make more tests type check correctly

4b857aa

johnduffell force-pushed the jd/gu-subscription-model branch from 0ac651a to 4b857aa Compare February 9, 2026 13:11

johnduffell added 2 commits February 9, 2026 13:12

Merge branch 'main' into jd/gu-subscription-model

8701f29

remove unneeded lint exceptions

14004bc

johnduffell force-pushed the jd/gu-subscription-model branch from ea81c8e to 14004bc Compare February 9, 2026 13:15

fix test that used badly formed product ids

412d539