Analytics 2.0

[jmduke]

TL;DR

The current analytics page is bad. Gonna make it better, then gonna add more data.

What's the problem?

I'm not gonna beat around the bush with faux-optimism here. The current analytics page is bad.

It's slow. It's visually cumbersome. It does not answer meaningful questions. It was built in a time where I didn't really have a good sense of what was important, and I've never really sat down and tried to reason about it from first principles.

A good analytics page should answer a whole host of questions, but at the very least the following:

1. How many subscribers did I gain in X period?
2. Which posts in particular are causing people to upscribe?
3. What countries are reading my posts?
4. Are people reading on mobile or desktop?
5. What’s my average open and clickthrough rate?

What's the solution?

A work-in-progress look at the new page:

I'm breaking this down into two milestones:

• First, ship a net-better version of the existing analytics page that surfaces existing data (subscriber growth, open and click through rate) in a more ergonomic and useful fashion.
• Next, augment that page with pageview data gathered via Fathom (for folks who are opted in to tracking, of course).

Open questions

• This page works best when people use Buttondown's hosted archive pages and subscription pages. How do I make sure it's still a useful experience for folks who are using Buttondown in 'headless' mode and have archives turned off?

Out of scope

• As of right now, filtering the analytics by non-datetime attributes (ie filtering by tags, metadata, etc.) This should be possible in the future, and the architectural changes to both the backend and the frontend will make this easier, but I don't think it's something people will care about.
• "Vanity" topline metrics, such as 'number of responses to surveys' or 'number of words written' and things of that nature. I'm trying pretty hard to be opinionated about what really matters.

[slorber]

Quick feedback as a newsletter operator of thisweekinreact.com based on a sponsorship model and spending money on paid acquisition

To me one of the most important is to be able to track reliably the source of acquisition of each subscriber so that I know where to double down my marketing effort.

It should also be possible to know the average subscriber quality on a per-acquisition source granularity.

I'm on Convertkit currently and they don't make it easy, but I managed to track things with custom fields. I could change for another software that does this well (Beehiiv do that well). Hope it helps and that Buttondown could become a potential migration candidate for me in the future.

[jmduke]

@slorber Super useful feedback, thank you! Two followups:

• What do you currently use for attribution beyond referrer_url + utm sources?
• Can you talk more about how you assess subscriber quality? Is it mostly a factor of open + click rate for a given cohort?

[slorber]

What do you currently use for attribution beyond referrer_url + utm sources?

I'm automatically saving many querystring attributes to convertkit custom fields + some extra like referrer, first accessed landing pathname and other meaningful things. This is not a convertkit feature, I had to figure how to save all that myself to their system on user signup.

Saving a lot of data and figuring out what to do with it later allow me to create segments retroactively. I am able to track the ROI of each marketing action I do so that I can double-down on what truly works.

Can you talk more about how you assess subscriber quality? Is it mostly a factor of open + click rate for a given cohort?

ConvertKit has a "subscriber scoring" feature, but it is quite shitty. Anyone that has opened any email in the past 3 months is considered 5*, so this is mostly relevant for low-quality newsletters (mine has 60% open-rate).

I can get the score distribution on a per-segment basis. For example the score of subscribers acquired from Twitter ads:

Here again, it's quite irrelevant because I only started tracking acquisition source properly... a few months ago, so all subscribers are 5* despite some not reading the last few issues.

Not satisfied with ConvertKit although it still works:

• the UX is so confusing, and not convenient: they don't even persist filters in querystring so impossible to bookmark an analytics dashboard (you have to click many buttons to know details on a segment, everytime)
• not possible to configure the scoring algo which is irrelevant by default

Pretty sure Beehiiv is the only platform that understands well the paid acquisition model currently, on which many very large newsletters rely today for growth.
I don't have practical experience though but that seems better than what I currently have on ConvertKit: https://blog.beehiiv.com/p/beehiiv-analytics

[slorber]

BTW if you want to know more about this business of newsletters with sponsors and paid acquisition, one great tech example is TLDR.tech (but there are many similar non-tech newsletters too, even larger)

TLDR quickly grew with paid advertising to almost a million subscribers with a solid business model. And now it's becoming a pool of tech newsletters with various topics so it's becoming even more profitable.

Read more about the details here:

• older from TLDR author: https://www.indiehackers.com/product/tldrnewsletter/how-i-grew-my-newsletter-to-130k-subs-in-20-months--M5WHZz_7XZQFKIKZzxs
• newer from external analysis of the business https://growthinreverse.com/tldr/

Media kit: https://tldr.tech/tldr-sponsor-kit.pdf

Unless the sponsorship calendar is not full, TLDR makes 36000€/day, 5 days a week. That's 9m€/year 🤯 Each subscriber brings 10€/year alone thanks to advertisers, so this newsletter is able to spend up to 5-10€ per subscriber on paid acquisition (which is a lot), and still be profitable.

---

On my side, I am not a daily newsletter but a weekly one: I am closer to 3€/year of income per subscriber. The quality ensure there's more word of mouth than there's churn, so I'm also trying to acquire subscribers for max of 2-3€/subscriber. But... I need to measure the quality of these subscribers because if subscribers from one source churn too fast, then the business is not profitable in the long term. I can spend more on sources where subscribers are high quality, even if the source is more expensive. I don't plan to become an empire like TLDR but still plan to grow to the 100k-200k range, and increase my "subscriber lifetime value" over time (products + maybe bi-weekly issues?) so that I can spend more on acquisition.

Hope you understood why these analytics are very important for that specific business model 🤪