RFC - Move on-disk caching implementation to separate npm library

[Vadorequest]

I've implemented a hybrid memory/disk cache storage at #92

It works locally and on Vercel.

It's meant to help caching stuff that is shared by several Next.js SSG pages during the server initial build, because in-memory caching won't work in such case.

I've used it to reduce from 140 Airtable API calls to only 7, and considerably decrease build time for the whole app.

It could make sense to implement it as a separate library at some point in the future, to help re-usage. But I don't have time for this now.

Also, I'm interested in getting feedback about the limitations of the current implementation. It's meant to be simple (and stay that way). It doesn't rely on any other caching library.

[samuelcastro]

I'm wondering whether this would be helpful with GraphCMS? I know you have implemented another cache system here https://github.com/UnlyEd/GraphCMS-cache-boilerplate as well.

What do you think?

[Vadorequest]

The purpose of both caches is completely different.

• https://github.com/UnlyEd/GraphCMS-cache-boilerplate is a Redis cache meant to act as a proxy between any consumer and GraphCMS API, so that even if GCMS breaks then our apps don't.
• Add hybrid memory/disk cache storages to optimise Airtable API data fetching during server initial build #92 is meant to avoid each page to fetch its own data, which is completely useless when those data are identical (like with the Airtable preset, because I fetch 3 tables and all pages get the same data). Instead it stores in disk cache the results so that the other pages benefit from this cache instead of fetching the Airtable API hundreds of times. Also, Airtable API is very different from any GraphQL API as it won't allow to fetch results across multiple tables at once, those must be done through multiple API requests, which makes everything much slower. (GraphQL capabilities are far greater)

Now, to answer whether the GraphCMS preset could benefit from it ... 🤔
I think so, I'm not sure. I never needed such disk cache with that preset, maybe it would make the build step faster, but Redis is really fast already, so that may not be significant.
So, for what we're currently doing with it, I don't think it'd help.
But it could be useful for people who also fetch data from other data sources.

[samuelcastro]

Make sense now.

Thanks.

[Vadorequest]

HybridCache has been added to gcms preset 98f6db3

[samuelcastro]

Could you share the results? Have you noticed any improvements?

[Vadorequest]

I've just added the code source, but haven't used it and don't plan to 😉

But feel free to play around and let us know!