Add ability to increase fetch cache size

[simonknittel]

Goals

Add the ability to set a custom maximum size for the fetch cache

Non-Goals

Any implications on platforms like Vercel. Those platforms probably need a way to cap the cache size anyway otherwise users could abuse them.

Background

We are working on a self-hosted backoffice application in which one of our route handler has to request an endpoint which returns a 4MB+ payload. Sadly we can't use the built-in cache of Next.js 13/app directory since it's capped to 1MB.

We would like to be able to set a custom maximum size for this cache.

Failed to set fetch cache https://example.com/ Error: fetch for over 1MB of data can not be cached
    at IncrementalCache.set (/Users/simon.knittel/Documents/my-app/node_modules/next/dist/server/lib/incremental-cache/index.js:224:23)
    at /Users/simon.knittel/Documents/my-app/node_modules/next/dist/server/lib/patch-fetch.js:132:70

Proposal

This configuration could probably fit into a Route Segment Option.

[onurcagan]

Anyone looking into this? 1 MB is kind of low for our use case as well. Some things we can split but some we can't.

[dan-ville]

According to my interactions with Vercel's support team, this should be possible by creating custom configs in the api routes as shown here https://nextjs.org/docs/api-routes/request-helpers#custom-config.

I have not tested this yet, however Vercel has expressed that any options from previous versions of NextJS should still be available for the app/api route handlers unless marked with "legacy" or "deprecated", or something to that extent.

[onurcagan]

Did anyone get this to work?

[mmomtchev]

No, and I don't think anyone will anytime soon, the limit is hardcoded:

next.js/packages/next/src/server/lib/incremental-cache/index.ts

Line 431 in 789f253

if (fetchCache && JSON.stringify(data).length > 2 * 1024 * 1024) {

(by the way, it is 2MB, the message used to be wrong)

[Pirulax]

Also, just a quick note on this, it seems incredibly wasteful to encode (possibly) 2 MB of data just to get it's length...

[mcapodici]

Hack:

Edit these:

node_modules\next\dist\esm\server\lib\incremental-cache\index.js
node_modules\next\dist\server\lib\incremental-cache\index.js

e.g. to this, which increases the size AND gives a server side log if it is exceeded and by how much

        if (fetchCache) {
            const dataLength = JSON.stringify(data).length;
            if (dataLength > 10 * 1024 * 1024) {
                console.warn(`fetch for over 10MB of data can not be cached. Length: ${dataLength}`);
                return;
            } else {
                console.log(`caching an object of length ${dataLength}`)
            }
        }

Run this:

npx patch-package next

Commit the generated patch file

In Vercel change Install Command to npm i && npx patch-package

As an aside: This is it showed me than in Vercel, cached values survive deployments (because only the > 2Mb thing was fetched, the others were not). Which is really nice.

[TechnicallyWeb3]

Good Hack! Here's an improvement on this method:

1. Change the same files as mentioned

node_modules\next\dist\esm\server\lib\incremental-cache\index.js:
node_modules\next\dist\server\lib\incremental-cache\index.js:

// populate the incremental cache with new data
    async set(pathname, data, revalidateSeconds, fetchCache, fetchUrl, fetchIdx) {
        if (this.dev && !fetchCache) return;
        // fetchCache has upper limit of 2MB per-entry currently
        const max_fetch_size = Number(process.env.MAX_FETCH_SIZE) || 2 //gets value set in next.config
        if (fetchCache && JSON.stringify(data).length > max_fetch_size * 1024 * 1024) {
            if (this.dev) {
                throw new Error(`fetch for over ${max_fetch_size}MB of data can not be cached`);
            }
            return;
        }
    //rest of code...

2. Modify the next.config file
   next.config.js:

/** @type {import('next').NextConfig} */
const nextConfig = {
    env:{
        MAX_FETCH_SIZE: 10 //in MB
    }
}

module.exports = nextConfig

3. Run the patch command as mentioned

This way you can easily change it isn't hardcoded anymore.

[YoavGrosman]

@mcapodici @TechnicallyWeb3 is this hack working well for you? did you encounter any caching issues with using it or is it safe to use?

[iloabn]

I've used this method for about a month now without any obvious issues. However my use case is that I generate a static export, so all the caching happens at buildtime and there's no real risk for me getting memory leaks or stuff like that. So all I can really say is that: increasing the cache limit manually with a patch seems to have no negative impact when generating a static export.

[remorses]

This is a very important feature, if you can't use the fetch cache because of the 2 MB limit and you are using headers() or cookies() it's impossible to use any type of caching with Next.js app router (because the "full router cache" is disabled)

[muhaimincs]

any solid flow or is there any configurable settings for this in place?

[kasvith]

Just came across this, Limit is increased to 2MB, thats it

[oniatsu]

I understand that unstable_cache is in a similar situation.
I use node-cache - npm as an alternative for this purpose.

If you want an individual data cache rather than a Route cache, this may be suitable for you since it has a more relaxed size limit and is stable rather than unstable.

[EryouHao]

@oniatsu But if this plan is adopted, once the application is redeployed, will the cache be reset or cleared?

[Antonytm]

I faced with same issue.

I have a non-optimized API endpoint that I don't control. I want to fetch data, and process it(leave only what I need).
But I can't do it because of this limitation.

[mprync]

This is an important thing to have. I'm also trying to cache a subset of data from an endpoint I can't optimise. This is possible in the pages router but not the app router..

No idea why this has been open for 6 months without a response.

@leerob sorry to tag you mate, but is there any magic we can do in the meantime to get this working in app router?

[pedrofrxncx]

i'm facing the same issue here. next.js docs should AT LEAST warn you about the 2mb cache limit for fetch requests

[ekimcem]

Don't fetched data are served from CDN ? we should be able to decide it since it will just use bandwidth, am I wrong ?

[mmomtchev]

After lots of fiddling with this value, my personal recommendation to everyone who stumbles upon this limit is to abstain from raising it. If you need to cache more than 2MB, try to refactor whatever there is to refactor. In my own case, I was loading a huge JSON and I ended having all kinds of problems (I ended creating everything-json for this reason which includes a lengthy explanation of why you should not load huge JSON files in server-side JavaScript and why no one has ever fixed this).

[marclelamy]

I'm my case, I'm moving away from using api endpoint to using server actions. It's supposed to fetch an image to then save it but throws an error when the files are >2mb.

I'm still new to this, should I keep in the api? Anything else I can do?

[mmomtchev]

I am afraid that you have missed my point - I am not saying that there are no valid reasons to load huge files. The problem is that this goes far beyond a simple hardcoded value in some Next.js file. The problem is Node.js itself, processing huge files in an async heavy environment is simply a weak point of the whole architecture. There is no easy fix. The only way you can do it without killing performance is to use a C++ addon that spawns a shared memory thread to do this job.

[lucajung]

+1

[4D5A90]

+1

[thereis]

+1

[anhonestboy]

I have a photography blog with a lot of pages with a lot of image content. I come from Gatsby and would love to still use the advantage of the static rendering. I've got the same error by fetching all the posts :(

[aaaronme]

We are getting data from a 3rd party API which is 2.1MB big... juuust above the limit. I guess we'll now just store it in a dynamodb table with a TTL

[Cizzl]

+1

[letanloc1998]

why this value is 1MB or 2MB?
I stuck in this issue and cannot find any solution. Maybe I must change to python or express.js. There are "pure" sever side rendering instead of next.js ~~

[sbouafif]

For anyone who dont need the cache by default, you can use the option cache: 'no-store'

You can read more about this here : https://nextjs.org/docs/app/building-your-application/data-fetching/fetching-caching-and-revalidating#opting-out-of-data-caching

[letanloc1998]

cache: 'no-store' causes another problem.
We should fix in the framework instead of working around.

[sbouafif]

Totally agree with you but a work around is better than nothing

[Notenlish]

why do we need to do use some hacky solution to get this to work, increasing the cache size shouldnt be that much of an issue, right? why is it limited to 2mb

[letanloc1998]

why do we need to do use some hacky solution to get this to work, increasing the cache size shouldnt be that much of an issue, right? why is it limited to 2mb

Totally agree, nowadays, data is getting larger, so 2MB is not enough.

[semenerror]

The simplest workaround to deal with 2 mb limit with Next 14.2.5 is to use custom cacheHandler in next.config.js . Limit is not being checked if custom handler is used.

You can implement your own handler or use third party handlers or just reimport default file-system-cache.js from Next distrib like this:

import FileSystemCache from 'next/dist/server/lib/incremental-cache/file-system-cache.js';

export default FileSystemCache;

[tefkah]

nice, good find!

[rvr31]

Can also do the following in next.config.js

    cacheHandler: require.resolve(
        'next/dist/server/lib/incremental-cache/file-system-cache.js',
    ),

[Notenlish]

For people who don't want to implement a custom cache handler, if you're just above the cache limit, you can try minifying and then de-minifying your keys if your response has a lot of them. You can also try gzip compression using https://www.npmjs.com/package/zlib .

[DarioLopes]

each days goes by and I discover some crazy stuff Next team is doing, and not in a good way. So no cache where it matters the most, fine. Thanks for nothing.

[terijaki]

A fix is on the way. Check out the new "use cache" coming soon: https://youtu.be/OWmRn74CQKY
This fixes the issue since the entire function return is cached.

[mcapodici]

A fix is on the way. Check out the new "use cache" coming soon: https://youtu.be/OWmRn74CQKY This fixes the issue since the entire function return is cached.

What is the size limit here? (If any)

[zaynelovecraft]

I think it is still 2mb because it uses next js built in data cache