sitemap.xml cached somehow?

[darrylmorley]

Summary

I'm trying to update my sitemap.xml, but it appears to be cached somehow as changes aren't reflected on my live site when updating with a new version.

I use an SEO tool generated sitemap.xml file, which is placed in the app directory. Pushed to Github and then the site is rebuilt on my Caprover server.

I've tested it in various browsers with the same result. Where could this cache issue be?

Additional information

next 13.4.19
caprover server - docker / ngninx

[neelratanguria]

Same problem, no help. I am using nextjs and vercel for deployment.

[fellmann]

Maybe a result of #60998 ?

[neelratanguria]

I am using Nextjs 14.

I fixed the caching problem for API request using export const dynamic = 'force-dynamic'

Here is an example snippet

export async function GET(req, res) {
    try {
        // Make a GET request to the WorldTimeAPI
        const worldTimeResponse = await fetch('http://worldtimeapi.org/api/timezone/Asia/Kolkata', {
            cache:"no-cache"
        });
        
        if (!worldTimeResponse.ok) {
          throw new Error(`Failed to fetch data from WorldTimeAPI. Status: ${worldTimeResponse.status}`);
        }
    
        // Get the JSON response from the API
        const worldTimeData = await worldTimeResponse.json();
    
        // Convert the response data to a string (for demonstration purposes)
        const responseDataString = JSON.stringify(worldTimeData);
    
        // Return the response
        return new Response(responseDataString, { headers: { "Content-Type": "application/json" } });
      } catch (error) {
        // Handle errors, e.g., log the error and return an error response
        console.error('Error:', error.message);
        return new Response('Error fetching data from WorldTimeAPI', { status: 500 });
      }
}

export const dynamic = 'force-dynamic'

[jrhizor]

It looks like it only updates on deploy without force-dynamic for some reason. It really doesn't seem like that should be the case for sitemap.ts since there doesn't seem to be any other reason to have a programmatic sitemap?

[NabilNYMansour]

Just use https://nextjs.org/docs/app/api-reference/functions/unstable_noStore

For example:

import { myFetch } from '@/data';
import { unstable_noStore as noStore } from 'next/cache';

export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
  noStore();
  const data = await myFetch();

  return ...
}

That worked fine for me. Though keep in mind that there is a limit of 50K links in a sitemap for Google. You will need to use https://nextjs.org/docs/app/api-reference/functions/generate-sitemaps if you have more than that.

[fallenleavesguy]

Works well. awesome

[fellmann]

These solutions fix the issue, but not the underlying problem. Next.js framework should set caching headers correctly. It should not be necessary to fix it individually.

[NabilNYMansour]

My take is that Next.js should not cache until that is declared explicitly. So like instead of having a noStore function, we would have a cachePage function that needs to be declared on the top of the component. Or maybe something like "use cache"; on the top of the file.

[HelaGone]

In my case our sitemaps are generated by the CMS and stored in a S3, but for some reason they got cached as well since they pass through the frontend domain and Idk how to prevent this to happen. Please help! 😢

[mboynes]

According to the sitemap documentation:

Good to know: sitemap.js is a special Route Handler that is cached by default unless it uses a Dynamic API or dynamic config option.

The answer to the larger question is that the sitemap route is cached (indefinitely) by next's Full Route Cache. The Full Route Cache is cleared on new deployments. However, it should be noted that the Data Cache persists across deployments. Therefore, (as I understand it) it is theoretically possible that a Full Route Cache is regenerated after a deployment using data in the Data Cache.

In my use case, I didn't want to disable the Full Route Cache (or the Data Cache, for that matter); I want some caching on the sitemap, just not indefinite caching, since I'm pulling content dynamically from a CMS. I opted to go with using the revalidate config option for the page, setting that to 5 minutes. Here's an example of that:

import { MetadataRoute } from 'next';

export const revalidate = 300;

export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
  // ...
}