vite build crashes with "JavaScript heap out of memory"

[jofflin]

Checklist

• I have searched for existing issues to avoid duplicates.
• The bug is reproducible in the latest version of Paraglide JS.
• I am willing to submit a pull request for this issue.

Describe the Bug

I am using the following vite config

export default defineConfig({
  server: {
    port: 3000,
  },
  build: {
    chunkSizeWarningLimit: 1000,
    sourcemap: false,
  },
  plugins: [
    paraglideVitePlugin({
      project: './project.inlang',
      outdir: './src/lib/paraglide',
      strategy: ['localStorage', 'baseLocale'],
      cookieName: 'company-language',
      localStorageKey: 'company-language',
      outputStructure: 'locale-modules',
      // outputStructure: 'message-modules',
    }),
    tsConfigPaths({
      projects: ['./tsconfig.json'],
    }),
    cloudflare({ viteEnvironment: { name: 'ssr' } }),
    tanstackStart({
      sitemap: {
        enabled: true,
      },
    }),
    viteReact(),
    sentryVitePlugin({
      org: 'my-company',
      project: 'saas',
      authToken: process.env.SENTRY_AUTH_TOKEN,
      disable: process.env.NODE_ENV === 'development',
    }),
  ],
});

If I use the locale-modules everything works fine (although slow in build) but if I use the suggested and prefered message-modules the system crashes with

<--- Last few GCs --->

[91421:0x138008000]    87812 ms: Mark-Compact 3687.4 (4177.6) -> 3663.1 (4129.6) MB, pooled: 192 MB, 3068.88 / 0.00 ms  (average mu = 0.137, current mu = 0.034) allocation failure; GC in old space requested
[91421:0x138008000]    90593 ms: Mark-Compact 3711.7 (4181.2) -> 3687.2 (4133.2) MB, pooled: 240 MB, 2685.50 / 0.00 ms  (average mu = 0.090, current mu = 0.034) allocation failure; GC in old space requested

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 0x1001bb56c node::OOMErrorHandler(char const*, v8::OOMDetails const&) [/Users/jofflin/.local/share/fnm/node-versions/v24.7.0/installation/bin/node]
 2: 0x1003afea4 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/Users/jofflin/.local/share/fnm/node-versions/v24.7.0/installation/bin/node]
 3: 0x10060e030 v8::internal::Heap::stack() [/Users/jofflin/.local/share/fnm/node-versions/v24.7.0/installation/bin/node]
 4: 0x100611404 v8::internal::Heap::HasLowYoungGenerationAllocationRate() [/Users/jofflin/.local/share/fnm/node-versions/v24.7.0/installation/bin/node]
 5: 0x100622620 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags)::$_1::operator()() const [/Users/jofflin/.local/share/fnm/node-versions/v24.7.0/installation/bin/node]
 6: 0x100621ef4 void heap::base::Stack::SetMarkerAndCallbackImpl<v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags)::$_1>(heap::base::Stack*, void*, void const*) [/Users/jofflin/.local/share/fnm/node-versions/v24.7.0/installation/bin/node]
 7: 0x100fd748c PushAllRegistersAndIterateStack [/Users/jofflin/.local/share/fnm/node-versions/v24.7.0/installation/bin/node]
 8: 0x10060bd34 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/jofflin/.local/share/fnm/node-versions/v24.7.0/installation/bin/node]
 9: 0x100600a60 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/jofflin/.local/share/fnm/node-versions/v24.7.0/installation/bin/node]
10: 0x1005cc6d8 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArrayWithFiller(v8::internal::DirectHandle<v8::internal::Map>, int, v8::internal::DirectHandle<v8::internal::HeapObject>, v8::internal::AllocationType) [/Users/jofflin/.local/share/fnm/node-versions/v24.7.0/installation/bin/node]
11: 0x10094f524 v8::internal::Handle<v8::internal::OrderedHashMap>::MaybeType v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, 2>::Rehash<v8::internal::Handle>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::OrderedHashMap>, int) [/Users/jofflin/.local/share/fnm/node-versions/v24.7.0/installation/bin/node]
12: 0x100a67db0 v8::internal::Runtime_MapGrow(int, unsigned long*, v8::internal::Isolate*) [/Users/jofflin/.local/share/fnm/node-versions/v24.7.0/installation/bin/node]
13: 0x1010c5f74 Builtins_CEntry_Return1_ArgvOnStack_NoBuiltinExit [/Users/jofflin/.local/share/fnm/node-versions/v24.7.0/installation/bin/node]
14: 0x10108d960 Builtins_MapPrototypeSet [/Users/jofflin/.local/share/fnm/node-versions/v24.7.0/installation/bin/node]
15: 0x150d747d8 
16: 0x150f4e390 
17: 0x150d74768 
18: 0x1502407f4 
19: 0x150f4adb4 
20: 0x150f47e04 
21: 0x150f46fe0 
22: 0x150f46f08 
23: 0x150f47d24 
24: 0x150f46df0 
...

I assume this has to do with the files generated for each. Is there maybe a configuration issue on my side?

{
  "$schema": "https://inlang.com/schema/project-settings",
  "baseLocale": "de",
  "locales": [
    "en",
    "de"
  ],
  "modules": [
    "https://cdn.jsdelivr.net/npm/@inlang/plugin-message-format@4/dist/index.js",
    "https://cdn.jsdelivr.net/npm/@inlang/plugin-m-function-matcher@2/dist/index.js"
  ],
  "plugin.inlang.messageFormat": {
    "pathPattern": [
      "./messages/{locale}/account.json",
      "./messages/{locale}/admin.json",
      "./messages/{locale}/analytics.json",
      "./messages/{locale}/auth.json",
      "./messages/{locale}/cards.json",
      "./messages/{locale}/checkout.json",
      "./messages/{locale}/common.json",
      "./messages/{locale}/coupons.json",
      "./messages/{locale}/dashboard.json",
      "./messages/{locale}/devices.json",
      "./messages/{locale}/dialogs.json",
      "./messages/{locale}/errors.json",
      "./messages/{locale}/events.json",
      "./messages/{locale}/finance.json",
      "./messages/{locale}/forms.json",
      "./messages/{locale}/hooks.json",
      "./messages/{locale}/inventory.json",
      "./messages/{locale}/layout.json",
      "./messages/{locale}/onboarding.json",
      "./messages/{locale}/organization.json",
      "./messages/{locale}/products.json",
      "./messages/{locale}/public.json",
      "./messages/{locale}/roles.json",
      "./messages/{locale}/search.json",
      "./messages/{locale}/ui.json",
      "./messages/{locale}/venues.json",
      "./messages/{locale}/landing.json"
    ]
  }
}

We have around 10k keys per language.

{
  "extends": [
    "@company/typescript-config/tanstack.json"
  ],
  "compilerOptions": {
    "baseUrl": ".",
    "paths": {
      "@/*": [
        "./src/*"
      ]
    },
    "noUncheckedIndexedAccess": true,
    "strictNullChecks": true
  },
  "references": [
    {
      "path": "../api"
    }
  ],
  "exclude": [
    "node_modules",
    "dist"
  ]
}

What's the expected result?

The build should succeed no matter which output structure is chosen.

I will provide reproduction repo if there is not a silly mistake in my config

Link to Minimal Reproducible Example

https://github.com

Additional Information

No response

[samuelstroschein]

The bug seems to stem from holding all message (files) in memory before writing them to disk.

10k message files * N locales in memory could exceed the default 4GB limit in Node JS. A solution could be streaming the output to flush it from memory.

I ran a test but couldn't significantly reduce the memory footprint / analyze in the first place where things are coming from.

I happily accept a PR

[jofflin]

Okay, for now i go with locale-modules or export NODE_OPTIONS=--max-old-space-size. But I will look into this at some point!

[Samuel-Morgan-Tyghe]

@jofflin Im going to take a stab and say vite is cache-ing it when it shouldnt 🤔 and id be curious if optimizeDeps.exclude your translation build/dist/internals will fix it

[jofflin]

I will revert and test that after christmas, maybe already next week and give feedback