"Array buffer allocation failed" when finalizing with FileSystemWritableFileStreamTargetWriter

[rawmahn]

Hi,

So I'm trying to export a large file (around 1.5GB), and using FileSystemWritableFileStreamTargetWriter, but it crashes right upon trying to call muxer.finalize().

mp4-muxer.mjs:714 Uncaught (in promise) RangeError: Array buffer allocation failed
    at new ArrayBuffer (<anonymous>)
    at new Uint8Array (<anonymous>)
    at FileSystemWritableFileStreamTargetWriter.createChunk_fn (mp4-muxer.mjs:714:1)
    at FileSystemWritableFileStreamTargetWriter.writeDataIntoChunks_fn (mp4-muxer.mjs:665:1)
    at FileSystemWritableFileStreamTargetWriter.writeDataIntoChunks_fn (mp4-muxer.mjs:685:1)
    at FileSystemWritableFileStreamTargetWriter.write (mp4-muxer.mjs:650:1)
    at Muxer.finalize (mp4-muxer.mjs:849:1)

Using the latest version 5.1.5, but it did the same in 3.x that I had in my proejct previously.

If this helps this is how I initialize the muxer:

this.muxer = new Muxer({
            fastStart: 'in-memory',
            target: new FileSystemWritableFileStreamTarget(stream),
            video: {
                codec: 'avc',
                width: this.w,
                height: this.h
            },
            audio: {
                codec: 'aac',
                sampleRate: this.audioSampleRate,
                numberOfChannels: this.audioNumberOfChannels
            },
            firstTimestampBehavior: 'offset'
        });

It works well while I keep adding the frames, but crashes upon "finalize". It works fine for smaller files.

Any help would be appreciated.

[Vanilagy]

Thanks for reporting this! Looking at my code, the method simply allocates a 16 MB buffer, so that alone cannot be the issue. Perhaps something can't get garbage collected and you're simply running out of memory for the large file, which is of course not something that should happen. Does this error occur for small files too?

Do you think you can provide me with a demo that creates such a large file or should I simply mess around and come up with something myself?

[rawmahn]

For smaller files it works like a charm. While you might be correct about something else leaking in my app, this error occurs way too consistently when rendering large files. I will see if I get a chance to extract this in an isolated piece of code.. And please let me know if you think I could try some sort of a workaround.. Thanks!

…

On Tue, Nov 26, 2024, 11:09 AM Vanilagy ***@***.***> wrote: Thanks for reporting this! Looking at my code, the method simply allocates a 16 MB buffer, so that alone cannot be the issue. Perhaps something can't get garbage collected and you're simply running out of memory for the large file, which is of course not something that should happen. Does this error occur for small files too? Do you think you can provide me with a demo that creates such a large file or should I simply mess around and come up with something myself? — Reply to this email directly, view it on GitHub <#78 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABQ4ZOPYVW23O32V5RNK3A32CRCG5AVCNFSM6AAAAABSODC44WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMBQGE4TKNJWG4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[Vanilagy]

You could try using the regular StreamTarget (non-chunked) and see if that works. So just:

new StreamTarget({
    onData: (data, position) => stream.write({
        type: 'write',
        data,
        position
    })
})

where stream is a FileSystemWritableFileStream.

[kong303917]

I has the same issue.
When I used StreamTarget , the page crashed.
How can I fix it ?

[Vanilagy]

What is the error specifically?

[kong303917]

The page likes this.

[Vanilagy]

I guess this is an out of memory error. I can't help you without further detail

[0-mandrixx-0]

I have a similar issue when muxing large files (~2GB, 1h video) with ArrayBufferTarget
it stops after encoding 40 minutes of full hd video.

I have this error :
writer.ts:103 Uncaught (in promise) RangeError: Array buffer allocation failed
at new ArrayBuffer ()
at #ensureSize (writer.ts:103:19)
at ArrayBufferTargetWriter.write (writer.ts:112:8)
at #writeCurrentChunk (muxer.ts:362:65)
at #addSampleToTrack (muxer.ts:280:33)
at Muxer.addVideoChunkRaw (muxer.ts:241:8)
at Muxer.addVideoChunk (muxer.ts:228:8)
at MP4mux.addVideoChunk (MP4Mux.ts:67:22)

I have plenty of free ram on my computer and chrome is at around 3.5GB when the error occurs.

[Vanilagy]

Yes that is to be expected, a tab cannot use all your system memory. Don't use ArrayBufferTarget for such a large file!!

[0-mandrixx-0]

Yes. there seem to be a limit of 4GB with chrome, for security reasons.
I have no other choice, filesystemapi is not available on iOS devices and the storage options are limited ^
Ty anyway !

[Vanilagy]

Maybe you can try hacking something together using IndexedDB, but may be hacky. Or something with OPFS?

[0-mandrixx-0]

yes, I've implemented indexedDB and it's working fine, even with gigabytes of files stored. Another option is to use localstorage, but it requires base64 encoding/decoding.
I can't find much info about this 4GB limitation, But it seems to be a per tab limit. I will try to find out why it's stopping at 2gb, there must be something else eating the memory.