Fix Reader stale data, Writer buffer leak, and zero-length write#240
Conversation
lkinley-rythmos
commented
Feb 7, 2026
- Reader.read: clear r.data in direct mode so that a subsequent zero-length block does not copy stale pool data
- Writer.Reset: return data buffer to pool before resetting to prevent a memory leak on reuse
- Writer.ReadFrom: skip write and handler call when read returns zero bytes
- Reader.read: clear r.data in direct mode so that a subsequent zero-length block does not copy stale pool data - Writer.Reset: return data buffer to pool before resetting to prevent a memory leak on reuse - Writer.ReadFrom: skip write and handler call when read returns zero bytes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Fixes some bugs I have personally run into. Problem: When the source data is an exact multiple of the block size (e.g., 128MB with 4MB blocks), Problem: When reading in non-concurrent mode with a user buffer >= block size, the reader uses "direct mode" — decompressing directly into the user's buffer. But This is the reader-side companion to the first bug. Even with the writer fix, the reader should be robust against degenerate blocks from any encoder. Problem: |
|
Nice catch. Can you add tests for each fix please? |
…ero-length write - TestReader_DirectModeStaleData: synthetic LZ4 frame with a zero-length uncompressed block verifies direct-mode r.data clearing - TestReader_BufferIssue: end-to-end test with bundle testdata - TestWriter_ResetWithoutClose: Reset without Close returns buffer to pool - TestWriter_ReadFromZeroLengthRead: zero-byte reads are skipped in ReadFrom Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Tests added |
|
@lkinley-rythmos merged your patches but the "testdata/bundle.00001.part00000.lz4" file is missing, so the TestReader_BufferIssue fails. Can you provide it please? Thanks! |
|
I meant to remove that test -- the file is way too big to include. |
4.1.27 contains both pierrec/lz4#240 & pierrec/lz4#245 which are relevant when we're often compressing files with sizes that are a power of 2