Repository navigation
fix: reject malformed varints in compressed batches instead of panicking or misreading - #908
Open
solace-aross wants to merge 2 commits into
Open
solace-aross wants to merge 2 commits into
solace-aross wants to merge 2 commits into
Conversation
solace-aross
requested review from
sgamelin,
shortishly and
solace-wkourlas
as code owners
October 9, 2026 00:05
solace-aross
force-pushed
the
fix/sol-155081-snappy-fuzz-target
branch
from
October 9, 2026 17:04
b74d996 to
a372edc
Compare
The existing fuzz_deflated_batch target only parsed the batch header, so the decompressors and the record decoder were never fuzzed. A Snappy batch whose record data was the xerial magic followed by fewer than 12 bytes panicked in Compression::inflator and went unnoticed (SOL-155081). - Add fuzz_batch_records: the first input byte selects the codec, the next two give the record count, the rest is the record data. It runs Vec::<Record>::try_from and inflated::Batch::try_from, by reference and by value, and asserts the two Vec::<Record> conversions agree. - fuzz_deflated_batch now also decodes the records of a parsed batch. - generate_seeds writes a valid seed per codec, a xerial-framed Snappy seed, and the magic plus 0 to 11 bytes. - Fix the fuzz-generate-seed recipe (cargo fuzz run does not accept --package or --bin) and add a fuzz-batch-records recipe. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Andrea Ross <168456375+solace-aross@users.noreply.github.com>
The serde VarInt and LongVarInt deserializers, used to decode records from a compressed batch, shifted and added without overflow checks. A varint with more continuation bytes than its type can hold panicked with "attempt to shift left with overflow" in builds with overflow checks, and decoded to a silently wrapped value in builds without them. Use checked arithmetic and return an error, as UnsignedVarInt and both Decode implementations already do. Found by the fuzz_batch_records target (SOL-155081). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Andrea Ross <168456375+solace-aross@users.noreply.github.com>
solace-aross
force-pushed
the
fix/sol-155081-snappy-fuzz-target
branch
from
October 9, 2026 20:54
a372edc to
94e0abc
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes, and why?
A record in a compressed batch whose varint field has more continuation bytes than the field can hold is now rejected as corrupt. Before, it panicked in builds with overflow checks and decoded to a silently wrapped value in release builds (
[profile.release]sets nooverflow-checks).VarIntandLongVarIntdeserializers, used to decode records from a compressed batch, shifted and added without overflow checks. They now use checked arithmetic and return an error, asUnsignedVarIntand bothDecodeimplementations already do. Like those, this rejects a shift past the type's width; it does not reject high bits lost on the final byte.fuzz_batch_records(new), which found the bug. The existingfuzz_deflated_batchtarget only parsed the batch header, so the decompressors and the record decoder were never fuzzed. The first input byte selects the codec, the next two give the record count, and the rest is the record data. It runsVec::<Record>::try_fromandinflated::Batch::try_from, by reference and by value, as the storage engines do, and checks that the twoVec<Record>results agree.fuzz_deflated_batchnow also decodes the records of a batch it parses.generate_seedswrites a valid seed per codec, a xerial-framed Snappy seed, and the xerial magic followed by 0 to 11 bytes.justfile: fixfuzz-generate-seed(cargo fuzz rundoes not accept--packageor--bin) and addfuzz-batch-records.The Snappy xerial header slice panic that motivated this work is already fixed on main (#821). This change does not touch that code.
Upgrade impact
None. A batch that previously decoded to a wrapped value, or panicked, now fails to decode as corrupt.
How was this tested?
attempt to shift left with overflowatvarint.rs:159and:354) and pass with the fix.just fuzz-generate-seed, thenjust cargo-fuzz run fuzz_batch_records -- -max_total_time=600: 923,961 executions, no crash.just fmt,just clippy(workspace, all targets,-D warnings, which covers thefuzzcrate) andcargo nextest run -p nisshi-sans-io --all-features(400 passed) pass locally. CI runs the other crates' tests.🤖 Generated with Claude Code