Skip to content

Commit b6b85df

Browse files
clalancetteclaude
andauthored
cpp, go: default LZ4 to fast mode, matching the other writers (#1807)
### Changelog The C++ and Go writers' default LZ4 compression was changed from the high-compression codepaths (LZ4HC level 9 and pierrec Level3 respectively) to fast mode, matching the Rust and Python writers. Default LZ4 writes get roughly 3-5x faster and produce files roughly 15-30% larger. Callers that prefer ratio over speed can keep the old behavior through the existing slower CompressionLevel settings. ### Docs None. ### Description CompressionLevel::Default diverged for LZ4 across the writers: C++ mapped it to LZ4HC_CLEVEL_DEFAULT (LZ4HC level 9) and Go to pierrec lz4.Level3, while Rust and Python compress in LZ4's fast mode. (TypeScript does not ship an LZ4 writer.) High-compression LZ4 defeats the reason to pick LZ4 in the first place: both writers' own zstd defaults strictly dominate their LZ4 defaults after #1804 — on the same workload C++ zstd wrote in 710 ms producing a 71 MB file while C++ LZ4 took 2130 ms to produce 78 MB, slower and larger. Users choosing LZ4 over zstd are choosing speed. Map CompressionLevel::Default to fast mode in both writers. On 1M 100-byte messages sharing one payload blob: - C++: 2.23 s -> 0.46 s (~4.8x faster), 81 MB -> 103 MB - Go: 2.19 s -> 0.77 s (~2.9x faster), 82 MB -> 93 MB - Rust, for reference, writes the same workload in 0.67 s at its existing fast-mode default. The high-compression levels remain reachable: C++ keeps LZ4HC through CompressionLevel::Slow (LZ4HC_CLEVEL_OPT_MIN) and Slowest (LZ4HC_CLEVEL_MAX), and Go keeps lz4.Level6/Level9 through CompressionLevelBetter/Best. Go's fallback for unknown level values also follows the default to fast mode, mirroring how its zstd mapping falls back to its own default. As with #1804, rosbag2 is unaffected out of the box since it defaults to uncompressed writing. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 45b64e3 commit b6b85df

2 files changed

Lines changed: 9 additions & 4 deletions

File tree

cpp/mcap/include/mcap/writer.inl

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -165,10 +165,12 @@ int LZ4CompressionLevel(CompressionLevel level) {
165165
case CompressionLevel::Fastest:
166166
return -1; // "fast acceleration"
167167
case CompressionLevel::Fast:
168-
return 0; // "fast mode"
169168
case CompressionLevel::Default:
170169
default:
171-
return LZ4HC_CLEVEL_DEFAULT;
170+
// Fast mode, matching the default of the other MCAP writer implementations (Go, Rust,
171+
// Python). LZ4HC remains available through Slow/Slowest for callers that prefer ratio
172+
// over the speed that makes LZ4 worth choosing.
173+
return 0; // "fast mode"
172174
case CompressionLevel::Slow:
173175
return LZ4HC_CLEVEL_OPT_MIN;
174176
case CompressionLevel::Slowest:

go/mcap/writer.go

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -899,15 +899,18 @@ type WriterOptions struct {
899899
func encoderLevelFromLZ4(level CompressionLevel) lz4.CompressionLevel {
900900
switch level {
901901
case CompressionLevelDefault:
902-
return lz4.Level3
902+
// Fast mode, matching the default of the other MCAP writer implementations (C++,
903+
// Rust, Python). The compressing levels remain available through Better/Best for
904+
// callers that prefer ratio over the speed that makes LZ4 worth choosing.
905+
return lz4.Fast
903906
case CompressionLevelFastest:
904907
return lz4.Fast
905908
case CompressionLevelBetter:
906909
return lz4.Level6
907910
case CompressionLevelBest:
908911
return lz4.Level9
909912
default:
910-
return lz4.Level3
913+
return lz4.Fast
911914
}
912915
}
913916

0 commit comments

Comments
 (0)