cpp: align zstd default level and chunk flush behavior with other writers (#1804)

### Changelog

The C++ zstd compression defaults were aligned with the rest of the
languages by switching to compression level 3, which is slightly slower
but provides better compression.

### Docs

None.

### Description

The C++ writer diverged from the Go, Rust, Python, and TypeScript MCAP
writers in two ways that produce noticeably larger zstd files:

1. CompressionLevel::Default mapped to zstd level 1, while every other
implementation compresses at libzstd's default of level 3 (Go's
SpeedDefault is the equivalent). Change the mapping to 3.

2. chunkSize was treated as a hard cap: the writer flushed the current
chunk before writing any message that would push it past the target. All
other implementations treat chunkSize as a threshold, flushing only
after the chunk meets or exceeds it. Remove the pre-write flush so chunk
boundaries match, and update the chunkSize documentation to describe the
threshold semantics.

On a simulated 10-second robot recording (5 channels, 3750 messages,
~102 MB uncompressed), these two changes together shrink the C++ zstd
output from 42.7 MB to 29.3 MB, with chunk boundaries now byte-identical
to those produced by the Rust/Go/Python writers. Writes get
correspondingly slower (level 3 costs ~35% more CPU than level 1) but
remain the fastest of the five implementations on the same workload.

The "Message index records" unit test assumed the cap behavior (its
second message relied on the pre-write flush to start a new chunk); size
each message to cross the threshold on its own instead, which preserves
the regression the test guards against.

One important compatibility note: while this does change the defaults,
it will not affect rosbag2 out-of-the-box since it explicitly chooses
uncompressed as its default.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Author: clalancette

cpp/mcap/include/mcap/writer.hpp

@@ -56,11 +56,12 @@ struct MCAP_PUBLIC McapWriterOptions {
   bool noSummary = false;
   /**
    * @brief Target uncompressed Chunk payload size in bytes. Once a Chunk's
-   * uncompressed data is about to exceed this size, the Chunk will be
-   * compressed (if enabled) and written to disk. Note that this is a 'soft'
-   * ceiling as some Chunks could exceed this size due to either indexing
-   * data or when a single message is larger than `chunkSize`, in which case,
-   * the Chunk will contain only this one large message.
+   * uncompressed data meets or exceeds this size, the Chunk will be
+   * compressed (if enabled) and written to disk. Note that this is a
+   * threshold rather than a ceiling: a Chunk grows until the record that
+   * meets or exceeds this size has been written to it, so Chunks may exceed
+   * this size by up to one record. This matches the behavior of the other
+   * MCAP writer implementations (Go, Rust, Python, TypeScript).
    * This option is ignored if `noChunking=true`.
    */
   uint64_t chunkSize = DefaultChunkSize;

cpp/mcap/include/mcap/writer.inl

@@ -241,7 +241,7 @@ int ZStdCompressionLevel(CompressionLevel level) {
       return -3;
     case CompressionLevel::Default:
     default:
-      return 1;
+      return 3;
     case CompressionLevel::Slow:
       return 5;
     case CompressionLevel::Slowest:
@@ -577,14 +577,7 @@ Status McapWriter::write(const Message& message) {
     channelMessageCounts.emplace(message.channelId, 0);
   }
 
-  // Before writing a message that would overflow the current chunk, close it.
   auto* chunkWriter = getChunkWriter();
-  if (chunkWriter != nullptr && /* Chunked? */
-      uncompressedSize_ != 0 && /* Current chunk is not empty/new? */
-      9 + getRecordSize(message) + uncompressedSize_ >= chunkSize_ /* Overflowing? */) {
-    auto& fileOutput = *output_;
-    writeChunk(fileOutput, *chunkWriter);
-  }
 
   // For the chunk-local message index.
   const uint64_t messageOffset = uncompressedSize_;

cpp/test/unit_tests.cpp

@@ -552,7 +552,7 @@ TEST_CASE("McapReader::readMessages()", "[reader]") {
 /**
  * @brief ensures that message index records are only written for the channels present in the
  * previous chunk. This test writes two chunks with one message each in separate channels, with
- * the second message being large enough to guarantee the current chunk will be written out.
+ * each message being large enough to meet the chunk size threshold and close its chunk.
  * If the writer is working correctly, there will be one message index record after each chunk,
  * one for each message.
  */
@@ -574,10 +574,9 @@ TEST_CASE("Message index records", "[writer]") {
   writer.addChannel(channel2);
 
   mcap::Message msg;
-  // First message should not fill first chunk.
-  WriteMsg(writer, channel1.id, 0, 100, 100, std::vector<std::byte>{20});
-  // Second message fills current chunk and triggers a new one.
-  WriteMsg(writer, channel2.id, 0, 200, 200, std::vector<std::byte>{400});
+  // Each message meets the chunk size threshold, closing its chunk once written.
+  WriteMsg(writer, channel1.id, 0, 100, 100, std::vector<std::byte>(400));
+  WriteMsg(writer, channel2.id, 0, 200, 200, std::vector<std::byte>(400));
 
   writer.close();