[log] Fix the 'Destination buffer is too small' error while decompress data in ZstdArrowCompressionCodec

[swuferhong]

Purpose

Linked issue: #462

Currently, if we define a String in the table and stream write a very large size of data, such as a single row size 5k, the ZstdArrowCompressionCodec will definitely throw a RuntimeException with the message 'Destination buffer is too small' during decompression.

This pr is aims to fix this error. Through investigation, we found that if we replace the compression way using Zstd.compressUnsafe(long dst, long dstSize, long src, long srcSize, int level) with memory copy methods like Zstd.compressDirectByteBuffer(ByteBuffer dst, int dstOffset, int dstSize, ByteBuffer src, int srcOffset, int srcSize, int level), or switch to using ZstdOutputStreamNoFinalizer, the error no longer occurs. The possible reason is that the compressUnsafe way has some memory operation anomalies that are not being handled correctly.

Why we use Zstd.compressDirectByteBuffer() here is that the stream output way ZstdOutputStreamNoFinalizer will consume more cpu than previous way.

Tests

API and Format

Documentation