fix: accept FixedSizeList dtype when deserializing ListValue scalars#8696
Merged
Conversation
Merging this PR will not alter performance
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
robert3005
approved these changes
Jul 9, 2026
Scalar serialization writes ScalarValue::Tuple as a protobuf ListValue regardless of whether the dtype is List or FixedSizeList, but list_from_proto only accepted List. Since vortex-data#8667 made constant detection unconditional in the cascading compressor, a constant FixedSizeList column (e.g. a UUID stored as fixed_size_list(u8)[16]) is written as a ConstantArray whose scalar metadata then fails to deserialize on read: expected List dtype for ListValue, got fixed_size_list(u8)[16] Scalar validation already handles FixedSizeList + Tuple (including the size check), so deserialization was the only gap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
Covers the ConstantArray metadata path for fixed-size list dtypes (e.g. a UUID column stored as fixed_size_list(u8)[16]): serialization writes the scalar as a protobuf ListValue, and deserialization previously rejected it with "expected List dtype for ListValue". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
Unable to generate the performance reportThere was an internal error while processing the run's data. We're working on fixing the issue. Feel free to contact us on Discord or at support@codspeed.io if the issue persists. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scalar serialization writes
ScalarValue::Tupleas a protobufListValuefor bothListandFixedSizeListdtypes, butlist_from_protoonly acceptedList. Since #8667 made constant detection unconditional, a constant fixed-size list column (e.g. UUIDs stored asfixed_size_list(u8)[16]) is compressed to aConstantArraywhose scalar metadata then fails to deserialize on read:so files written this way come back unreadable. Scalar validation already handles
FixedSizeList+Tuple(including the size check), so deserialization was the only gap.Includes a proto round-trip test that fails without the fix.