You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For most List objects, there are two FieldNode objects in the RecordBatchList: one for the List itself, and one for the underlying values. For some strange reason, the arrow creators decided that strings are special, so if you have an array of strings, there is only a single FieldNode representing the array itself.
The only reason I can think of to do this would be a tiny bit of compression of the metadata. Consistency of the format seems like a horrible price to pay for that.
Anyway, I'm keeping this issue open because I'm worried that this problem that the underlying data of arrays cannot be constructed from knowledge of the underlying data alone, but requires additional metadata, might be an ongoing issue.
The text was updated successfully, but these errors were encountered:
For most
List
objects, there are twoFieldNode
objects in theRecordBatchList
: one for theList
itself, and one for the underlying values. For some strange reason, the arrow creators decided that strings are special, so if you have an array of strings, there is only a singleFieldNode
representing the array itself.The only reason I can think of to do this would be a tiny bit of compression of the metadata. Consistency of the format seems like a horrible price to pay for that.
Anyway, I'm keeping this issue open because I'm worried that this problem that the underlying data of arrays cannot be constructed from knowledge of the underlying data alone, but requires additional metadata, might be an ongoing issue.
The text was updated successfully, but these errors were encountered: