-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Open
Labels
Description
Describe the bug
When casting a dictionary with string value to binary view, run into an unexpected error.
To Reproduce
This test should pass:
// arrow-cast/src/cast/mod.rs
#[test]
fn test_string_dict_to_binary_view() {
let values = StringArray::from_iter(VIEW_TEST_DATA);
let keys = Int8Array::from_iter([Some(1), Some(0), None, Some(3), None, Some(1), Some(4)]);
let string_dict_array =
DictionaryArray::<Int8Type>::try_new(keys, Arc::new(values)).unwrap();
let casted = cast(&string_dict_array, &DataType::BinaryView).unwrap();
assert_eq!(casted.data_type(), &DataType::BinaryView);
}Currently failing:
thread 'cast::tests::test_string_dict_to_binary_view' (4994586) panicked at arrow-cast/src/cast/mod.rs:6651:70:
called `Result::unwrap()` on an `Err` value: ComputeError("Internal Error: Cannot cast BinaryView to BinaryArray of expected type")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
test cast::tests::test_string_dict_to_binary_view ... FAILEDExpected behavior
Should successfully cast to a binary view.
Additional context
Discovered in DataFusion: apache/datafusion#18662 (comment)
Related code is here:
arrow-rs/arrow-cast/src/cast/dictionary.rs
Lines 107 to 125 in 2bc269c
| BinaryView => { | |
| // `unpack_dictionary` can handle Utf8View/BinaryView types, but incurs unnecessary data copy of the value buffer. | |
| // we handle it here to avoid the copy. | |
| let dict_array = array | |
| .as_dictionary::<K>() | |
| .downcast_dict::<BinaryArray>() | |
| .ok_or_else(|| { | |
| ArrowError::ComputeError( | |
| "Internal Error: Cannot cast BinaryView to BinaryArray of expected type" | |
| .to_string(), | |
| ) | |
| })?; | |
| let binary_view = view_from_dict_values::<K, BinaryViewType, BinaryType>( | |
| dict_array.values(), | |
| dict_array.keys(), | |
| )?; | |
| Ok(Arc::new(binary_view)) | |
| } |
- It's likely the same will happen for vice-versa situation, i.e. binary dictionary to string view (see above arm)
Related PR: #5871