You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It seems that a BinaryViewArray of a single Some(&[]) comes back as None (null) after being roundtripped through parquet. A non-empty byte slice, such as Some(&[1]) works, however.
To Reproduce
Using version 52.1.0
#[test]
fn binary_view_bug() {
#[track_caller]
fn roundtrips(x: &[u8]) {
let before = BinaryViewArray::from(vec![x]);
let batch =
RecordBatch::try_from_iter(vec![("val", Arc::new(before.clone()) as ArrayRef)])
.unwrap();
let mut buf = Vec::new();
let mut writer = ArrowWriter::try_new(&mut buf, batch.schema(), None).unwrap();
writer.write(&batch).unwrap();
writer.close().unwrap();
let builder = ParquetRecordBatchReaderBuilder::try_new(Bytes::from(buf)).unwrap();
let mut reader = builder.build().unwrap();
let batch = reader.next().unwrap().unwrap();
let after = batch.columns()[0].as_binary_view();
assert_eq!(
before.iter().collect::<Vec<_>>(),
after.iter().collect::<Vec<_>>()
);
}
roundtrips(&[1]); // This one works
roundtrips(&[]); // This one fails
}
I have reproduced the bug from 52.1.0 on crates.io. We recently revamped the StringViewArray and ByteViewArray in arrow, but those changes do not propagate to 52.1.0, it should be fixed in the next release! cc @alamb
Describe the bug
It seems that a BinaryViewArray of a single
Some(&[])
comes back asNone
(null) after being roundtripped through parquet. A non-empty byte slice, such asSome(&[1])
works, however.To Reproduce
Using version 52.1.0
The output on failure is
Expected behavior
The value should be
Some([])
on decode.Additional context
The text was updated successfully, but these errors were encountered: