-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Python] Support mask
in FixedSizeListArray.from_arrays
#34316
Comments
That doesn't appear to work: In [8]: primitive_array = pa.array([1, 2, 3, 4, 5], mask=pa.array([True, False, True, False, True]))
In [9]: list_arr = pa.FixedSizeListArray.from_arrays(primitive_array, 1)
In [10]: list_arr.is_valid()
Out[10]:
<pyarrow.lib.BooleanArray object at 0x1028a4dc0>
[
true,
true,
true,
true,
true
] In your example the validity is assigned to the underlying primitive array, not the fixed size list array itself. According to https://arrow.apache.org/docs/format/Columnar.html#buffer-listing-for-each-layout the FixedSizeListArray should store its own validity array. I want to ensure FFI compatibility between Arrow implementations, which means it's important to be able to set the fixed size list array's mask correctly. I'm looking to store geoarrow points, not multidimensional arrays, so those discussions don't appear to be relevant to me |
Indeed |
@kylebarron thanks for raising the issue! Similarly as we added this keyword to Short term, as @rok notes,
|
HI, I am working on this issue. |
…eter (#39396) ### What changes are included in this PR? Add `mask` / `null_bitmap` parameters in corresponding Cython / C++ `FixedSizeListArray` methods, and propagate this bitmap instead of using the current dummy `validity_buf`. ### Are these changes tested? Yes ### Are there any user-facing changes? Yes, `mask` parameter has been added to `FixedSizeListArray.from_arrays` * Closes: #34316 Authored-by: LucasG0 <guillermou.lucas@gmail.com> Signed-off-by: Will Jones <willjones127@gmail.com>
… parameter (apache#39396) ### What changes are included in this PR? Add `mask` / `null_bitmap` parameters in corresponding Cython / C++ `FixedSizeListArray` methods, and propagate this bitmap instead of using the current dummy `validity_buf`. ### Are these changes tested? Yes ### Are there any user-facing changes? Yes, `mask` parameter has been added to `FixedSizeListArray.from_arrays` * Closes: apache#34316 Authored-by: LucasG0 <guillermou.lucas@gmail.com> Signed-off-by: Will Jones <willjones127@gmail.com>
… parameter (apache#39396) ### What changes are included in this PR? Add `mask` / `null_bitmap` parameters in corresponding Cython / C++ `FixedSizeListArray` methods, and propagate this bitmap instead of using the current dummy `validity_buf`. ### Are these changes tested? Yes ### Are there any user-facing changes? Yes, `mask` parameter has been added to `FixedSizeListArray.from_arrays` * Closes: apache#34316 Authored-by: LucasG0 <guillermou.lucas@gmail.com> Signed-off-by: Will Jones <willjones127@gmail.com>
… parameter (apache#39396) ### What changes are included in this PR? Add `mask` / `null_bitmap` parameters in corresponding Cython / C++ `FixedSizeListArray` methods, and propagate this bitmap instead of using the current dummy `validity_buf`. ### Are these changes tested? Yes ### Are there any user-facing changes? Yes, `mask` parameter has been added to `FixedSizeListArray.from_arrays` * Closes: apache#34316 Authored-by: LucasG0 <guillermou.lucas@gmail.com> Signed-off-by: Will Jones <willjones127@gmail.com>
Describe the enhancement requested
In
ListArray.from_arrays
it's possible to pass in a mask array defining which list elements are null.FixedSizeListArray.from_arrays
does not currently have amask
parameter, so it appears impossible to create a fixed size list array where some elements are null?Component(s)
Python
The text was updated successfully, but these errors were encountered: