-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GH-134: Add Validity functions #135
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you check CI failures?
arrow/compute/scalar_compare.go
Outdated
@@ -134,4 +136,50 @@ func RegisterScalarComparisons(reg FunctionRegistry) { | |||
reg.AddFunction(ltFn, false) | |||
lteFn := makeFlippedCompare("less_equal", gteFn, EmptyFuncDoc) | |||
reg.AddFunction(lteFn, false) | |||
|
|||
isOrNotNullKns := kernels.IsNullKernels() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
isNullKns
or isNullOrNotNullKns
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
renamed kernels.IsNullKernels()
to kernels.IsNullNotNullKernels()
to make it more obvious
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, sorry. I mentioned a local variable name not a function name here.
@kou it took a while, but i figured out the cause of the failures. it was the upgrade to the new version of |
@kou I don't know why the pre-commit is failing though, it passes just fine when I run locally on this branch. Also for some reason it's not showing the output here so i can know what the actual failure is caused by, which is weird. When i run locally, if there's failures on the linting it outputs it properly to the terminal. |
The lint failure was caused by mixing Go 1.22 and 1.23. See also: golangci/golangci-lint#4908 I've pushed a commit to ensure using Go 1.23 for pre-commit. |
Adding
is_null
,is_not_null
, andis_nan
functions and tests to the Compute implementations for Go.This also removes the
-ffast-math
option from the SIMD optimized compiler Makefile so that our comparisons properly handleNaN
values (-ffast-math
assumes that no values areNaN
and removes the checks).is_nan
can be easily implemented for float and double values by dispatching toA != A
.Tests are also added for the new functions.
Ultimately the driving force behind this is as these are necessary for implementing scanning reads in iceberg-go