You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
I filed #7485 to fix some potentially problematic cases with biwiseMergeAndSetValidity. Originally I wanted to stop using the API wherever possible, but the fix for the operator itself is not that bad, so we decided to go that route. But this is a really dangerous API and I just don't know if the performance savings using it outweigh the inherent danger in it.
Like if the column that is passed in, which will have validity set for it, has nulls in it at all, this API can stomp over the top of it and make it not null. This can result in us using/reading bogus data.
I still think we should try to stop using this API and run some performance tests to see what the impact is of moving away from it.
The text was updated successfully, but these errors were encountered:
It also helped to make the API safer and showed that there is a performance improvement over doing an if/else (20% to 40% better). So it makes me rethink this a bit, but I still don't like bitwiseMergeAndSetValidity as an API.
It it is confusing to use, especially compared to something like if/else. So as an implementation I think rapidsai/cudf#13335 has addressed my concerns. And an API no it has not.
Is your feature request related to a problem? Please describe.
I filed #7485 to fix some potentially problematic cases with
biwiseMergeAndSetValidity
. Originally I wanted to stop using the API wherever possible, but the fix for the operator itself is not that bad, so we decided to go that route. But this is a really dangerous API and I just don't know if the performance savings using it outweigh the inherent danger in it.Like if the column that is passed in, which will have validity set for it, has nulls in it at all, this API can stomp over the top of it and make it not null. This can result in us using/reading bogus data.
I still think we should try to stop using this API and run some performance tests to see what the impact is of moving away from it.
The text was updated successfully, but these errors were encountered: