-
Notifications
You must be signed in to change notification settings - Fork 80
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
port the numba integration prototype from Enterprise to OSS, fixes feature-523 #638
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, mostly. Couple changes needed. Hard for me to tell if there are any big departures from how I originally structured my patch?
I'll be testing this out locally now.
DB/src/main/java/io/deephaven/db/v2/sources/chunk/BooleanChunk.java
Outdated
Show resolved
Hide resolved
Great - was able to test the following, and it works: from deephaven import TableTools, java, numba as dnb
@dnb.vectorize
def my_manual(x: java.int, y: java.int) -> java.int:
return (x % 3) + y
IJ = TableTools.emptyTable(1000000) \
.view("I=(int)i", "J=(int)(i * 2)") \
.update(java.formula(my_manual, name = "K", column_names = ["I", "J"])) |
Good example of the chunking working: from deephaven import TableTools, java, numba as dnb
@dnb.vectorize
def my_vectorized(x: java.int, y: java.int) -> java.int:
return (x % 3) + y
vectorized_sum = TableTools.emptyTable(100000000) \
.view("I=(int)i", "J=(int)(i * 2)") \
.view(java.formula(my_manual, name = "K", column_names = ["I", "J"])) \
.sumBy() is very fast vs from deephaven import TableTools, java
def my_plain(x: java.int, y: java.int) -> java.int:
return (x % 3) + y
plain_sum = TableTools.emptyTable(100000000) \
.view("I=(int)i", "J=(int)(i * 2)") \
.view("K = (int)my_plain(I, J)") \
.sumBy() is very slow. |
…-check * upstream/main: Check documentation labels, fixes deephaven#627 (deephaven#671) Fix integer overflow in SortedRanges.invertOnNew. Fixes deephaven#666. (deephaven#667) Fix integer overflow in SortedRanges.subRangeByPos. Fixes deephaven#664 (deephaven#665) Deploy example scripts and data (deephaven#562) port the numba integration prototype from Enterprise to OSS, fixes feature-523 (deephaven#638) Fix IllegalStateException in TwoValuesContainer triggered from Index.insert(Chunk). Fixes deephaven#652. (deephaven#653) Javadoc search (deephaven#645) Support for jpy integration junit tests (deephaven#632) Populate snapshot request in js api, handle optional left table (deephaven#648) Fix a link formatting bug in CONTRIBUTING.md (deephaven#629)
* tag 'v0.0.2': (29 commits) Move label information to CI remove labels try pushing to my own org try actor Upgrade to GITHUB_TOKEN instead of PAT Explicit about docker/build-push-action version Explicit about docker/login-action version Explicit about docker/setup-buildx-action version Remove extraneous comments Update docker metadata steps Remove top level building of deephaven-jpy in CI, is handled implicitly in grpc-api step Fix nightly check (deephaven#641) Address `QueryTableTest.testReverseClipping` table reachability bug, and unit test listeners using `RetentionCache` (deephaven#644) Rename symbols in .proto files to follow standard conventions (deephaven#674) Check documentation labels, fixes deephaven#627 (deephaven#671) Fix integer overflow in SortedRanges.invertOnNew. Fixes deephaven#666. (deephaven#667) Fix integer overflow in SortedRanges.subRangeByPos. Fixes deephaven#664 (deephaven#665) Deploy example scripts and data (deephaven#562) port the numba integration prototype from Enterprise to OSS, fixes feature-523 (deephaven#638) Fix IllegalStateException in TwoValuesContainer triggered from Index.insert(Chunk). Fixes deephaven#652. (deephaven#653) ...
…ature-523 (deephaven#638) * port the numba integration prototype from Enterprise to OSS, WIP, fixes feature-523 * changes to make python unittest happy * restore important javadoc comments to the <type>Chunk.java files, fixes deephaven#523
this prototype in its current design must be used in an unpublished, explicit way. So it should not break anything visible to users.