build(deps): datafusion 41 #2917
Conversation
| // See: https://github.com/apache/datafusion/issues/11477 | ||
| for planner in context_provider.state.expr_planners() { | ||
| sql_to_rel = sql_to_rel.with_user_defined_planner(planner.clone()); | ||
| } |
There was a problem hiding this comment.
resolved in apache/datafusion#11485
eddyxu
requested review from
westonpace and
wjones127
and removed request for
westonpace
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2917 +/- ##
=======================================
Coverage 78.99% 78.99%
=======================================
Files 234 234
Lines 72829 72849 +20
Branches 72829 72849 +20
=======================================
+ Hits 57530 57546 +16
+ Misses 12329 12326 -3
- Partials 2970 2977 +7
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
@wjones127 thanks! the python tests exposed another issue with |
|
I ran this from the github workflow locally and it was successful. Any ideas on the failing java run in github? export JAVA_TOOL_OPTIONS "$JAVA_TOOL_OPTIONS -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED -Djdk.reflect.useDirectMethodHandle=false -Dio.netty.tryReflectionSetAccessible=true"
mvn clean install |
This succeeds on Mac for me, but fails on Linux. So I think that's the reason. |
|
Apologies for the delay. I've been trying to figure out what to do about the Java failure. It seems like we don't have the bandwidth or tooling yet to debug this issue, so we are going to just comment those tests out for now and leaving fixing them as a TODO. |
Upgrades datafusion and associated crates to version 41. 42 was just released which also has an arrow update to 53.
We're looking to adopt lance in our service where we've already upgraded to 41 for some other dependencies.