Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] SusbtraitToVeloxPlan error #1028

Closed
FelixYBW opened this issue Feb 25, 2023 · 7 comments
Closed

[VL] SusbtraitToVeloxPlan error #1028

FelixYBW opened this issue Feb 25, 2023 · 7 comments
Labels
bug Something isn't working velox backend works for Velox backend

Comments

@FelixYBW
Copy link
Contributor

FelixYBW commented Feb 25, 2023

Describe the bug
Observed by customer
image

To Reproduce
customer env.

@FelixYBW FelixYBW added the bug Something isn't working label Feb 25, 2023
@weiting-chen weiting-chen moved this to 🆕 New in Gluten 0.5.0 Mar 1, 2023
@weiting-chen weiting-chen added the velox backend works for Velox backend label Mar 1, 2023
@FelixYBW
Copy link
Contributor Author

FelixYBW commented Mar 2, 2023

@jinchengchenghh @rui-mo @JkSelf Looks it's another case which passed validation but failed on execution. Is GetFlatConditions a sql function?

@jinchengchenghh
Copy link
Contributor

No. This is in SubstraitToVelox plan conversion, SelectionNode is not suported. which should be validated but seems not.

@JkSelf
Copy link
Contributor

JkSelf commented Mar 3, 2023

The filter condition may be scalar, AND, OR expression. The SelectionNode may be the core case.

@JkSelf
Copy link
Contributor

JkSelf commented Mar 3, 2023

@FelixYBW Can we get the physical plan for this case?

@rui-mo rui-mo changed the title [VL] Row2Column conversion error [VL] SusbtraitToVeloxPlan error Mar 3, 2023
@rui-mo
Copy link
Contributor

rui-mo commented Mar 3, 2023

Tried to throw exception in flattenConditions as below.

void SubstraitVeloxPlanConverter::flattenConditions(
    const ::substrait::Expression& substraitFilter,
    std::vector<::substrait::Expression_ScalarFunction>& scalarFunctions,
    std::vector<::substrait::Expression_SingularOrList>& singularOrLists,
    std::vector<::substrait::Expression_IfThen>& ifThens) {
  VELOX_FAIL("Failed.");

But found the validation could take effect and scan fallbacked with success, so I'm not sure why fallback does not occur in this issue.

WARNING: Logging before InitGoogleLogging() is written to STDERR
E0303 10:28:57.100342 3581697 Exceptions.h:68] Line: ../../velox/substrait/SubstraitToVeloxPlan.cpp:1066, Function:flattenConditions, Expression:  Failed., Source: RUNTIME, ErrorCode: INVALID_STATE
ReadRel validation failed due to:Failed.

Screenshot from 2023-03-03 03-10-06

@jinchengchenghh
Copy link
Contributor

What's the config spark.gluten.sql.enable.native.validation? It should be true, I cannot find any code path can run to here with validation

@jinchengchenghh
Copy link
Contributor

Can we close this issue, it should have been fixed by https://github.com/oap-project/gluten/pull/952/files

@FelixYBW FelixYBW closed this as completed Apr 4, 2023
@github-project-automation github-project-automation bot moved this from 🆕 New to ✅ Done in Gluten 0.5.0 Apr 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working velox backend works for Velox backend
Projects
None yet
Development

No branches or pull requests

5 participants