Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL] Support ignoreNulls for NthValue window function #3857

Merged
merged 4 commits into from
Nov 29, 2023

Conversation

PHILO-HE
Copy link
Contributor

How was this patch tested?

Spark UTs & a new test added.

Copy link

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

Copy link

Run Gluten Clickhouse CI

Copy link

Run Gluten Clickhouse CI

val frame = wExpression.windowSpec.frameSpecification.asInstanceOf[SpecifiedWindowFrame]
val childrenNodeList = new JArrayList[ExpressionNode]()
childrenNodeList.add(
ExpressionConverter
.replaceWithExpressionTransformer(input, attributeSeq = originalInputAttributes)
.doTransform(args))
childrenNodeList.add(LiteralTransformer(offset).doTransform(args))
childrenNodeList.add(LiteralTransformer(Literal(ignoreNulls)).doTransform(args))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use makeLiteral to get an expression node.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will fix it in a separate PR. Thanks!

@PHILO-HE PHILO-HE merged commit 2528841 into apache:main Nov 29, 2023
17 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_3857_time.csv log/native_master_11_28_2023_e0f197fc2_time.csv difference percentage
q1 33.72 33.03 -0.690 97.95%
q2 25.00 24.97 -0.031 99.87%
q3 37.91 37.84 -0.079 99.79%
q4 37.39 38.37 0.978 102.62%
q5 72.58 71.84 -0.738 98.98%
q6 5.45 7.08 1.633 129.99%
q7 86.22 83.42 -2.803 96.75%
q8 86.70 86.34 -0.355 99.59%
q9 123.56 124.39 0.836 100.68%
q10 46.44 45.34 -1.104 97.62%
q11 20.63 20.27 -0.356 98.27%
q12 26.36 26.06 -0.300 98.86%
q13 47.21 47.23 0.019 100.04%
q14 14.48 13.83 -0.648 95.52%
q15 27.33 29.53 2.202 108.06%
q16 15.40 15.40 -0.001 99.99%
q17 103.24 102.38 -0.869 99.16%
q18 151.46 150.59 -0.874 99.42%
q19 12.83 13.04 0.216 101.69%
q20 27.50 27.68 0.181 100.66%
q21 225.76 223.37 -2.386 98.94%
q22 13.30 14.21 0.915 106.88%
total 1240.46 1236.20 -4.255 99.66%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants