Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CORE] Add logical link to rewritten spark plan #4817

Merged
merged 3 commits into from
Mar 1, 2024

Conversation

ulysses-you
Copy link
Contributor

What changes were proposed in this pull request?

After pull out project, the rewritten spark plan does not hold logical link any more, however, the fallback reason framework depends on the logcial link.
This pr adds the original plan logical link to the rewritten spark plan.

How was this patch tested?

add test

Copy link

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

Copy link

Run Gluten Clickhouse CI

@ulysses-you
Copy link
Contributor Author

cc @liujiayi771 @PHILO-HE thank you

spark.sparkContext.addSparkListener(listener)
withSQLConf(GlutenConfig.EXPRESSION_BLACK_LIST.key -> "add") {
try {
val df = spark.sql("select sum(id +1) from range(10)")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: missing a space after "+".

@liujiayi771
Copy link
Contributor

LGTM.

Copy link

github-actions bot commented Mar 1, 2024

Run Gluten Clickhouse CI

@yaooqinn yaooqinn merged commit e6f8e3e into apache:main Mar 1, 2024
6 checks passed
Copy link

github-actions bot commented Mar 1, 2024

Run Gluten Clickhouse CI

@ulysses-you ulysses-you deleted the logicallink branch March 1, 2024 07:00
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_4817_time.csv log/native_master_02_29_2024_22d9fe3c8_time.csv difference percentage
q1 34.32 33.44 -0.878 97.44%
q2 24.54 27.05 2.515 110.25%
q3 37.72 35.95 -1.762 95.33%
q4 37.48 39.22 1.738 104.64%
q5 72.32 71.30 -1.017 98.59%
q6 5.80 7.43 1.629 128.10%
q7 86.02 85.83 -0.186 99.78%
q8 85.94 85.20 -0.731 99.15%
q9 122.84 118.37 -4.462 96.37%
q10 42.64 43.82 1.175 102.76%
q11 20.21 20.13 -0.074 99.63%
q12 29.12 26.26 -2.862 90.17%
q13 45.61 46.41 0.798 101.75%
q14 20.43 20.53 0.108 100.53%
q15 28.17 28.82 0.641 102.28%
q16 14.26 12.90 -1.362 90.45%
q17 102.80 103.50 0.697 100.68%
q18 149.66 149.47 -0.190 99.87%
q19 12.77 12.55 -0.213 98.33%
q20 28.08 28.87 0.795 102.83%
q21 224.04 227.08 3.041 101.36%
q22 15.74 13.78 -1.961 87.54%
total 1240.49 1237.92 -2.562 99.79%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants