Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GLUTEN-6887][VL] Daily Update Velox Version (2024_08_28) #7041

Merged
merged 7 commits into from
Aug 30, 2024

Conversation

GlutenPerfBot
Copy link
Contributor

Upstream Velox's New Commits:

afd0ce2bc by Chengcheng Jin, Re-write Spark decimal add, subtract, multiple, divide as simple functions (10619)
edf5b9eb4 by Jacob Wujciak-Jens, Remove wall time from BM report (10829)
42c06ca9a by Christian Zentgraf, Fix several build issues in CI when using Clang (10800)
1497dc366 by xiaoxmeng, Add back memory reclaim async callback to remove async dependency on … (10845)
f996a0b71 by yan ma, Fix parsing array struct when repetition_type is 'REPEATED' (10753)
81a77f99f by Chengcheng Jin, Change FileInputStream Stats readTimeUs to readTimeNanos (10822)
046726faf by Masha Basmanova, Add registerMapFunction helper function (10857)
763c19c0c by duanmeng, Trace metadata during task creation (10815)
72c4921d2 by Serge Druzkin, Fix minor doc typos in SelectiveColumnReader.h (10835)
c74b5e19e by xiaoxmeng, Relax the reclaimed bytes check for hash build (10833)

Copy link

#6887

@github-actions github-actions bot added the CORE works for Gluten Core label Aug 29, 2024
Copy link

Run Gluten Clickhouse CI

1 similar comment
Copy link

Run Gluten Clickhouse CI

@zhouyuan
Copy link
Contributor

@GlutenPerfBot benchmark

@GlutenPerfBot
Copy link
Contributor Author

ACK, will benchmark TPCH/DS on this pull request

@github-actions github-actions bot removed the CORE works for Gluten Core label Aug 30, 2024
@@ -316,22 +316,27 @@ bool VeloxMemoryManager::tryDestructSafe() {

// Velox memory manager considered safe to destruct when no alive pools.
if (veloxMemoryManager_) {
if (veloxMemoryManager_->numPools() > 1) {
if (veloxMemoryManager_->numPools() > 2) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note this is due to
facebookincubator/velox@763c19c

@zhouyuan
Copy link
Contributor

@GlutenPerfBot benchmark

@zhouyuan zhouyuan merged commit 61ac8da into apache:main Aug 30, 2024
43 checks passed
@GlutenPerfBot
Copy link
Contributor Author

===== Performance report for TPCDS SF2000 with Velox backend, for reference only ====

query log/native_master_08_30_2024_time.csv log/native_master_08_29_2024_e7caab1da4_time.csv difference percentage
q1 14.27 13.88 -0.394 97.24%
q2 14.16 13.25 -0.916 93.53%
q3 4.27 4.36 0.088 102.07%
q4 69.94 72.35 2.411 103.45%
q5 9.20 8.05 -1.155 87.45%
q6 2.35 2.25 -0.102 95.65%
q7 6.96 6.53 -0.429 93.83%
q8 5.02 4.87 -0.151 96.99%
q9 24.45 24.26 -0.184 99.25%
q10 9.98 10.56 0.579 105.80%
q11 39.04 39.02 -0.020 99.95%
q12 1.69 1.53 -0.165 90.25%
q13 6.52 6.50 -0.029 99.56%
q14a 45.90 47.24 1.339 102.92%
q14b 42.71 43.24 0.537 101.26%
q15 4.03 2.78 -1.250 68.98%
q16 46.20 46.45 0.246 100.53%
q17 5.25 5.50 0.251 104.78%
q18 6.70 8.00 1.299 119.39%
q19 2.15 2.42 0.275 112.80%
q20 1.47 1.60 0.130 108.84%
q21 1.18 1.24 0.055 104.69%
q22 7.82 7.98 0.163 102.08%
q23a 103.78 104.75 0.970 100.93%
q23b 128.53 128.72 0.185 100.14%
q24a 114.54 111.28 -3.256 97.16%
q24b 115.33 111.05 -4.285 96.28%
q25 4.18 4.11 -0.068 98.38%
q26 3.21 3.02 -0.190 94.09%
q27 4.00 4.20 0.205 105.13%
q28 33.33 33.64 0.309 100.93%
q29 11.78 11.19 -0.589 95.00%
q30 5.06 5.93 0.873 117.25%
q31 9.13 7.94 -1.191 86.96%
q32 1.18 1.17 -0.016 98.66%
q33 4.11 4.30 0.187 104.53%
q34 3.80 3.81 0.010 100.26%
q35 7.47 7.89 0.423 105.66%
q36 4.72 4.73 0.004 100.08%
q37 4.78 4.73 -0.049 98.97%
q38 14.18 13.59 -0.587 95.86%
q39a 3.12 3.44 0.312 109.97%
q39b 3.12 3.06 -0.062 98.02%
q40 3.98 3.84 -0.143 96.42%
q41 0.64 0.64 -0.003 99.53%
q42 0.92 1.02 0.103 111.22%
q43 4.66 4.84 0.174 103.72%
q44 9.87 9.99 0.121 101.22%
q45 3.26 3.23 -0.036 98.91%
q46 3.86 3.82 -0.040 98.96%
q47 19.17 18.93 -0.242 98.74%
q48 5.40 5.46 0.061 101.14%
q49 8.91 9.03 0.116 101.30%
q50 21.75 21.58 -0.162 99.26%
q51 10.24 9.97 -0.275 97.31%
q52 1.06 1.13 0.067 106.27%
q53 2.47 2.44 -0.027 98.92%
q54 4.01 3.99 -0.024 99.40%
q55 1.19 1.10 -0.082 93.08%
q56 4.23 4.35 0.117 102.78%
q57 10.99 11.07 0.075 100.68%
q58 2.48 2.51 0.031 101.23%
q59 11.12 11.12 -0.000 100.00%
q60 4.09 4.10 0.013 100.32%
q61 4.18 4.19 0.008 100.18%
q62 4.64 4.89 0.255 105.50%
q63 2.41 2.50 0.092 103.84%
q64 62.31 60.58 -1.724 97.23%
q65 17.39 17.29 -0.108 99.38%
q66 3.71 4.06 0.357 109.62%
q67 395.26 408.72 13.463 103.41%
q68 3.72 3.79 0.071 101.92%
q69 5.33 5.56 0.235 104.42%
q70 11.92 12.10 0.177 101.49%
q71 2.47 2.41 -0.056 97.72%
q72 213.19 217.90 4.712 102.21%
q73 2.36 2.46 0.098 104.14%
q74 24.77 24.09 -0.680 97.25%
q75 26.57 27.30 0.737 102.78%
q76 12.52 11.60 -0.916 92.69%
q77 2.42 2.44 0.018 100.74%
q78 50.69 50.23 -0.462 99.09%
q79 3.85 4.08 0.230 105.97%
q80 12.08 12.35 0.272 102.25%
q81 5.28 4.88 -0.399 92.43%
q82 7.35 7.29 -0.054 99.26%
q83 1.65 1.69 0.033 101.97%
q84 2.76 2.86 0.093 103.35%
q85 7.35 7.12 -0.239 96.75%
q86 4.07 4.25 0.173 104.24%
q87 13.48 15.68 2.200 116.32%
q88 21.86 20.60 -1.254 94.26%
q89 3.80 3.72 -0.080 97.89%
q90 3.34 3.07 -0.266 92.04%
q91 2.57 2.21 -0.357 86.11%
q92 1.40 1.36 -0.046 96.70%
q93 38.81 39.22 0.410 101.06%
q94 24.92 24.32 -0.603 97.58%
q9 91.12 89.61 -1.510 98.34%
q5 2.76 2.80 0.041 101.47%
q96 17.59 17.49 -0.104 99.41%
q97 2.00 1.89 -0.102 94.88%
q98 9.84 10.03 0.182 101.85%
q99 9.84 10.03 0.182 101.85%
total 2208.68 2219.17 10.496 100.48%

@GlutenPerfBot
Copy link
Contributor Author

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_08_30_2024_time.csv log/native_master_08_29_2024_e7caab1da4_time.csv difference percentage
q1 39.40 39.06 -0.337 99.14%
q2 29.94 30.61 0.666 102.22%
q3 53.26 53.88 0.625 101.17%
q4 43.78 41.59 -2.190 95.00%
q5 104.06 104.66 0.603 100.58%
q6 10.91 11.08 0.168 101.54%
q7 114.46 118.69 4.223 103.69%
q8 116.41 115.51 -0.903 99.22%
q9 168.56 168.74 0.180 100.11%
q10 65.23 64.93 -0.299 99.54%
q11 29.62 27.73 -1.889 93.62%
q12 29.56 31.17 1.613 105.46%
q13 52.09 52.16 0.075 100.14%
q14 26.49 27.25 0.765 102.89%
q15 53.24 54.32 1.085 102.04%
q16 18.33 18.16 -0.172 99.06%
q17 131.02 130.29 -0.725 99.45%
q18 198.64 199.00 0.365 100.18%
q19 27.11 30.88 3.771 113.91%
q20 44.47 43.14 -1.331 97.01%
q21 375.61 387.80 12.196 103.25%
q22 15.55 17.22 1.677 110.79%
total 1747.72 1767.88 20.166 101.15%

sharkdtu pushed a commit to sharkdtu/gluten that referenced this pull request Nov 11, 2024
Upstream Velox's New Commits:

afd0ce2bc by Chengcheng Jin, Re-write Spark decimal add, subtract, multiple, divide as simple functions (10619)
edf5b9eb4 by Jacob Wujciak-Jens, Remove wall time from BM report (10829)
42c06ca9a by Christian Zentgraf, Fix several build issues in CI when using Clang (10800)
1497dc366 by xiaoxmeng, Add back memory reclaim async callback to remove async dependency on … (10845)
f996a0b71 by yan ma, Fix parsing array struct when repetition_type is 'REPEATED' (10753)
81a77f99f by Chengcheng Jin, Change FileInputStream Stats readTimeUs to readTimeNanos (10822)
046726faf by Masha Basmanova, Add registerMapFunction helper function (10857)
763c19c0c by duanmeng, Trace metadata during task creation (10815)
72c4921d2 by Serge Druzkin, Fix minor doc typos in SelectiveColumnReader.h (10835)
c74b5e19e by xiaoxmeng, Relax the reclaimed bytes check for hash build (10833)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants