[GLUTEN-3749][VL] fix redundant Velox build

[zhouyuan]

What changes were proposed in this pull request?

remove redundant velox build

(Fixes: #3749)

How was this patch tested?

pass GHA

[github-actions]

#3749

[GlutenPerfBot]

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query	log/native_3759_time.csv	log/native_master_11_19_2023_5f5d18abe_time.csv	difference	percentage
q1	34.90	34.50	-0.394	98.87%
q2	24.90	24.57	-0.329	98.68%
q3	37.96	37.14	-0.818	97.85%
q4	36.35	36.74	0.387	101.07%
q5	70.86	70.93	0.066	100.09%
q6	7.16	7.09	-0.069	99.04%
q7	82.45	83.62	1.161	101.41%
q8	87.04	85.87	-1.174	98.65%
q9	125.69	126.22	0.523	100.42%
q10	45.61	46.75	1.145	102.51%
q11	20.70	19.85	-0.853	95.88%
q12	24.13	25.57	1.444	105.98%
q13	45.08	45.26	0.186	100.41%
q14	17.10	18.39	1.290	107.54%
q15	29.47	28.36	-1.109	96.24%
q16	15.42	15.85	0.424	102.75%
q17	100.08	100.27	0.189	100.19%
q18	147.54	147.67	0.130	100.09%
q19	13.10	13.00	-0.107	99.18%
q20	27.11	26.75	-0.362	98.66%
q21	221.12	222.79	1.678	100.76%
q22	13.09	13.20	0.103	100.78%
total	1226.86	1230.37	3.511	100.29%