Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(wip)(test)Adding Kerberos DNS check to the Hive connection case #41361

Closed
wants to merge 7 commits into from
Closed

(wip)(test)Adding Kerberos DNS check to the Hive connection case #41361

wants to merge 7 commits into from

Conversation

CalvinKirs
Copy link
Member

Proposed changes

Issue Number: close #xxx

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@CalvinKirs
Copy link
Member Author

run buildall

@CalvinKirs
Copy link
Member Author

run buildall

@CalvinKirs
Copy link
Member Author

run buildall

@CalvinKirs
Copy link
Member Author

run buildall

@CalvinKirs CalvinKirs changed the title (chore)(test)Adding Kerberos DNS check to the Hive connection case (wip)(test)Adding Kerberos DNS check to the Hive connection case Sep 27, 2024
@CalvinKirs
Copy link
Member Author

run external

@CalvinKirs
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40630 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 742c48f63de3fc515a2e7e4d468dc65d3ed6277e, data reload: false

------ Round 1 ----------------------------------
q1	17574	7396	7351	7351
q2	2011	277	273	273
q3	12092	1073	1171	1073
q4	10570	758	786	758
q5	7692	2875	2763	2763
q6	245	151	151	151
q7	976	627	629	627
q8	9379	1968	1939	1939
q9	6567	6410	6405	6405
q10	6935	2327	2273	2273
q11	437	253	241	241
q12	409	229	222	222
q13	17808	2989	2979	2979
q14	250	221	209	209
q15	563	515	524	515
q16	646	587	592	587
q17	970	613	510	510
q18	7206	6720	6571	6571
q19	1399	933	1020	933
q20	494	227	215	215
q21	3951	3143	3050	3050
q22	1105	985	1007	985
Total cold run time: 109279 ms
Total hot run time: 40630 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7225	7305	7217	7217
q2	324	225	233	225
q3	2981	2939	2951	2939
q4	2018	1842	1849	1842
q5	5747	5685	5723	5685
q6	235	150	145	145
q7	2216	1853	1792	1792
q8	3375	3547	3441	3441
q9	8868	8877	8867	8867
q10	3588	3545	3500	3500
q11	577	495	496	495
q12	826	629	652	629
q13	10682	3244	3139	3139
q14	317	286	286	286
q15	591	513	531	513
q16	682	673	638	638
q17	1837	1601	1583	1583
q18	8195	7903	7471	7471
q19	1699	1430	1531	1430
q20	2113	1856	1877	1856
q21	5575	5418	5428	5418
q22	1175	1078	1023	1023
Total cold run time: 70846 ms
Total hot run time: 60134 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191622 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 742c48f63de3fc515a2e7e4d468dc65d3ed6277e, data reload: false

query1	904	393	404	393
query2	6269	2096	2025	2025
query3	8696	197	200	197
query4	33632	23591	23494	23494
query5	3689	485	466	466
query6	286	167	170	167
query7	4197	301	301	301
query8	293	230	231	230
query9	9518	2675	2680	2675
query10	472	282	292	282
query11	17912	15101	15156	15101
query12	146	102	98	98
query13	1547	423	405	405
query14	9542	7350	7374	7350
query15	251	170	177	170
query16	8003	455	467	455
query17	1638	609	588	588
query18	2151	304	305	304
query19	361	154	155	154
query20	125	112	110	110
query21	219	116	108	108
query22	4724	4691	4522	4522
query23	35462	33863	33846	33846
query24	10965	2899	2861	2861
query25	635	430	415	415
query26	1209	162	167	162
query27	2330	299	304	299
query28	7496	2460	2421	2421
query29	866	443	436	436
query30	252	155	152	152
query31	1027	755	796	755
query32	95	55	54	54
query33	768	301	300	300
query34	932	504	480	480
query35	875	727	747	727
query36	1109	938	922	922
query37	162	88	87	87
query38	3963	3928	3984	3928
query39	1453	1419	1412	1412
query40	211	96	97	96
query41	53	50	48	48
query42	117	101	100	100
query43	529	462	488	462
query44	1304	816	809	809
query45	199	164	163	163
query46	1161	727	734	727
query47	1962	1855	1862	1855
query48	451	376	389	376
query49	891	409	417	409
query50	842	414	418	414
query51	6958	6934	6937	6934
query52	104	83	89	83
query53	247	184	180	180
query54	1146	458	461	458
query55	74	72	75	72
query56	285	259	265	259
query57	1217	1080	1094	1080
query58	227	214	243	214
query59	3063	2968	2935	2935
query60	295	269	259	259
query61	105	103	101	101
query62	810	649	647	647
query63	217	189	183	183
query64	3999	646	646	646
query65	3233	3169	3192	3169
query66	726	307	298	298
query67	15917	15621	15387	15387
query68	4253	587	585	585
query69	463	290	284	284
query70	1171	1129	1049	1049
query71	352	264	265	264
query72	7269	4286	4085	4085
query73	774	339	345	339
query74	9630	8907	8979	8907
query75	3432	2705	2660	2660
query76	2893	964	966	964
query77	606	302	294	294
query78	10377	9587	9546	9546
query79	1869	591	599	591
query80	1342	460	457	457
query81	574	239	237	237
query82	722	135	135	135
query83	287	134	136	134
query84	272	77	81	77
query85	1383	301	288	288
query86	407	292	299	292
query87	4481	4313	4252	4252
query88	3883	2406	2366	2366
query89	408	283	288	283
query90	1913	185	182	182
query91	183	143	140	140
query92	61	49	47	47
query93	1098	563	534	534
query94	858	305	290	290
query95	353	255	257	255
query96	609	279	276	276
query97	3236	3145	3125	3125
query98	213	205	186	186
query99	1612	1318	1282	1282
Total cold run time: 297501 ms
Total hot run time: 191622 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.97 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 742c48f63de3fc515a2e7e4d468dc65d3ed6277e, data reload: false

query1	0.05	0.05	0.04
query2	0.05	0.03	0.02
query3	0.23	0.06	0.05
query4	1.65	0.10	0.10
query5	0.50	0.49	0.51
query6	1.14	0.72	0.72
query7	0.02	0.01	0.02
query8	0.04	0.03	0.02
query9	0.55	0.51	0.49
query10	0.57	0.59	0.54
query11	0.14	0.11	0.10
query12	0.13	0.11	0.11
query13	0.61	0.59	0.59
query14	2.72	2.76	2.84
query15	0.91	0.82	0.82
query16	0.38	0.38	0.38
query17	1.04	1.03	1.03
query18	0.20	0.19	0.19
query19	1.94	1.86	1.92
query20	0.01	0.01	0.01
query21	15.35	0.59	0.57
query22	2.61	1.73	2.12
query23	16.97	0.97	0.92
query24	2.59	0.38	1.27
query25	0.29	0.11	0.04
query26	0.35	0.14	0.14
query27	0.05	0.04	0.04
query28	11.40	1.09	1.06
query29	12.58	3.18	3.18
query30	0.24	0.06	0.05
query31	2.88	0.38	0.38
query32	3.26	0.47	0.46
query33	2.97	2.99	3.02
query34	17.10	4.48	4.53
query35	4.48	4.45	4.46
query36	0.68	0.47	0.49
query37	0.08	0.07	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.15	0.12	0.12
query41	0.09	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 107.13 s
Total hot run time: 31.97 s

@CalvinKirs
Copy link
Member Author

run External

@CalvinKirs
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40959 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7c87a26b5cdfb78b92bd85f0611784a8a6f1b843, data reload: false

------ Round 1 ----------------------------------
q1	17675	7677	7372	7372
q2	2042	291	272	272
q3	12325	1093	1219	1093
q4	10576	719	686	686
q5	7725	2954	2868	2868
q6	251	153	152	152
q7	1081	665	622	622
q8	9386	1986	2063	1986
q9	6619	6483	6483	6483
q10	7017	2283	2308	2283
q11	445	251	252	251
q12	418	217	216	216
q13	17783	3000	2997	2997
q14	250	211	205	205
q15	581	540	529	529
q16	644	587	589	587
q17	986	584	649	584
q18	7406	6620	6666	6620
q19	1358	961	1036	961
q20	501	214	204	204
q21	4056	3101	2997	2997
q22	1115	991	1000	991
Total cold run time: 110240 ms
Total hot run time: 40959 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7351	7326	7314	7314
q2	330	235	233	233
q3	3113	2955	2974	2955
q4	2090	1873	1876	1873
q5	5798	5766	5798	5766
q6	228	145	151	145
q7	2321	1877	1819	1819
q8	3451	3474	3569	3474
q9	8976	8960	8961	8960
q10	3613	3578	3573	3573
q11	598	511	490	490
q12	840	601	627	601
q13	10038	3210	3232	3210
q14	316	284	280	280
q15	573	539	516	516
q16	675	664	644	644
q17	1867	1649	1640	1640
q18	8290	7796	7525	7525
q19	1734	1514	1604	1514
q20	2118	1930	1880	1880
q21	5650	5458	5556	5458
q22	1142	1071	1046	1046
Total cold run time: 71112 ms
Total hot run time: 60916 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191195 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7c87a26b5cdfb78b92bd85f0611784a8a6f1b843, data reload: false

query1	889	388	416	388
query2	6242	2096	2015	2015
query3	8681	202	206	202
query4	33611	23621	23513	23513
query5	3302	493	464	464
query6	273	173	167	167
query7	4199	313	325	313
query8	291	228	221	221
query9	9368	2670	2642	2642
query10	471	282	292	282
query11	17813	15392	15174	15174
query12	157	100	96	96
query13	1584	465	442	442
query14	9350	7341	6993	6993
query15	246	173	172	172
query16	8024	451	435	435
query17	1669	607	626	607
query18	2128	316	310	310
query19	371	146	152	146
query20	123	109	117	109
query21	208	103	99	99
query22	4897	4485	4592	4485
query23	35127	33779	34868	33779
query24	11031	2817	2863	2817
query25	634	417	410	410
query26	1189	164	167	164
query27	2257	298	307	298
query28	7361	2437	2415	2415
query29	856	437	447	437
query30	290	155	149	149
query31	1039	790	803	790
query32	100	54	55	54
query33	767	290	281	281
query34	928	508	518	508
query35	866	749	734	734
query36	1102	920	960	920
query37	153	88	92	88
query38	4040	3932	3883	3883
query39	1479	1423	1434	1423
query40	212	97	100	97
query41	47	45	44	44
query42	116	97	93	93
query43	528	481	493	481
query44	1240	804	804	804
query45	200	169	166	166
query46	1131	696	715	696
query47	1908	1852	1854	1852
query48	433	348	341	341
query49	909	439	415	415
query50	837	436	405	405
query51	7047	6928	6912	6912
query52	99	87	84	84
query53	249	178	184	178
query54	1182	465	469	465
query55	81	78	76	76
query56	280	257	261	257
query57	1250	1146	1150	1146
query58	234	247	235	235
query59	3363	3058	3041	3041
query60	303	264	263	263
query61	102	100	100	100
query62	843	663	665	663
query63	221	188	183	183
query64	4006	642	646	642
query65	3242	3189	3206	3189
query66	728	291	303	291
query67	15883	15635	15484	15484
query68	3510	573	577	573
query69	493	301	301	301
query70	1188	1081	1141	1081
query71	352	273	265	265
query72	7143	4246	3987	3987
query73	756	352	338	338
query74	9855	8957	9031	8957
query75	3438	2679	2613	2613
query76	2311	854	903	854
query77	634	295	288	288
query78	10471	9594	9492	9492
query79	3376	610	599	599
query80	1192	440	457	440
query81	587	241	245	241
query82	629	145	139	139
query83	289	148	133	133
query84	279	80	77	77
query85	1323	299	283	283
query86	448	302	282	282
query87	4346	4312	4324	4312
query88	4481	2412	2393	2393
query89	414	285	313	285
query90	1837	184	188	184
query91	146	107	107	107
query92	68	48	48	48
query93	2974	545	550	545
query94	763	276	264	264
query95	355	260	258	258
query96	626	286	276	276
query97	3261	3104	3085	3085
query98	223	203	197	197
query99	1547	1301	1270	1270
Total cold run time: 299105 ms
Total hot run time: 191195 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.25 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7c87a26b5cdfb78b92bd85f0611784a8a6f1b843, data reload: false

query1	0.05	0.04	0.05
query2	0.06	0.02	0.03
query3	0.23	0.06	0.06
query4	1.65	0.11	0.10
query5	0.53	0.52	0.49
query6	1.13	0.74	0.72
query7	0.02	0.01	0.02
query8	0.04	0.03	0.04
query9	0.55	0.50	0.49
query10	0.55	0.55	0.54
query11	0.15	0.11	0.11
query12	0.13	0.12	0.12
query13	0.61	0.60	0.60
query14	2.71	2.71	2.74
query15	0.90	0.83	0.83
query16	0.39	0.37	0.39
query17	1.04	1.05	1.04
query18	0.19	0.19	0.20
query19	1.95	1.87	2.02
query20	0.01	0.02	0.01
query21	15.37	0.59	0.57
query22	2.70	1.67	2.30
query23	16.95	1.04	0.77
query24	2.92	1.07	0.86
query25	0.14	0.17	0.08
query26	0.62	0.14	0.14
query27	0.04	0.04	0.05
query28	10.73	1.11	1.07
query29	12.58	3.17	3.18
query30	0.24	0.07	0.06
query31	2.86	0.39	0.37
query32	3.27	0.47	0.45
query33	2.99	3.02	3.00
query34	17.06	4.43	4.50
query35	4.51	4.45	4.47
query36	0.67	0.48	0.49
query37	0.08	0.05	0.06
query38	0.04	0.03	0.03
query39	0.02	0.02	0.02
query40	0.16	0.12	0.12
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.97 s
Total hot run time: 32.25 s

@CalvinKirs
Copy link
Member Author

run external

1 similar comment
@CalvinKirs
Copy link
Member Author

run external

@CalvinKirs
Copy link
Member Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 40630 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit aca6c5edf0d25446a0ca1e0442a0e8a09162c655, data reload: false

------ Round 1 ----------------------------------
q1	17912	7534	7306	7306
q2	2030	281	280	280
q3	12263	1051	1199	1051
q4	10392	748	697	697
q5	7743	2826	2791	2791
q6	236	148	144	144
q7	1024	646	599	599
q8	9352	1938	1987	1938
q9	6570	6423	6404	6404
q10	7019	2259	2304	2259
q11	439	251	252	251
q12	410	215	214	214
q13	17764	2950	2968	2950
q14	253	220	209	209
q15	561	528	526	526
q16	668	580	578	578
q17	977	516	511	511
q18	7348	6704	6782	6704
q19	1350	952	975	952
q20	484	225	198	198
q21	3949	3271	3069	3069
q22	1100	1008	999	999
Total cold run time: 109844 ms
Total hot run time: 40630 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7257	7284	7559	7284
q2	331	225	227	225
q3	3028	2896	2910	2896
q4	2131	1941	1846	1846
q5	5738	5756	5724	5724
q6	231	148	143	143
q7	2197	1835	1833	1833
q8	3360	3594	3393	3393
q9	8842	8864	8796	8796
q10	3611	3564	3525	3525
q11	566	493	473	473
q12	812	632	640	632
q13	9235	3185	3136	3136
q14	300	265	267	265
q15	562	540	511	511
q16	675	640	639	639
q17	1834	1622	1603	1603
q18	8344	7891	7494	7494
q19	1700	1487	1354	1354
q20	2119	1870	1885	1870
q21	5634	5421	5407	5407
q22	1172	1038	1038	1038
Total cold run time: 69679 ms
Total hot run time: 60087 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191893 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit aca6c5edf0d25446a0ca1e0442a0e8a09162c655, data reload: false

query1	905	395	413	395
query2	6282	2129	2051	2051
query3	8689	196	203	196
query4	33758	23557	23464	23464
query5	3529	479	454	454
query6	262	162	171	162
query7	4197	302	302	302
query8	284	217	216	216
query9	9198	2654	2637	2637
query10	464	283	283	283
query11	17701	15090	15113	15090
query12	130	98	98	98
query13	1575	458	447	447
query14	9677	7014	7459	7014
query15	245	169	172	169
query16	7645	475	471	471
query17	1635	627	583	583
query18	1894	365	329	329
query19	299	154	167	154
query20	122	112	117	112
query21	221	110	105	105
query22	4797	4732	4734	4732
query23	35187	34014	34115	34014
query24	10989	2758	2844	2758
query25	614	398	389	389
query26	1202	160	166	160
query27	2566	290	301	290
query28	7733	2437	2434	2434
query29	750	431	420	420
query30	262	151	162	151
query31	1039	805	815	805
query32	96	53	53	53
query33	771	296	296	296
query34	910	499	528	499
query35	875	743	733	733
query36	1101	943	955	943
query37	145	92	81	81
query38	3994	3857	3845	3845
query39	1463	1421	1415	1415
query40	249	98	100	98
query41	46	46	44	44
query42	114	93	94	93
query43	522	486	492	486
query44	1262	795	802	795
query45	192	165	167	165
query46	1134	715	708	708
query47	1985	1863	1843	1843
query48	428	340	333	333
query49	933	401	392	392
query50	826	420	413	413
query51	7062	7034	6946	6946
query52	99	88	84	84
query53	251	184	184	184
query54	1252	469	481	469
query55	79	81	77	77
query56	273	261	275	261
query57	1256	1186	1180	1180
query58	245	259	230	230
query59	3211	2949	2927	2927
query60	302	269	258	258
query61	101	100	99	99
query62	872	652	682	652
query63	217	181	182	181
query64	4475	626	606	606
query65	3237	3190	3160	3160
query66	1357	301	301	301
query67	15926	15817	15769	15769
query68	4506	568	565	565
query69	513	291	309	291
query70	1213	1083	1148	1083
query71	360	263	275	263
query72	7336	3983	3947	3947
query73	777	343	341	341
query74	10143	9049	8979	8979
query75	3438	2657	2671	2657
query76	2842	858	872	858
query77	576	286	277	277
query78	10452	9594	9680	9594
query79	2060	592	609	592
query80	2442	445	436	436
query81	588	237	238	237
query82	683	138	135	135
query83	292	142	135	135
query84	284	79	78	78
query85	1638	299	294	294
query86	433	296	310	296
query87	4441	4307	4279	4279
query88	3335	2442	2399	2399
query89	396	287	284	284
query90	2084	185	186	185
query91	155	107	107	107
query92	63	48	51	48
query93	1706	548	544	544
query94	1194	291	283	283
query95	351	252	256	252
query96	613	288	279	279
query97	3267	3186	3201	3186
query98	221	195	189	189
query99	1516	1296	1298	1296
Total cold run time: 301030 ms
Total hot run time: 191893 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 33.54 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit aca6c5edf0d25446a0ca1e0442a0e8a09162c655, data reload: false

query1	0.04	0.05	0.04
query2	0.06	0.03	0.03
query3	0.23	0.06	0.06
query4	1.64	0.11	0.10
query5	0.52	0.51	0.51
query6	1.14	0.72	0.72
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.56	0.51	0.50
query10	0.54	0.55	0.56
query11	0.14	0.11	0.11
query12	0.13	0.10	0.11
query13	0.62	0.60	0.60
query14	2.71	2.77	2.73
query15	0.90	0.83	0.81
query16	0.37	0.37	0.38
query17	1.03	1.04	1.03
query18	0.20	0.20	0.20
query19	1.84	1.76	1.98
query20	0.01	0.01	0.01
query21	15.36	0.57	0.58
query22	2.71	2.58	2.38
query23	17.07	0.88	0.84
query24	2.84	1.60	1.36
query25	0.31	0.14	0.04
query26	0.59	0.14	0.13
query27	0.03	0.04	0.04
query28	10.10	1.10	1.06
query29	12.54	3.34	3.27
query30	0.24	0.05	0.06
query31	2.86	0.38	0.37
query32	3.29	0.47	0.46
query33	2.95	3.00	2.99
query34	17.26	4.43	4.44
query35	4.55	4.56	4.50
query36	0.66	0.47	0.50
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.57 s
Total hot run time: 33.54 s

@CalvinKirs CalvinKirs closed this Oct 8, 2024
@CalvinKirs CalvinKirs deleted the master-tc-hive-two-kerberos branch October 8, 2024 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@CalvinKirs @doris-robot