Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](restore) Add synchronized to avoid concurrent modification #43172 #43277

Merged
merged 2 commits into from
Nov 5, 2024
Merged

[fix](restore) Add synchronized to avoid concurrent modification #43172 #43277

merged 2 commits into from
Nov 5, 2024

Conversation

w41ter
Copy link
Contributor

@w41ter w41ter commented Nov 5, 2024

cherry pick from #43172

@w41ter
[fix](restore) Add synchronized to avoid concurrent modification (apa…
e3e238c 
…che#43172)

Without synchronized protection, when the user manually executes cancel,
the backupHandler may also trigger cancel simultaneously. Modifying the
snapshotInfo by both at the same time will cause a concurrent
modification exception.

The pending state will synchronously wait for the task of creating a
replica to end. Therefore, a creating state needs to be added to convert
the synchronous wait into asynchronous polling to avoid blocking the
user's cancel/list job info requests.
@w41ter
Copy link
Contributor Author

w41ter commented Nov 5, 2024

run buildall

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@w41ter
Copy link
Contributor Author

w41ter commented Nov 5, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49170 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e4dc82946778601225f6b2f56faa20716154c53b, data reload: false

------ Round 1 ----------------------------------
q1	17871	4435	4322	4322
q2	2068	165	148	148
q3	10257	1954	1960	1954
q4	10107	1317	1301	1301
q5	8417	3886	3952	3886
q6	236	124	125	124
q7	2080	1606	1601	1601
q8	9522	2734	2707	2707
q9	13474	10010	9764	9764
q10	8646	3544	3523	3523
q11	417	244	262	244
q12	461	307	297	297
q13	18334	3950	4005	3950
q14	355	327	327	327
q15	522	470	473	470
q16	529	453	468	453
q17	1133	981	965	965
q18	7338	6884	6833	6833
q19	1673	1576	1490	1490
q20	522	312	315	312
q21	4508	4119	4092	4092
q22	483	407	407	407
Total cold run time: 118953 ms
Total hot run time: 49170 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4277	4281	4329	4281
q2	334	231	226	226
q3	4196	4146	4153	4146
q4	2754	2766	2768	2766
q5	7203	7133	7062	7062
q6	234	122	124	122
q7	3280	2877	2875	2875
q8	4385	4532	4610	4532
q9	13687	13505	13428	13428
q10	4257	4298	4263	4263
q11	764	712	714	712
q12	1014	848	868	848
q13	7059	3764	3747	3747
q14	455	433	427	427
q15	506	465	460	460
q16	636	593	607	593
q17	3816	3894	3802	3802
q18	8765	8784	8742	8742
q19	1724	1697	1674	1674
q20	2391	2159	2094	2094
q21	8524	8481	8468	8468
q22	1027	961	914	914
Total cold run time: 81288 ms
Total hot run time: 76182 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 213131 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e4dc82946778601225f6b2f56faa20716154c53b, data reload: false

query1	945	429	384	384
query2	6539	2291	2290	2290
query3	6939	203	196	196
query4	23229	21654	21908	21654
query5	19739	6536	6485	6485
query6	290	228	260	228
query7	4338	302	295	295
query8	257	259	236	236
query9	3080	2674	2598	2598
query10	470	327	312	312
query11	15733	15112	14921	14921
query12	130	80	75	75
query13	1039	454	450	450
query14	17250	13647	13691	13647
query15	370	225	235	225
query16	6475	283	262	262
query17	1760	912	946	912
query18	892	317	329	317
query19	222	159	155	155
query20	111	105	99	99
query21	193	96	96	96
query22	5166	4992	4961	4961
query23	34376	33375	33628	33375
query24	7000	6419	6369	6369
query25	527	437	432	432
query26	1019	166	160	160
query27	2303	300	290	290
query28	6078	2246	2224	2224
query29	2984	2803	2735	2735
query30	259	171	166	166
query31	953	739	754	739
query32	80	65	66	65
query33	446	265	257	257
query34	872	480	490	480
query35	1126	940	969	940
query36	1170	1074	1121	1074
query37	101	60	64	60
query38	3135	2910	2935	2910
query39	1365	1314	1332	1314
query40	221	100	97	97
query41	41	39	38	38
query42	88	87	87	87
query43	656	687	581	581
query44	1164	728	724	724
query45	247	231	231	231
query46	1241	974	962	962
query47	1985	1791	1724	1724
query48	506	424	419	419
query49	633	376	396	376
query50	852	640	611	611
query51	4802	4668	4787	4668
query52	94	83	93	83
query53	236	186	185	185
query54	2666	2474	2499	2474
query55	97	82	88	82
query56	216	217	202	202
query57	1318	1218	1129	1129
query58	217	215	218	215
query59	3730	3241	3208	3208
query60	224	203	219	203
query61	99	98	96	96
query62	868	442	536	442
query63	216	178	182	178
query64	3331	1560	1520	1520
query65	3636	3553	3527	3527
query66	788	391	444	391
query67	15568	15580	15176	15176
query68	10256	627	637	627
query69	498	264	278	264
query70	1581	1534	1383	1383
query71	415	316	320	316
query72	6954	4920	4885	4885
query73	749	323	318	318
query74	6329	5852	5836	5836
query75	5450	3733	3647	3647
query76	5787	1151	1194	1151
query77	987	273	259	259
query78	12581	11797	11655	11655
query79	7619	650	676	650
query80	1595	390	398	390
query81	489	238	240	238
query82	1627	101	95	95
query83	164	130	135	130
query84	254	73	72	72
query85	884	318	321	318
query86	333	287	286	286
query87	3270	3054	3012	3012
query88	4533	2286	2321	2286
query89	477	284	285	284
query90	1934	214	221	214
query91	175	128	127	127
query92	61	53	55	53
query93	6232	602	576	576
query94	701	215	213	213
query95	1977	2059	2069	2059
query96	664	326	328	326
query97	6442	6393	6499	6393
query98	223	195	196	195
query99	2976	934	856	856
Total cold run time: 318918 ms
Total hot run time: 213131 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.01 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e4dc82946778601225f6b2f56faa20716154c53b, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.26	0.05	0.05
query4	1.79	0.06	0.08
query5	0.54	0.52	0.50
query6	1.25	0.62	0.62
query7	0.02	0.01	0.02
query8	0.03	0.03	0.02
query9	0.52	0.47	0.47
query10	0.55	0.55	0.53
query11	0.12	0.09	0.09
query12	0.12	0.10	0.09
query13	0.62	0.61	0.62
query14	0.79	0.79	0.78
query15	0.78	0.77	0.78
query16	0.39	0.36	0.36
query17	1.03	0.99	0.99
query18	0.21	0.29	0.22
query19	1.95	1.83	1.86
query20	0.02	0.01	0.01
query21	15.49	0.55	0.55
query22	2.06	2.12	1.74
query23	17.18	1.09	0.94
query24	6.02	1.16	1.52
query25	0.42	0.08	0.06
query26	0.66	0.16	0.16
query27	0.05	0.04	0.04
query28	6.89	0.74	0.79
query29	12.59	2.31	2.26
query30	0.57	0.51	0.50
query31	2.80	0.39	0.38
query32	3.37	0.50	0.49
query33	3.10	3.09	3.04
query34	15.25	4.79	4.78
query35	4.84	4.85	4.86
query36	1.06	1.01	1.02
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.15
query41	0.06	0.01	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.01
Total cold run time: 103.81 s
Total hot run time: 31.01 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit e4dc82946778601225f6b2f56faa20716154c53b with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.8 seconds inserted 10000000 Rows, about 458K ops/s

@dataroaring dataroaring merged commit 53af206 into apache:branch-2.0 Nov 5, 2024
21 of 23 checks passed
@w41ter w41ter deleted the pick_43172_to_origin_branch-2.0 branch November 6, 2024 06:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
kind/test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@w41ter @doris-robot @dataroaring