Adding sync_wait_with_variant #5956

hcq9102 · 2022-07-17T05:44:31Z

Any background context you want to provide?

P2300R5: sync_wait_with_variant
sync_wait_with_variant, which accepts any sender, but returns an optional whose value type is the variant of all the possible tuples sent by the input sender:

Checklist

I have added a new feature and have added tests to go along with it.

hkaiser

Very nice! Please add the new file here: https://github.com/STEllAR-GROUP/hpx/blob/master/libs/core/execution/CMakeLists.txt#L26, and add the new test to the build system (here: https://github.com/STEllAR-GROUP/hpx/blob/master/libs/core/execution/tests/unit/CMakeLists.txt#L19).

Also, now that you have a working version that duplicates 95% of the sync_wait code, would you see a way to combine the implementations of sync_wait and sync_wait_with_variant to reduce the amount of code repetition?

hcq9102 · 2022-07-17T19:55:15Z

Also, now that you have a working version that duplicates 95% of the sync_wait code, would you see a way to combine the implementations of sync_wait and sync_wait_with_variant to reduce the amount of code repetition?

Sure, will do.

StellarBot · 2022-07-18T15:32:39Z

Performance test report

HPX Performance

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR	PARALLEL_EXECUTOR	SCHEDULER_EXECUTOR
For Each	(=)	(=)	(=)

Info

Property	Before	After
HPX Datetime	2022-05-31T12:57:29+00:00	2022-07-18T15:18:52+00:00
HPX Commit	`d5655f8`	`32c7927`
Datetime	2022-05-31T15:13:01.357969+02:00	2022-07-18T17:31:13.698133+02:00
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Hostname	nid01193	nid01193
Clustername	daint	daint
Envfile

Comparison

BENCHMARK	NO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch	=

Info

Property	Before	After
HPX Datetime	2022-05-31T12:57:29+00:00	2022-07-18T15:18:52+00:00
HPX Commit	`d5655f8`	`32c7927`
Datetime	2022-05-31T15:13:18.026239+02:00	2022-07-18T17:31:30.343351+02:00
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Hostname	nid01193	nid01193
Clustername	daint	daint
Envfile

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATOR	PARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATOR	SCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add	(=)	-	(=)
Stream Benchmark - Scale	(=)	(=)	(=)
Stream Benchmark - Triad	(=)	(=)	(=)
Stream Benchmark - Copy	(=)	(=)	(=)

Info

Property	Before	After
HPX Datetime	2021-11-10T19:14:21+00:00	2022-07-18T15:18:52+00:00
HPX Commit	`71d8dbe`	`32c7927`
Datetime	2021-11-10T20:28:18.266961+01:00	2022-07-18T17:31:45.627798+02:00
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Hostname	nid00120	nid01193
Clustername	daint	daint
Envfile

Explanation of Symbols

Symbol	MEANING
=	No performance change (confidence interval within ±1%)
(=)	Probably no performance change (confidence interval within ±2%)
(+)/(-)	Very small performance improvement/degradation (≤1%)
+/-	Small performance improvement/degradation (≤5%)
++/--	Large performance improvement/degradation (≤10%)
+++/---	Very large performance improvement/degradation (>10%)
?	Probably no change, but quite large uncertainty (confidence interval with ±5%)
??	Unclear result, very large uncertainty (±10%)
???	Something unexpected…

StellarBot · 2022-07-18T15:48:00Z

Performance test report

HPX Performance

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR	PARALLEL_EXECUTOR	SCHEDULER_EXECUTOR
For Each	(=)	=	(=)

Info

Property	Before	After
HPX Datetime	2022-05-31T12:57:29+00:00	2022-07-18T15:33:17+00:00
HPX Commit	`d5655f8`	`44ef1d3`
Hostname	nid01193	nid00435
Clustername	daint	daint
Datetime	2022-05-31T15:13:01.357969+02:00	2022-07-18T17:47:08.838396+02:00
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Envfile

Comparison

BENCHMARK	NO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch	=

Info

Property	Before	After
HPX Datetime	2022-05-31T12:57:29+00:00	2022-07-18T15:33:17+00:00
HPX Commit	`d5655f8`	`44ef1d3`
Hostname	nid01193	nid00435
Clustername	daint	daint
Datetime	2022-05-31T15:13:18.026239+02:00	2022-07-18T17:47:25.501653+02:00
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Envfile

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATOR	PARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATOR	SCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add	(=)	(=)	(=)
Stream Benchmark - Scale	(=)	(=)	(=)
Stream Benchmark - Triad	(=)	(=)	(=)
Stream Benchmark - Copy	(=)	-	(=)

Info

Property	Before	After
HPX Datetime	2021-11-10T19:14:21+00:00	2022-07-18T15:33:17+00:00
HPX Commit	`71d8dbe`	`44ef1d3`
Hostname	nid00120	nid00435
Clustername	daint	daint
Datetime	2021-11-10T20:28:18.266961+01:00	2022-07-18T17:47:40.865125+02:00
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Envfile

Explanation of Symbols

Symbol	MEANING
=	No performance change (confidence interval within ±1%)
(=)	Probably no performance change (confidence interval within ±2%)
(+)/(-)	Very small performance improvement/degradation (≤1%)
+/-	Small performance improvement/degradation (≤5%)
++/--	Large performance improvement/degradation (≤10%)
+++/---	Very large performance improvement/degradation (>10%)
?	Probably no change, but quite large uncertainty (confidence interval with ±5%)
??	Unclear result, very large uncertainty (±10%)
???	Something unexpected…

StellarBot · 2022-07-19T17:12:50Z

Performance test report

HPX Performance

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR	PARALLEL_EXECUTOR	SCHEDULER_EXECUTOR
For Each	(=)	(=)	(=)

Info

Property	Before	After
HPX Commit	`d5655f8`	`258527e`
HPX Datetime	2022-05-31T12:57:29+00:00	2022-07-19T16:50:56+00:00
Datetime	2022-05-31T15:13:01.357969+02:00	2022-07-19T19:11:51.673858+02:00
Envfile
Clustername	daint	daint
Hostname	nid01193	nid01323
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0

Comparison

BENCHMARK	NO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch	-

Info

Property	Before	After
HPX Commit	`d5655f8`	`258527e`
HPX Datetime	2022-05-31T12:57:29+00:00	2022-07-19T16:50:56+00:00
Datetime	2022-05-31T15:13:18.026239+02:00	2022-07-19T19:12:08.672710+02:00
Envfile
Clustername	daint	daint
Hostname	nid01193	nid01323
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATOR	PARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATOR	SCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add	(=)	-	(=)
Stream Benchmark - Scale	(=)	(=)	=
Stream Benchmark - Triad	(=)	-	(=)
Stream Benchmark - Copy	(=)	(=)	(=)

Info

Property	Before	After
HPX Commit	`71d8dbe`	`258527e`
HPX Datetime	2021-11-10T19:14:21+00:00	2022-07-19T16:50:56+00:00
Datetime	2021-11-10T20:28:18.266961+01:00	2022-07-19T19:12:24.086403+02:00
Envfile
Clustername	daint	daint
Hostname	nid00120	nid01323
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0

Explanation of Symbols

Symbol	MEANING
=	No performance change (confidence interval within ±1%)
(=)	Probably no performance change (confidence interval within ±2%)
(+)/(-)	Very small performance improvement/degradation (≤1%)
+/-	Small performance improvement/degradation (≤5%)
++/--	Large performance improvement/degradation (≤10%)
+++/---	Very large performance improvement/degradation (>10%)
?	Probably no change, but quite large uncertainty (confidence interval with ±5%)
??	Unclear result, very large uncertainty (±10%)
???	Something unexpected…

StellarBot · 2022-07-31T14:23:36Z

Performance test report

HPX Performance

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR	PARALLEL_EXECUTOR	SCHEDULER_EXECUTOR
For Each	(=)	=	(=)

Info

Property	Before	After
HPX Datetime	2022-05-31T12:57:29+00:00	2022-07-31T14:05:57+00:00
HPX Commit	`d5655f8`	`a866cc4`
Hostname	nid01193	nid01193
Datetime	2022-05-31T15:13:01.357969+02:00	2022-07-31T16:22:36.824275+02:00
Clustername	daint	daint
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Envfile

Comparison

BENCHMARK	NO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch	(=)

Info

Property	Before	After
HPX Datetime	2022-05-31T12:57:29+00:00	2022-07-31T14:05:57+00:00
HPX Commit	`d5655f8`	`a866cc4`
Hostname	nid01193	nid01193
Datetime	2022-05-31T15:13:18.026239+02:00	2022-07-31T16:22:53.800696+02:00
Clustername	daint	daint
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Envfile

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATOR	PARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATOR	SCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add	(=)	-	(=)
Stream Benchmark - Scale	(=)	(=)	(=)
Stream Benchmark - Triad	(=)	-	(=)
Stream Benchmark - Copy	(=)	(=)	(=)

Info

Property	Before	After
HPX Datetime	2021-11-10T19:14:21+00:00	2022-07-31T14:05:57+00:00
HPX Commit	`71d8dbe`	`a866cc4`
Hostname	nid00120	nid01193
Datetime	2021-11-10T20:28:18.266961+01:00	2022-07-31T16:23:09.089371+02:00
Clustername	daint	daint
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Envfile

Explanation of Symbols

Symbol	MEANING
=	No performance change (confidence interval within ±1%)
(=)	Probably no performance change (confidence interval within ±2%)
(+)/(-)	Very small performance improvement/degradation (≤1%)
+/-	Small performance improvement/degradation (≤5%)
++/--	Large performance improvement/degradation (≤10%)
+++/---	Very large performance improvement/degradation (>10%)
?	Probably no change, but quite large uncertainty (confidence interval with ±5%)
??	Unclear result, very large uncertainty (±10%)
???	Something unexpected…

StellarBot · 2022-08-10T18:27:41Z

Performance test report

HPX Performance

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR	PARALLEL_EXECUTOR	SCHEDULER_EXECUTOR
For Each	(=)	(=)	(=)

Info

Property	Before	After
HPX Datetime	2022-05-31T12:57:29+00:00	2022-08-10T18:11:09+00:00
HPX Commit	`d5655f8`	`723fce0`
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Envfile
Hostname	nid01193	nid01159
Datetime	2022-05-31T15:13:01.357969+02:00	2022-08-10T20:26:22.517883+02:00
Clustername	daint	daint

Comparison

BENCHMARK	NO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch	-

Info

Property	Before	After
HPX Datetime	2022-05-31T12:57:29+00:00	2022-08-10T18:11:09+00:00
HPX Commit	`d5655f8`	`723fce0`
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Envfile
Hostname	nid01193	nid01159
Datetime	2022-05-31T15:13:18.026239+02:00	2022-08-10T20:26:39.667478+02:00
Clustername	daint	daint

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATOR	PARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATOR	SCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add	(=)	(=)	(=)
Stream Benchmark - Scale	(=)	(=)	(=)
Stream Benchmark - Triad	(=)	(=)	(=)
Stream Benchmark - Copy	(=)	(=)	(=)

Info

Property	Before	After
HPX Datetime	2021-11-10T19:14:21+00:00	2022-08-10T18:11:09+00:00
HPX Commit	`71d8dbe`	`723fce0`
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0
Envfile
Hostname	nid00120	nid01159
Datetime	2021-11-10T20:28:18.266961+01:00	2022-08-10T20:26:54.843070+02:00
Clustername	daint	daint

Explanation of Symbols

Symbol	MEANING
=	No performance change (confidence interval within ±1%)
(=)	Probably no performance change (confidence interval within ±2%)
(+)/(-)	Very small performance improvement/degradation (≤1%)
+/-	Small performance improvement/degradation (≤5%)
++/--	Large performance improvement/degradation (≤10%)
+++/---	Very large performance improvement/degradation (>10%)
?	Probably no change, but quite large uncertainty (confidence interval with ±5%)
??	Unclear result, very large uncertainty (±10%)
???	Something unexpected…

hkaiser · 2022-08-18T17:55:19Z

@hcq9102 things should be fine now. Please note that I also have rebased this branch onto master.

StellarBot · 2022-08-29T17:16:01Z

Performance test report

HPX Performance

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR	PARALLEL_EXECUTOR	SCHEDULER_EXECUTOR
For Each	(=)	(=)	(=)

Info

Property	Before	After
HPX Datetime	2022-05-31T12:57:29+00:00	2022-08-29T16:20:27+00:00
HPX Commit	`d5655f8`	`4830586`
Datetime	2022-05-31T15:13:01.357969+02:00	2022-08-29T19:15:04.416369+02:00
Hostname	nid01193	nid00013
Envfile
Clustername	daint	daint
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0

Comparison

BENCHMARK	NO-EXECUTOR
Future Overhead - Create Thread Hierarchical - Latch	-

Info

Property	Before	After
HPX Datetime	2022-05-31T12:57:29+00:00	2022-08-29T16:20:27+00:00
HPX Commit	`d5655f8`	`4830586`
Datetime	2022-05-31T15:13:18.026239+02:00	2022-08-29T19:15:21.732327+02:00
Hostname	nid01193	nid00013
Envfile
Clustername	daint	daint
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0

Comparison

BENCHMARK	FORK_JOIN_EXECUTOR_DEFAULT_FORK_JOIN_POLICY_ALLOCATOR	PARALLEL_EXECUTOR_DEFAULT_PARALLEL_POLICY_ALLOCATOR	SCHEDULER_EXECUTOR_DEFAULT_SCHEDULER_EXECUTOR_ALLOCATOR
Stream Benchmark - Add	(=)	(=)	(=)
Stream Benchmark - Scale	(=)	(=)	(=)
Stream Benchmark - Triad	(=)	(=)	(=)
Stream Benchmark - Copy	(=)	(=)	(=)

Info

Property	Before	After
HPX Datetime	2021-11-10T19:14:21+00:00	2022-08-29T16:20:27+00:00
HPX Commit	`71d8dbe`	`4830586`
Datetime	2021-11-10T20:28:18.266961+01:00	2022-08-29T19:15:37.032623+02:00
Hostname	nid00120	nid00013
Envfile
Clustername	daint	daint
Compiler	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0	/apps/daint/SSL/HPX/packages/llvm-11.0.0/bin/clang++ 11.0.0

Explanation of Symbols

Symbol	MEANING
=	No performance change (confidence interval within ±1%)
(=)	Probably no performance change (confidence interval within ±2%)
(+)/(-)	Very small performance improvement/degradation (≤1%)
+/-	Small performance improvement/degradation (≤5%)
++/--	Large performance improvement/degradation (≤10%)
+++/---	Very large performance improvement/degradation (>10%)
?	Probably no change, but quite large uncertainty (confidence interval with ±5%)
??	Unclear result, very large uncertainty (±10%)
???	Something unexpected…

hkaiser · 2022-10-05T18:55:36Z

This can be closed as it is subsumed by #6015

hcq9102 · 2022-10-05T18:55:55Z

#6015 got merged. We don't need this one anymore.(sync_wait_with_variant integrating with sync_wait)

hcq9102 marked this pull request as ready for review July 17, 2022 14:09

hcq9102 requested review from aurianer, biddisco, hkaiser and msimberg as code owners July 17, 2022 14:09

hkaiser added type: enhancement type: compatibility issue category: senders/receivers Implementations of the p0443r14 / p2300 + p1897 proposals labels Jul 17, 2022

hkaiser added this to the 1.9.0 milestone Jul 17, 2022

hkaiser requested changes Jul 17, 2022

View reviewed changes

hcq9102 force-pushed the sync_wait_variant branch from 46b08aa to a21b560 Compare July 18, 2022 15:33

hcq9102 force-pushed the sync_wait_variant branch 5 times, most recently from 13b84ab to 722ec9a Compare July 19, 2022 15:25

hcq9102 force-pushed the sync_wait_variant branch 9 times, most recently from 7253a3c to dd3af2d Compare July 22, 2022 15:14

hcq9102 force-pushed the sync_wait_variant branch 2 times, most recently from 32b50fe to 3ea62d8 Compare July 31, 2022 14:05

hcq9102 force-pushed the sync_wait_variant branch 6 times, most recently from 2248cad to c7e9ba4 Compare August 5, 2022 05:28

hcq9102 force-pushed the sync_wait_variant branch 2 times, most recently from 7a3e9c3 to 5663170 Compare August 10, 2022 18:11

hcq9102 force-pushed the sync_wait_variant branch 2 times, most recently from 88d09a7 to 210f1c9 Compare August 12, 2022 16:08

hkaiser force-pushed the sync_wait_variant branch from 210f1c9 to 097bf5f Compare August 18, 2022 17:54

aurianer force-pushed the master branch from 0b1a7e3 to 92ab22e Compare August 27, 2022 15:56

hcq9102 and others added 6 commits August 29, 2022 11:14

adding sync_wait_with_variant

de9c83f

CMakeLists update

e5af7c9

fix test

b391dc3

adding multi_tuple test

83fa674

custom sender with dif value_type_tuple

436c8e8

Fixing test

e2079fa

hkaiser force-pushed the sync_wait_variant branch from 097bf5f to e2079fa Compare August 29, 2022 16:20

hkaiser mentioned this pull request Sep 21, 2022

Integrating sync_wait & sync_wait_with_variant #6015

Merged

hkaiser closed this Oct 5, 2022

gonidelis removed this from the 1.9.0 milestone Feb 1, 2023

Adding sync_wait_with_variant #5956

Adding sync_wait_with_variant #5956

Conversation

hcq9102 commented Jul 17, 2022 • edited Loading

Any background context you want to provide?

Checklist

hkaiser left a comment

Choose a reason for hiding this comment

hcq9102 commented Jul 17, 2022

StellarBot commented Jul 18, 2022

HPX Performance

Comparison

Info

Comparison

Info

Comparison

Info

Explanation of Symbols

StellarBot commented Jul 18, 2022

HPX Performance

Comparison

Info

Comparison

Info

Comparison

Info

Explanation of Symbols

StellarBot commented Jul 19, 2022

HPX Performance

Comparison

Info

Comparison

Info

Comparison

Info

Explanation of Symbols

StellarBot commented Jul 31, 2022

HPX Performance

Comparison

Info

Comparison

Info

Comparison

Info

Explanation of Symbols

StellarBot commented Aug 10, 2022

HPX Performance

Comparison

Info

Comparison

Info

Comparison

Info

Explanation of Symbols

hkaiser commented Aug 18, 2022

StellarBot commented Aug 29, 2022

HPX Performance

Comparison

Info

Comparison

Info

Comparison

Info

Explanation of Symbols

hkaiser commented Oct 5, 2022

hcq9102 commented Oct 5, 2022

hcq9102 commented Jul 17, 2022 •

edited

Loading