Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Put back the old QuickSort and PartialQuickSort algorithms #47788

Merged
merged 2 commits into from
Dec 20, 2022

Conversation

LilithHafner
Copy link
Member

@LilithHafner LilithHafner commented Dec 3, 2022

Put back the old QuickSort and PartialQuickSort, and MergeSort algorithms, rename the new PartialQuickSort to QuickerSort, improve the documentation and API for constructing QuickerSort, and test the QuickerSort API.

Most of the additions are copied verbatim from 05cfe24, the commit before #45222 which removed the old implementations of QuickSort and PartialQuickSort.

I'm open to bikesheding on the name QuickerSort. Names I already considered:

StableQuickSort — this algorithm can be stable, but it takes a bit of extra work (which we currently always perform). In future versions of julia, perhaps sort will take a stable keyword, and if stable is false, then this algorithm need not perform the cheap-but-not-free reversals which stabilize it. I don't like sort(v, alg=StableQuickSort, stable=false).

OutOfPlaceQuickSort — this is long and fails to represent the fact that the algorithm it refers to is typically faster than quicksort.

Fixes #47715
Fixes #47766
Fixes #47304 by reverting the expanded functionality that would need a compat entry (QuickerSort is still technically internal)
Closes #47297 because QuickerSort can use a fresh API with no need to deprecate any PartialQuickSort methods.

…s...

...as they were in 1.8 and rename the new PartialQuickSort to QuickerSort
Also improve the documentation and API for constructing QuickerSort and test
the API
@LilithHafner LilithHafner added performance Must go faster sorting Put things in order backport 1.9 Change should be backported to release-1.9 labels Dec 3, 2022
@LilithHafner
Copy link
Member Author

@nanosoldier runbenchmarks("sort")

@nanosoldier
Copy link
Collaborator

Your job failed.

@LilithHafner
Copy link
Member Author

Strange.
@nanosoldier runbenchmarks(["sort"])

@LilithHafner
Copy link
Member Author

The nanosoldier issue is not blocking.

@LilithHafner
Copy link
Member Author

Local perf tests demonstrate that this restores previous allocation levels:

julia> x = rand(Int, 10000);

julia> @btime sort!(rand!($x));
  233.624 μs (3 allocations: 86.36 KiB)

julia> @btime sort!(rand!($x); alg=QuickSort);
  419.169 μs (0 allocations: 0 bytes)

julia> @btime sort!(rand!($x); alg=PartialQuickSort(1:3));
  7.815 μs (0 allocations: 0 bytes)

@maleadt
Copy link
Member

maleadt commented Dec 3, 2022

@vtjnash probably hasn't updated the benchmark bot to the latest Nanosoldier.jl, so it might still require the :vs keyword:

@nanosoldier runbenchmarks("sort", vs = ":master")

@nanosoldier
Copy link
Collaborator

Your job failed.

@maleadt
Copy link
Member

maleadt commented Dec 3, 2022

Your job failed.

@vtjnash will have to look at the server logs for this one.

@vtjnash
Copy link
Member

vtjnash commented Dec 4, 2022

I don't understand why it failed to upload the logs for this as it should have

      From worker 3:    2022-12-03T05:26:16.573 | starting benchscript.jl (STDOUT/STDERR will be redirected to the result folder)
┌ Info: [Node 3 | 2022-12-03T05:28:00.583]: failed job: BenchmarkJob JuliaLang/julia@cc25f42 vs. JuliaLang/julia@cee0a04
│ On worker 3:                                                                                       
│ failed process: Process(`sudo -n -- /nanosoldier/cset/bin/cset shield -e -- sudo -n -u nanosoldier-worker -- /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_lN3A
c3/benchscript.sh`, ProcessSignaled(11)) [0]                                                                                                                                                              

the log content is

[761710] signal (11.1): Segmentation fault
in expression starting at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_lN3Ac3/benchscript.jl:18
jl_gc_pool_alloc_inner at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/gc.c:1332 [inlined]
jl_gc_pool_alloc_noinline at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/gc.c:1385
jl_gc_alloc_ at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/julia_internal.h:460 [inlined]
jl_new_uninitialized_datatype at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/datatype.c:98
inst_datatype_inner at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/jltypes.c:1525
jl_inst_arg_tuple_type at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/jltypes.c:1665
arg_type_tuple at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/gf.c:1965 [inlined]
jl_lookup_generic_ at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/gf.c:2650 [inlined]
ijl_apply_generic at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/gf.c:2702
getindex at ./range.jl:419
jfptr_getindex_26325 at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/usr/lib/julia/sys.so (unknown line)
_jl_invoke at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/gf.c:2524 [inlined]
ijl_apply_generic at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/gf.c:2706
#_run#48 at /home/nanosoldier-worker/.julia/packages/BenchmarkTools/0owsb/src/execution.jl:100
kwcall at /home/nanosoldier-worker/.julia/packages/BenchmarkTools/0owsb/src/execution.jl:92
unknown function (ip: 0x7f87d055934d)
_jl_invoke at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/gf.c:2524 [inlined]
ijl_apply_generic at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/gf.c:2706
jl_apply at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/julia.h:1876 [inlined]
jl_f__call_latest at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/builtins.c:778
#invokelatest#2 at ./essentials.jl:818 [inlined]
kwcall at ./essentials.jl:813 [inlined]
#run_result#45 at /home/nanosoldier-worker/.julia/packages/BenchmarkTools/0owsb/src/execution.jl:34 [inlined]
kwcall at /home/nanosoldier-worker/.julia/packages/BenchmarkTools/0owsb/src/execution.jl:34 [inlined]
#run#49 at /home/nanosoldier-worker/.julia/packages/BenchmarkTools/0owsb/src/execution.jl:117
kwcall at /home/nanosoldier-worker/.julia/packages/BenchmarkTools/0owsb/src/execution.jl:117 [inlined]
kwcall at /home/nanosoldier-worker/.julia/packages/BenchmarkTools/0owsb/src/execution.jl:117 [inlined]
macro expansion at /home/nanosoldier-worker/.julia/packages/BenchmarkTools/0owsb/src/execution.jl:135 [inlined]
macro expansion at ./timing.jl:393 [inlined]
#51 at /home/nanosoldier-worker/.julia/packages/BenchmarkTools/0owsb/src/execution.jl:134
#_withprogress#47 at /home/nanosoldier-worker/.julia/packages/BenchmarkTools/0owsb/src/execution.jl:73
unknown function (ip: 0x7f87d0558477)
_jl_invoke at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/gf.c:2524 [inlined]
ijl_apply_generic at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/gf.c:2706
kwcall at /home/nanosoldier-worker/.julia/packages/BenchmarkTools/0owsb/src/execution.jl:63
unknown function (ip: 0x7f87d0555e52)
#run#50 at /home/nanosoldier-worker/.julia/packages/BenchmarkTools/0owsb/src/execution.jl:125
kwcall at /home/nanosoldier-worker/.julia/packages/BenchmarkTools/0owsb/src/execution.jl:125
macro expansion at /home/nanosoldier-worker/.julia/packages/BenchmarkTools/0owsb/src/execution.jl:135 [inlined]
macro expansion at ./timing.jl:393 [inlined]
#51 at /home/nanosoldier-worker/.julia/packages/BenchmarkTools/0owsb/src/execution.jl:134
#_withprogress#47 at /home/nanosoldier-worker/.julia/packages/BenchmarkTools/0owsb/src/execution.jl:73
unknown function (ip: 0x7f87d0558477)
_jl_invoke at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/gf.c:2524 [inlined]
ijl_apply_generic at /home/nanosoldier/.julia/scratchspaces/89f34f1a-2e6b-52eb-a20f-77051b03b735/workdir/jl_yTI0w1/julia/src/gf.c:2706

@LilithHafner
Copy link
Member Author

@nanosoldier runbenchmarks("sort")

@nanosoldier
Copy link
Collaborator

Your benchmark job has completed - successfully executed benchmarks. A full report can be found here.

@LilithHafner
Copy link
Member Author

@nanosoldier runbenchmarks("sort", vs=":master")

@nanosoldier
Copy link
Collaborator

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here.

@LilithHafner
Copy link
Member Author

This restores the previous performance of QuickSort; the fact that it is typically an improvement is disturbing and likely connected to #47383 (comment)

@KristofferC KristofferC merged commit 8cdb17b into master Dec 20, 2022
@KristofferC KristofferC deleted the put-back-original-quicksort branch December 20, 2022 16:07
KristofferC pushed a commit that referenced this pull request Dec 20, 2022
…s... (#47788)

...as they were in 1.8 and rename the new PartialQuickSort to QuickerSort
Also improve the documentation and API for constructing QuickerSort and test
the API

Co-authored-by: Lilith Hafner <Lilith.Hafner@gmail.com>
(cherry picked from commit 8cdb17b)
@KristofferC KristofferC removed the backport 1.9 Change should be backported to release-1.9 label Dec 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Must go faster sorting Put things in order
Projects
None yet
5 participants