Dispose parameters, kill benchmarking process when it hangs after printing the results #1571
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#1383 contained a very unusual bug: benchmarks were executing fine, but hanging after printing "// AfterAll" to the console
I've dug into that and found out (#1383 (comment)) that there is a locking finalizer which prevents the process from exiting which was a thing until it got fixed in .NET 5 (dotnet/runtime#314)
The problem was that disposing the finalizable instance in the
[GlobalCleanup]
was not enough: https://github.com/JohnMasen/VirusSimulator/blob/432e185866b80aac69b95abbeb00d1bb2e3735c1/src/ComputeShaderTest/ComputeShaderPerformanceILGPU.cs#L63For two reasons:
[ParamsSource]
or[ArgumentsSource]
the boilerplate code was so far calling.ToArray()[index]
on it, whereindex
was the index of the test case. It meant that if the benchmark had 3 test cases, we were always creating 3 of them in the benchmarking process. The solution was to introduce a new methodParameterExtractor.GetParameter
that enumerates over the arguments source and disposes unused arguments. It also does not create more than needed. So if we want just the first argument, it stops iterating after first occurrence and hence creates only one instance of the given type.BenchmarkConverter
is also materializing all objects from[ParamsSource]
or[ArgumentsSource]
to be able to run the benchmarks. The solution was to add aDispose
after running the benchmarks. Thanks to that, the host process stopped hanging after printing// * Artifacts cleanup *
at the endLast but not least, the fix was specific to finalizable arguments. To make it more robust, I've extended the executors with the following logic:
AfterAll
) we stop processing the output stream of the benchmark process. we know it has finished, there is no need to try to read the stream (and hung while trying to do that).BenchmarkDotNet/src/BenchmarkDotNet/Templates/BenchmarkProgram.txt
Line 75 in 0928895
I know that the changes are quite complex, but I have tested them very carefully and ensured that with this extra logic it will be now much harder to hang BenchmarkDotNet.
Fixes #1383
Fixes #828