Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problems
Cause
We appear to have a case of over-parallelization.
RelationshipResolver.ModList
is parallel (for speeding up very large changesets), andVersions.checkInstallable
calls it in parallel (viaModuleInstaller.CanInstall
; that's what feeds theToList
call highlighted above).When parallel calls are nested, I think the outer one sets up some thread pooling and marshalling and so on to distribute load to your N CPUs, and then the inner calls each do the same thing, so you effectively have threads competing for N*N CPUs, which is N*(N-1) more than you have.
Changes
EnumerableExtensions.AsParallelIf
is added to create a way to make a PLINQ call parallel only if its parameter is trueRelationshipResolver.ModList
has aparallel
parameter (defaulting to true to keep it parallel for the situations where it's helpful) and passes it toAsParallelIf
ModuleInstaller.CanInstall
tellsModList
not to load in parallelThis follows the recommendation of the above link to keep the outer loop parallel while making the inner loop non-parallel, and it makes the Versions tab load much quicker now.
It's unfortunate that we have to hard-code whether a specific call should be parallel instead of just writing optimal queries and letting the framework sort out the parallelization, but that's where we are. Ideally we would create our own way to toggle it automatically, but this will work for now, and we can revisit it later if we think of a better solution.
Side changes:
CkanModule.FromJson
andCkanModule
's constructor now initialize the_comparator
field in a simpler wayBeginUpdate
/EndUpdate
to minimize flickering of the listview while setting multiple row colors in a rowFixes #4046.