Fix 'InvalidOperationException' with message Collection was modified; enumeration operation may not execute in MultithreadedAnalyzeCommandBase, which is raised when analyzing with the --hashes switch.

[eddynaka]

Description

BinSkim uses the MulthtireadedAnalyzeCommandBase and it was throwing InvalidOperationException with the following stack:

System.InvalidOperationException with message "Collection was modified; enumeration operation may not execute."
         at System.Collections.Generic.Dictionary`2.Enumerator.MoveNext()
         at Microsoft.CodeAnalysis.Sarif.Driver.MultithreadedAnalyzeCommandBase`2.LogCachingLogger(TContext rootContext, IAnalysisLogger logger, TContext context, Boolean clone) in C:\Microsoft\sarif-sdk\src\Sarif.Driver\Sdk\MultithreadedAnalyzeCommandBase.cs:line 311
         at Microsoft.CodeAnalysis.Sarif.Driver.MultithreadedAnalyzeCommandBase`2.WriteResultsAsync(TContext rootContext) in C:\Microsoft\sarif-sdk\src\Sarif.Driver\Sdk\MultithreadedAnalyzeCommandBase.cs:line 272

This happens when multiple files are being analyzed and they share the same context (the files have the same hash). The first thread finishes and starts to write, and the second thread is still writing in the same context, generating the exception.

The following issues were created:

• Add tests for AnalyzeCommandBase and MultithtereadAnalyzeCommandBase passing absolute/relative paths #2456
• Update HashUtilities to receive IFileSystem in all methods that works with files #2455

[michaelcfanning]

If you pursue this change, please be sure to explore a test approach for the problem. Embedding a stress harness at unit-test time isn't a very reliable approach and can slow down testing. One idea is to either test or build into the product itself some enforcement of whatever invariant is being broken here.

What I think is happening based on your analysis is that someone is writing to a caching logger after another thread thinks an analysis is finished and it can retrieve the results list. And so, what we could try is to alter the caching logger to put some sort of block between these phases. Think about it. :) Remember, we have the analysis finished event that should tell us when we think we're done.

But the trickiness here is that our logger is handing back a results object that is a mutable IList instance. What if, after receiving the analysis completed event, we converted this data to an immutable list?

As you consider that approach, also consider other possible threading issues in this class. One trick here is to retain data that points to the single thread that we think has permission to mutate the results collection. And to raise an exception in cases where someone appears to be attempting a mutating operation on an incorrect thread. This won't help us fix anything in itself but it should help use raise failures much earlier in the process (even in cases where the broken thread access doesn't cause an apparent problem/exception today). This approach also makes our stress more efficient, again because it forces earlier manifestation of problems. All of this commentary, however, hinges on providing a different mutation/access strategy for the results collection. Today, we just hand off the list of results with no good sense of what happens next.

[michaelcfanning]

expectedCacheSize

Can you explain what you learned here?

[michaelcfanning]

specifier

Always writeline the random seed (in a test that uses the random instance) on entrance of the test. See other tests for a nice reporting pattern. #Resolved

[michaelcfanning]

InvalidOperationException

Always report the exception type, the exception message and where the exception source.

Fix 'InvalidOperationException' with message Collection was modified; enumeration operation may not execute in MultithreadedAnalyzeCommandBase, which is raised when analyzing with the --hashes switch.

#Resolved

[michaelcfanning]