EnC: Allow adding/removing await expressions and yield statements

[tmat]

Adds support to the compiler to emit changes to async methods and iterators that add, remove or modify await expressions and yield statements.

When such change is made we need to make sure that existing instances of the state machine whose methods are being updated resume in a correct state. We can't easily find these instances on heap and update their state field. We can however make sure that when such state machine returns to the new version of its MoveNext method the state number that is stored in its state field dispatches to the right label.

We do so by mapping syntax of each node that is associated with a state machine state (i.e. await expression, yield return statement) to its previous version. This mapping is already established by the IDE syntax diffing algorithm and is available to the compiler via EncVariableSlotAllocator. In addition, we need to store mapping of these syntax nodes to the state numbers to the PDB. We add a new record EditAndContinueStateMachineStateMap for that purpose. This record encodes map from syntax offset to state number.

The end-to-end mapping logic then maps syntax node in the new source to its corresponding syntax node in the old source code and then the syntax offset of the old node to the old state number. If the corresponding old syntax node is not found that means a new await/yield return is being added and thus a new state number is assigned that is greater than all previously assigned state numbers (including those assigned in the previous generations). Once all states of the new state machine are determined but before we generate the dispatch switch we add cases for all deleted states - i.e. states that were present in the previous state machine but were not matches with the new one. If the state machine resumes in one of these states we let it throw InvalidOperationException since we can't tell where exactly it should resume execution. Guessing would likely result in unexpected behavior, e.g. execution of code that has already been executed or skipping code that should execute.

EditAndContinueStateMachineStateMap also contains mapping for finalization states of iterator state machine that are used to invoke finally blocks around yield returns in its Dispose method. Although no state machine instance can be suspended and resumed in these states, we need to perform the same mapping (this time with try statements) in order to map <>__FinallyN methods synthesized for the new state machine to the ones synthesized for the previous one.

TODO as follow-up:

• Support for VB - this PR adds basic infra but VB iterators are different from C# and need to make more changes, which are probably better in a separate PR.
• IDE support - this PR does not enable the feature in the IDE yet.

Implements #54095

[tmat]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 4 pipeline(s).

[tmat]

@davidwengier PTAL

[davidwengier]

Concept makes sense, and code looked okay, as much as I could understand. Pity I can't read IL. Surprised that some debugging scenarios don't break with out-of-order state machine states.

[tmat]

@dotnet/roslyn-compiler PTAL

[jaredpar]

@cston, @jcouv PTAL

[cston]

IL_0012: ble.un.s IL_0016

What does this represent? I wasn't expecting to see any changes in Dispose().

---

In reply to: 1140294434

---

Refers to: src/Compilers/CSharp/Test/Emit/Emit/EditAndContinue/EditAndContinueStateMachineTests.cs:4825 in c58cbe1. [](commit_id = c58cbe1, deletion_comment = False)

[tmat]

IL_0012: ble.un.s IL_0016

Yield was added within scope of the using, so one more state needs to be dispatched to the finally block. The difference is between state == 1 and state <= 2.

---

In reply to: 1140294434

---

Refers to: src/Compilers/CSharp/Test/Emit/Emit/EditAndContinue/EditAndContinueStateMachineTests.cs:4825 in c58cbe1. [](commit_id = c58cbe1, deletion_comment = False)

[jcouv]

nit: consider asserting !debugInfo.StateMachineStates.IsDefaultOrEmpty, possibly with comment

[jcouv]

LGTM Thanks (iteration 14)