Clarifies that processProposal may be called for set of transactions …

…different from the one returned in the preceding prepareProposal (cometbft#1033)

If a proposer fails after calling prepareProposal and before calling processProposal, then the following may happen upon restarting:
- if failed before signing another message, then will invoke prepareProposal again, sign a new block, probably empty, and propose it;
- if failed after signing a proposal but before writing the  proposal message into the WAL, then will invoke prepareProposal and produce a new, probably empty block, [fail to sign it](https://github.com/cometbft/cometbft/blob/2789a59a9cc61c6ea56a6b266eeadf0f26ca2456/consensus/state.go#L1221), and not invoke processProposal; prevote timeouts will ensure the CometBFT is not stuck;
- if failed after writing the proposal message to the WAL, then will invoke prepareProposal, produce a new, probably empty block, fail to sign it, and invoke processProposal with the block signed before crashing.

---

#### PR checklist

- [ ] Tests written/updated
- [ ] Changelog entry added in `.changelog` (we use [unclog](https://github.com/informalsystems/unclog) to manage our changelog)
- [x] Updated relevant documentation (`docs/` or `spec/`) and code comments

spec/abci/abci++_basic_concepts.md

@@ -24,7 +24,7 @@ title: Overview and basic concepts
 
 # Overview and basic concepts
 
-## ABCI++ vs. ABCI
+## ABCI 2.0 vs. ABCI
 
 [↑ Back to Outline](#outline)
 
@@ -40,18 +40,18 @@ as the Application cannot require validators to do more than executing the trans
 finalized blocks. This includes features such as threshold cryptography, and guaranteed IBC
 connection attempts.
 
-ABCI++ addresses these limitations by allowing the application to intervene at three key places of
+ABCI 2.0 addresses these limitations by allowing the application to intervene at three key places of
 consensus execution: (a) at the moment a new proposal is to be created, (b) at the moment a
-proposal is to be validated, and (c) at the moment a (precommit) vote is sent/received. 
+proposal is to be validated, and (c) at the moment a (precommit) vote is sent/received.
 The new interface allows block proposers to perform application-dependent
 work in a block through the `PrepareProposal` method (a); and validators to perform application-dependent work
-and checks in a proposed block through the `ProcessProposal` method (b); and applications to require their validators 
-to do more than just validate blocks through the `ExtendVote` and `VerifyVoteExtensions` methods (c). 
+and checks in a proposed block through the `ProcessProposal` method (b); and applications to require their validators
+to do more than just validate blocks through the `ExtendVote` and `VerifyVoteExtensions` methods (c).
 
 Furthermore, ABCI 2.0 coalesces {`BeginBlock`, [`DeliverTx`], `EndBlock`} into `FinalizeBlock`, as a
 simplified, efficient way to deliver a decided block to the Application.
 
-## Method overview
+## Methods overview
 
 
 [↑ Back to Outline](#outline)
@@ -60,7 +60,7 @@ Methods can be classified into four categories: *consensus*, *mempool*, *info*,
 
 ### Consensus/block execution methods
 
-The first time a new blockchain is started, CometBFT calls `InitChain`. From then on, method 
+The first time a new blockchain is started, CometBFT calls `InitChain`. From then on, method
 `FinalizeBlock` is executed upon the decision of each block, resulting in an updated Application
 state. During the execution of an instance of consensus, which decides the block for a given
 height, and before method `FinalizeBlock` is called, methods `PrepareProposal`, `ProcessProposal`,
@@ -82,29 +82,32 @@ call sequences of these methods.
   can make changes to the raw proposal, such as modifying the set of transactions or the order
   in which they appear, and returns the
   (potentially) modified proposal, called *prepared proposal* in the `ResponsePrepareProposal`
-  call. The logic modifying the raw proposal can be non-deterministic.
+  call.
+  The logic modifying the raw proposal MAY be non-deterministic.
 
 - [**ProcessProposal:**](./abci++_methods.md#processproposal) It allows a validator to
   perform application-dependent work in a proposed block. This enables features such as immediate
   block execution, and allows the Application to reject invalid blocks.
 
   CometBFT calls it when it receives a proposal and _validValue_ is `nil`.
-  The Application cannot modify the proposal at this point but can reject it if it is
+  The Application cannot modify the proposal at this point but can reject it if
   invalid. If that is the case, the consensus algorithm will prevote `nil` on the proposal, which has
   strong liveness implications for CometBFT. As a general rule, the Application
   SHOULD accept a prepared proposal passed via `ProcessProposal`, even if a part of
   the proposal is invalid (e.g., an invalid transaction); the Application can
   ignore the invalid part of the prepared proposal at block execution time.
+  The logic in `ProcessProposal` MUST be deterministic.
 
-- [**ExtendVote:**](./abci++_methods.md#extendvote) It allows applications to force their
-  validators to do more than just validate within consensus. `ExtendVote` allows applications to
+- [**ExtendVote:**](./abci++_methods.md#extendvote) It allows applications to let their
+  validators do more than just validate within consensus. `ExtendVote` allows applications to
   include non-deterministic data, opaque to the consensus algorithm, to precommit messages (the final round of
   voting). The data, called *vote extension*, will be broadcast and received together with the
   vote it is extending, and will be made available to the Application in the next height,
   in the rounds where the local process is the proposer.
   CometBFT calls `ExtendVote` when the consensus algorithm is about to send a non-`nil` precommit message.
   If the Application does not have vote extension information to provide at that time, it returns
   a 0-length byte array as its vote extension.
+  The logic in `ExtendVote` MAY be non-deterministic.
 
 - [**VerifyVoteExtension:**](./abci++_methods.md#verifyvoteextension) It allows
   validators to validate the vote extension data attached to a precommit message. If the validation
@@ -116,6 +119,7 @@ call sequences of these methods.
   As a general rule, an Application that detects an invalid vote extension SHOULD
   accept it in `ResponseVerifyVoteExtension` and ignore it in its own logic. CometBFT calls it when
   a process receives a precommit message with a (possibly empty) vote extension.
+  The logic in `VerifyVoteExtension` MUST be deterministic.
 
 - [**FinalizeBlock:**](./abci++_methods.md#finalizeblock) It delivers a decided block to the
   Application. The Application must execute the transactions in the block deterministically and
@@ -202,23 +206,26 @@ More details on managing state across connections can be found in the section on
 
 ## Proposal timeout
 
-Immediate execution requires the Application to fully execute the prepared block
-before returning from `PrepareProposal`, this means that CometBFT cannot make progress
-during the block execution.
-This stands on the consensus algorithm critical path: if the Application takes a long time
-executing the block, the default value of *TimeoutPropose* might not be sufficient
-to accommodate the long block execution time and non-proposer nodes might time
-out and prevote `nil`. The proposal, in this case, will probably be rejected and a new round will be necessary.
+`PrepareProposal` stands on the consensus algorithm critical path,
+i.e., CometBFT cannot make progress while this method is being executed.
+Hence, if the Application takes a long time preparing a proposal,
+the default value of *TimeoutPropose* might not be sufficient
+to accommodate the method's execution and validator nodes might time out and prevote `nil`.
+The proposal, in this case, will probably be rejected and a new round will be necessary.
 
-
-Operators will need to adjust the default value of *TimeoutPropose* in CometBFT's configuration file,
+Timeouts are automatically increased for each new round of a height and, if the execution of `PrepareProposal` is bound, eventually *TimeoutPropose*  will be long enough to accommodate the execution of `PrepareProposal`.
+However, relying on this self adaptation could lead to performance degradation and, therefore,
+operators are suggested to adjust the initial value of *TimeoutPropose* in CometBFT's configuration file,
 in order to suit the needs of the particular application being deployed.
 
+This is particularly important if applications implement *immediate execution*.
+To implement this technique, proposers need to execute the block being proposed within `PrepareProposal`, which could take longer than *TimeoutPropose*.
+
 ## Deterministic State-Machine Replication
 
 [↑ Back to Outline](#outline)
 
-ABCI++ applications must implement deterministic finite-state machines to be
+ABCI applications must implement deterministic finite-state machines to be
 securely replicated by the CometBFT consensus engine. This means block execution
 must be strictly deterministic: given the same
 ordered set of transactions, all nodes will compute identical responses, for all
@@ -233,11 +240,13 @@ from block execution (`FinalizeBlock` calls), and not through
 any other kind of request. This is the only way to ensure all nodes see the same
 transactions and compute the same results.
 
-Some Applications may choose to implement immediate execution, which entails executing the blocks
-that are about to be proposed (via `PrepareProposal`), and those that the Application is asked to
-validate (via `ProcessProposal`). However, the state changes caused by processing those
+Applications that implement immediate execution (execute the blocks
+that are about to be proposed, in `PrepareProposal`, or that require validation, in `ProcessProposal`) produce a new candidate state before a block is decided.
+The state changes caused by processing those
 proposed blocks must never replace the previous state until `FinalizeBlock` confirms
-the block decided.
+that the proposed block was decided and `Commit` is invoked for it.
+
+The same is true to Applications that quickly accept blocks and execute the blocks optimistically in parallel with the remaining consensus steps to save time during `FinalizeBlock`; they must only apply state changes in `Commit`.
 
 Additionally, vote extensions or the validation thereof (via `ExtendVote` or
 `VerifyVoteExtension`) must *never* have side effects on the current state.
@@ -278,7 +287,7 @@ on them. All other fields in the `Response*` must be strictly deterministic.
 
 Method `FinalizeBlock` includes an `events` field at the top level in its
 `Response*`, and one `events` field per transaction included in the block.
-Applications may respond to this ABCI++ method with an event list for each executed
+Applications may respond to this ABCI 2.0 method with an event list for each executed
 transaction, and a general event list for the block itself.
 Events allow applications to associate metadata with transactions and blocks.
 Events returned via `FinalizeBlock` do not impact the consensus algorithm in any way

spec/abci/abci++_comet_expected_behavior.md

@@ -39,7 +39,7 @@ Application design should consider _any_ of these possible sequences.
 
 The following grammar, written in case-sensitive Augmented Backus–Naur form (ABNF, specified
 in [IETF rfc7405](https://datatracker.ietf.org/doc/html/rfc7405)), specifies all possible
-sequences of calls to ABCI++, taken by a correct process, across all heights from the genesis block,
+sequences of calls to ABCI++, taken by a **correct process**, across all heights from the genesis block,
 including recovery runs, from the point of view of the Application.
 
 ```abnf
@@ -56,7 +56,7 @@ consensus-exec      = (inf)consensus-height
 consensus-height    = *consensus-round decide commit
 consensus-round     = proposer / non-proposer
 
-proposer            = *got-vote [prepare-proposal process-proposal] [extend]
+proposer            = *got-vote [prepare-proposal [process-proposal]] [extend]
 extend              = *got-vote extend-vote *got-vote
 non-proposer        = *got-vote [process-proposal] [extend]
 
@@ -130,7 +130,8 @@ Let us now examine the grammar line by line, providing further details.
 >```
 
 * In recovery mode, CometBFT first calls `Info` to know from which height it needs to replay decisions
-  to the Application. After this, CometBFT enters normal consensus execution.
+  to the Application. After this, CometBFT enters consensus execution, first in replay mode and then
+  in normal mode.
 
 >```abnf
 >recovery            = info consensus-exec
@@ -155,17 +156,26 @@ Let us now examine the grammar line by line, providing further details.
 >consensus-round     = proposer / non-proposer
 >```
 
-* For every round, if the local process is the proposer of the current round, CometBFT calls `PrepareProposal`, followed by `ProcessProposal`. 
-These two always come together because they reflect the same proposal that the process
-also delivers to itself. 
+* For every round, if the local process is the proposer of the current round, CometBFT calls `PrepareProposal`.
+  A successful execution of `PrepareProposal` implies in a proposal block being (i)signed and (ii)stored
+  (e.g., in stable storage).
+
+  A crash during this step will direct how the node proceeds the next time it is executed, for the same round, after restarted.
+  If it crashed before (i), then, during the recovery, `PrepareProposal` will execute as if for the first time.
+  Following a crash between (i) and (ii) and in (the likely) case `PrepareProposal` produces a different block,
+  the signing of this block will fail, which means that the new block will not be stored or broadcast.
+  If the crash happened after (ii), then signing fails but nothing happens to the stored block.
+
+  If a block was stored, it is sent to all validators, including the proposer.
+  Receiving a proposal block triggers `ProcessProposal` with such a block.
 
   Then, optionally, the Application is
   asked to extend its vote for that round. Calls to `VerifyVoteExtension` can come at any time: the
   local process may be slightly late in the current round, or votes may come from a future round
   of this height.
 
 >```abnf
->proposer            = *got-vote [prepare-proposal process-proposal] [extend]
+>proposer            = *got-vote [prepare-proposal [process-proposal]] [extend]
 >extend              = *got-vote extend-vote *got-vote
 >```
 
@@ -228,7 +238,7 @@ Finally, `Commit`, which is kept in ABCI++, no longer returns the `AppHash`. It
 `FinalizeBlock` to do so. Thus, a slight refactoring of the old `Commit` implementation will be
 needed to move the return of `AppHash` to `FinalizeBlock`.
 
-## Accomodating for vote extensions
+## Accommodating for vote extensions
 
 In a manner transparent to the application, CometBFT ensures the node is provided with all
 the data it needs to participate in consensus. 
@@ -247,22 +257,18 @@ of the usage of `retain_height` stay the same.
 The decision to store 
 historical commits and potential optimizations, are discussed in detail in [RFC-100](./../../docs/rfc/rfc-100-abci-vote-extension-propag.md#current-limitations-and-possible-implementations)
 
-## Handling upgrades to ABCI 2.0 
+## Handling upgrades to ABCI 2.0
 
-If applications upgrade to ABCI 2.0, CometBFT internally ensures that the [application setup](./abci%2B%2B_app_requirements.md#application-configuration-required-to-switch-to-abci-20) is reflected in its operation. 
-CometBFT retrieves from the application configuration the value of `VoteExtensionsEnableHeight`( *h<sub>e</sub>*,), 
-the height at which vote extensions are required for consensus to proceed, and uses it to determine the data it stores and data it sends to a peer 
-that is catching up.
+If applications upgrade to ABCI 2.0, CometBFT internally ensures that the [application setup](./abci%2B%2B_app_requirements.md#application-configuration-required-to-switch-to-abci-20) is reflected in its operation.
+CometBFT retrieves from the application configuration the value of `VoteExtensionsEnableHeight`( *h<sub>e</sub>*,),
+the height at which vote extensions are required for consensus to proceed, and uses it to determine the data it stores and data it sends to a peer that is catching up.
 
 Namely, upon saving the block for a given height *h* in the block store at decision time
-- if *h ≥ h<sub>e</sub>*, the corresponding extended commit that was used to decide locally is saved as well
-- if *h < h<sub>e</sub>*, there are no changes to the data saved
+* if *h ≥ h<sub>e</sub>*, the corresponding extended commit that was used to decide locally is saved as well
+* if *h < h<sub>e</sub>*, there are no changes to the data saved
 
 In the catch-up mechanism, when a node *f* realizes that another peer is at height *h<sub>p</sub>*, which is more than 2 heights behind,
-- if *h<sub>p</sub> ≥ h<sub>e</sub>*, *f* uses the extended commit to
+* if *h<sub>p</sub> ≥ h<sub>e</sub>*, *f* uses the extended commit to
       reconstruct the precommit votes with their corresponding extensions
-- if *h<sub>p</sub> < h<sub>e</sub>*, *f* uses the canonical commit to reconstruct the precommit votes,
+* if *h<sub>p</sub> < h<sub>e</sub>*, *f* uses the canonical commit to reconstruct the precommit votes,
       as done for ABCI 1.0 and earlier.
-
-
-