Use Message Queue pallet for UMP dispatch

[gavofyork]

Depends on paritytech/substrate#12485, #4097, #7105
Fixes #6129
Partially addresses paritytech/polkadot-sdk#489
Partially addresses paritytech/polkadot-sdk#488

This utilises the new pallet-message-queue for UMP message dispatch. With the unified dispatch system, there is no need for "UMP" to have a pallet of its own, so it is removed in favour of there being a single ump_tests.rs file to ensure that the message-queue functionality is working as expected for UMP. The event UMP::ExecutedUpward was replace by a MessageQueue::Processed.

This also means that the two weight fields of the HostConfiguration struct are no longer needed, so there is now a v5 of said struct with them gone. A migration to v5 is provided and should be used when this is merged.

Small auxiliary changes:

• Respect MAX_UPWARD_MESSAGE_SIZE_BOUND in the configuration pallet.
• Add a WellKnownKey struct for typed access to well-known keys.

Migrations

• Configuration pallet v5 -> v6
• Enqueueing a HostConfiguration update

Also re-writes the V4 -> V5 migration of the config pallet to be re-usable.

HostConfiguration

An update for the HostConfiguration is queued to update the UMP constants. This has to happen in accord with upgrading the UMP queue and is therefore not done in a normal Referendum. Considerations about the affected values are in gdrive or here as
pdf:

Constants (per para):	Polkadot	Kusama	Wested	Rococo
max_upward_queue_size [bytes]	1048576	4194304	8388608	8388608
max_upward_message_num_per_candidate	16	128	512	1024
max_upward_message_size [bytes]	65531	65531	131067	32763
max_upward_queue_count (1 Byte msgs)	174762	699050	1398101	1398101
MessageQueueMaxStale	8	16	48	96

Note For Para* Teams

The UMP message queue has a lazy message deletion mechanisms to unclog it in case it got filled with overweight messages.
This reaping mechanisms can be triggered by any account on the relay chain by calling MessageQueue::reap_page. This basically requires a offchain worker on the relay either for all parachains at once, or one for each. This observation mainly comes from a security perspective, since a completely clogged UMP queue is a possible DoS vector. More convenience is planned in paritytech/substrate#13877.

TODO

• Introduce MessageQueue pallet and configure properly in Relay-chain runtimes.
• Enable HostConfiguration migration in Relay-chain runtimes.
• Manage avoidance of message deletion:
  • Prior to allowing a para to be retired, it should have an empty dispatch queue.
  • After a para is scheduled to be retired, it should not be allowed to receive any UMP messages.
  • Tests for both conditions
• Permanently remove queue from the MQ pallet after para offboarding.
• Fix XCM simulator
• Benchmark and weight for receive_upward_messages.
• OnFinalize hook to update the well-known-keys queue-capacity values. (I.e. after they've been executed this block.)
• Update implementers guide

Follow Ups

• Remove MAX_UPWARD_MESSAGE_SIZE_BOUND and use the bound from the MQ pallet.

[eskimor]

Nice!

When wrapping my head around xcmp/hrmp and in particular ump, one of my first question that comes to mind for UMP, why do we need queuing at all? I can think of a couple of reasons:

1. Execution is expected to be significantly more costly than storing.
2. We don't want to be forced to drop messages. Given 1, by queuing we buy
   us time to react, e.g. ramping up the price so we can economically back
   pressure. If we executed immediately and we are above weight limit, we would
   have no other way than dropping messages. In the case of UMP, parachains can see filled up queue and stop sending.
3. We already have a 1 element (per para) queue, via
   PendingAvailabilityCommitments. We could in theory back pressure right here
   and not free a core until we processed its messages. The problem apart from
   potentially worse performance (block times degrade on bursts) is fairness:
   The load might actually be coming from other parachains and we would
   potentially back-pressure on "light" ones, just because the others filled up
   the block already. Other issues?

Other observations:

• Since the message gets into state on backing but gets queued
  on inclusion: The block causing the enqueuing of the message, does not actually
  contain it. So far just an observation, triggered by me wondering whether we could still be forced to drop messages somewhere because we exceed the weight limit due to the needed state access for example.

• Missing apart from the already mentioned bullet points is actual processing of messages? There is ProcessXcmMessage::process_message, but it does not seem to be called anywhere and the corresponding test is also blank.

[gavofyork]

When wrapping my head around xcmp/hrmp and in particular ump, one of my first question that comes to mind for UMP, why do we need queuing at all?

It is crucial to understand that incoming messages might have arbitrary effects within a runtime. There may be constraints related to the conditions under which these effects may be realised. The most obvious example of this is that a certain amount of weight might be required. However, more exotic chains may need to impose further constraints relevant to its own abstractions and these we cannot possibly imagine right now.

Therefore, necessarily, from the point of view of the low level protocols XCMP, UMP and DMP (*MP), we must restrict our concern to ferrying datagrams and nothing more. The message queuing and execution subsystem(s) are left with the concern of ensuring that the proper effects take place ASAP given the datagrams received and taking into account quality of service and ephemeral constraints and resource limitations.

This message queue pallet provides this functionality, accepting datagrams infallibly as fast as they can be delivered by the underlying message passing system(s) and promising that (subject to certain well-understood static assumptions) they will be executed eventually. It takes into account 2D weight and QoS ensuring that one parachain's message-passing activity cannot unfairly starve another parachain from attention.

[ggwpez]

bot merge