Changing workflow of consuming buffered packets

[tao-stones]

Problem

For leader to prioritize transactions by total_fee/total_cu (#22820), it needs access all buffered packets as deserialized and sanitized transactions in order to get transaction CUs and total fee. In current retain_mut loop, no good place to convert all buffered unprocessed packets into transactions.

Proposal:
To replace current "process packets while looping through buffer" with following workflow:

1. consume_buffered_packets iterate through buffered packet_batches for all unprocessed packet, produce Vec< packet_locator >, where packet_locator is (batch_index, packet_index). Therefore, the total number of packet_locator is the number of unprocessed packets in buffer;
2. stake-weight shuffle can be optional applied to Vec<packet_locator>, so it'd be in stake shuffled order instead of FIFO;
3. Iter through Vec<packet_locator> to packets (located by buffer[batch_index, packet_index]) into SanitizedTransactions, produce HashMap< packet_locator, SanitizedTransaction >
4. Compute tot_fee / tot_CUs for each transaction in above HashMap, produce HashMap< fee/cu, packet_locator>
5. Sort above HashMap by its Key, iter though to pass chunk of SanitizedTransactions to BankingStage::process_transactions().

Concerns:

1. While the proposed flow does not introducing additional heavy ops, ( for example, each unprocessed packet in buffer still deserialized into SanitizedTransaction once per call of Consume_buffered_packets() ), it is now processing transaction-by-transaction, instead of can still process chunk of transactions to amortize account locking/loading cost and preserve entry size (thanks @sakridge )
2. If the deserialized transactions (in step 3 above) to be cached for unprocessed packets, there are some saving from avoid re-converting packets to transaction.

Summary of Changes

Fixes #

[tao-stones]

moved to #23257