Self reference in ReadAccountMapEntry

[Lichtso]

Problem

This single use of ouroboros::self_referencing (which was added in #12787) causes ~4.2% of all heap allocation operations during replay:

solana/accounts-db/src/accounts_index.rs

Line 348 in 8869d0c

#[self_referencing]

Self referencing structs are an anti-pattern and ouroboros should not be used casually. The reason is not that the Rust borrow checker is missing a feature or anything like that, but that structs can be moved and thus their address change. The only way to prevent that is to box (heap allocate) and pin the structure, effectively forbidding it to be reallocated (which is what ouroboros is doing under the hood).

In this case here the structure is tiny (it falls in the 16 byte size class, the smallest) and it is constructed a lot. Thus, it does not contribute much to the memory pressure, but it does to the memory management overhead.

Proposed Solution

Refactor the way these structures reference each other to avoid self references.

[brooksprumo]

@Lichtso Can you share how to do the profiling, so that I can compare fixes? Thanks in advance.

[carllin]

yaasss fix my bugs king @brooksprumo

[jeffwashington]

I did some digging.
on mnb, we are creating about 30k of these per slot. They do not get called during tx processing, but during new_from_parent type activities. I'll keep digging some more.

[jeffwashington]

see: #34918

[jeffwashington]

There is also great value in simplifying the client. getting rid of this use of self-ref does simplify things. Over time, the accounts index has drastically changed in its scalability, implementation, and binning. The result is the behavior is very different than it was before and the self-ref struct is complicated and no longer the best way to do this.