improve deposit processing performance

[etan-status]

When there are a lot of deposits, we decompress the public key into a crypto cache. To avoid having those caches grow unreasonably big, make sure to operate on the decompressed pubkey instead.

[arnetheduck]

this is the operation that is slow ;)

[etan-status]

Yes, but if not done here, it additionally ends up in the blsVerify cache which is unbounded and never pruned.

[etan-status]

this effectively skips blst_p1_affine_in_g1 just to assert :)

[arnetheduck]

Validator instances may come from an untrusted source (like the REST API or an invalid checkpoint sync state) - as long as this function is public, there's no guarantee that the pubkey inside is "valid enough"

[arnetheduck]

it's an odd function in general - I think it's worth rethinking this flow in general and see if there's low-hanging room for improvement

[mratsim]

To give some figures on the pubkey deserialization speed.

Source mratsim/constantine#183

Pubkey deserialization without check: 11us
Pubkey deserialization with checks: 45 ns (ratio ~4.5x)

This is the same speed in Constantine, which will allow me to extrapolate uncompressed deserialization speed

Compression explanation

A BLS12-381 G1 (pubkey) point has coordinate (x, y) with formula: y² = x³ + 4, when you compress, you only store the x coordinate and because sqrt(x³+4) has 2 solutions, you can store which y you refer to (+y or -y) in a single bit. Thankfully BLS12-381 only needs 381-bit and so we have 3 extra bits for metadata (since we can only store bytes, so 48 bytes = 384 bits).

Difference between compressed and uncompressed speed

So the differences between compressed and uncompressed representation are:

• The size, compressed is 2x smaller
• The sqrt operation which cost 11us in Constantine, and I assume about the same in BLST (I actually have optimizations to save ~15% compared to BLST but in this particular use-case they are irrelevant)
  

Conclusion

99.9% of deserialization cost without checks is due to compression and we can make it virtually instant with uncompressed pubkeys

[arnetheduck]

we can make it virtually instant with uncompressed pubkeys

unfortunately we don't have uncompressed deposit pubkeys - so the only thing we can do with deposits that I know is to batch-verify all deposits in a block (unless there's a way to batch-load keys as well?)..

oh, and as etan points out, we could avoid decompressing the keys twice, but that's probably a non-trivial flow to introduce

[arnetheduck]

hm, maybe we can have a variant pubkey type that holds either compressed or uncompressed key

[mratsim]

(unless there's a way to batch-load keys as well?)..

We can load them in parallel but there is no math / cryptography or number-theoretic construct to improve batch-load like we can batch-verify.

hm, maybe we can have a variant pubkey type that holds either compressed or uncompressed key

In terms of memory, the variant take as much space as the bigger type + 1 byte.
Also while in the previous case we were aligned, due to the runtime variant tag, the key will be unaligned, on x86 unaligned load aren't a problem, unsure about ARM.

Anyway the question becomes, are there cases where we would load a compressed pubkey without using it right away and so we want to delay paying that cost?

[etan-status]

The flow here is (1) deposit processing, as part of which pubkey is fully loaded and checked; (2) create Validator from it which contains compressed form once more; (3) when the entire block checks out, as part of saving it to DB, load the pubkeys once more to have efficient DB access from then onward.

In this PR, (1) is modified to not pollute the last-resort crypto key cache, and (3) is modified to use the decompression without repeating the entire verification process.

[mratsim]

(1) is modified to not pollute the last-resort crypto key cache

If someone becomes a validator they will attest once every 6 min anyway so it will be cached and we only delay the inevitable, or did I miss something?

[etan-status]

There is a separate crypto key cache when using ValidatorPubKey with blsVerify. We have a separate cache of CookedKey in beacon_chain_db that already will hold validator keys. The changes here avoid the use of the separate crypto key cache in addition to the DB cache.