Pseudo floats

[voltrevo]

What is this PR doing?

Adds pseudo-floats and uses them for ethValue and gas, to reduce the number of bytes used.

By optimizing for decimal values, this format achieves a size of just 2 bytes as long as you only use 3 significant figures.

The cool thing is, the format also supports all uint256 values, which means it can be used for the fallback case. If you need 5 significant figures, pseudo-float has still got you covered with just 3 bytes.

I believe this supersedes the fixed 3-byte format proposed here, which was going to deliver 6 significant figures (only one extra significant figure for 3 bytes, at the cost of flexibility and extra bytes for value=zero and 3 sig figs).

PseudoFloats are almost twice as good as RLP for non-zero values, based on real data. (Zeros not measured, since both formats simply use 0x00 for zero.)

// Excerpt from PseudoFloat.sol

/**
 * Like a float, but technically for integers. Also base 10.
 *
 * The pseudo-float is an encoding that can represent any uint256 value but
 * efficiently represents values with a small number of significant figures
 * (just 2 bytes for 3 significant figures).
 *
 * Zero is a special case, it's just 0x00.
 *
 * Otherwise, start with the value in scientific notation:
 *
 *     1.23 * 10^16 (e.g. 0.0123 ETH)
 *
 * Make the mantissa (1.23) a whole number by adjusting the exponent:
 *
 *     123 * 10^14
 *
 * We add 1 to the exponent and encode it in 5 bits:
 *
 *     01111 (=15)
 *
 *     Note: The maximum value we can encode here is 31 (11111). This means the
 *     maximum exponent is 30. Adjust the left side of the previous equation if
 *     needed.
 *
 * Encode the left side in binary:
 *
 *     1111011 (=123)
 *
 * Our first byte is the 5-bit exponent followed by the three lowest bits of the
 * mantissa:
 * 
 *     01111011
 *     ^^^^^-------- 15 => exponent is 14
 *          ^^^----- lowest 3 bits of the mantissa
 *
 * Encode the remaining bits of the mantissa as a VLQ:
 *
 *     00001111
 *     ^------------ special VLQ bit, zero indicates this is the last byte
 *      ^^^^^^^----- bits to use, put them together with 011 above to get
 *                   0001111011, which is 123.
 *
 * Putting it together is two bytes:
 *
 *     0x7b0f
 *
 * Example 2:
 *
 *     0.883887085 ETH uses 5 bytes: 0x55b4d7c27d
 *     883887085 * 10^9
 *     For exponent 9 we encode 10 as 5 bits: 01010
 *     883887085 is 110100101011110000101111101(101)
 *
 *     01010101 10110100 11010111 11000010 01111101
 *     ^^^^^------------------------------------------- 10 => exponent is 9
 *          ^^^---------------------------------------- lowest 3 bits
 *               ^^^^^^^--^^^^^^^--^^^^^^^--^^^^^^^     higher bits
 *              ^--------^--------^-------------------- 1 => not the last byte
 *                                         ^----------- 0 => the last byte
 *
 * Note that the *encode* process is described above for explanatory purposes.
 * On-chain we need to *decode* to recover the value from the encoded binary
 * instead.
 */
library PseudoFloat {
  ...
}

How can these changes be manually tested?

Use encodePseudoFloat to encode a value in js and pseudoFloat.decodePublic to decode it using the evm. See pseudoFloat-test.ts.

Does this PR resolve or contribute to any issues?

Resolves #521.

Checklist

• I have manually tested these changes
• Post a link to the PR in the group chat

Guidelines

• If your PR is not ready, mark it as a draft
• The resolve conversation button is for reviewers, not authors
  • (But add a 'done' comment or similar)

[jzaki]

Checking if this was the original version range (VLQ.sol too)? Was the code audited at a particular compiler version? We should ensure the project is configured to compile this lib with it (like the BLS lib PR comment).

[voltrevo]

I've been just copying that version range, I think it originates with BLSExpander.sol.

Was the code audited at a particular compiler version?

Nah this is new code.

We should ensure the project is configured to compile this lib with it (like the BLS lib PR comment).

I'm not sure what you mean by this. I believe that hardhat will compile it with whatever latest matching solidity version is available in the project, which is currently 0.8.15 (hardhat.config.ts has 0.6.12, 0.7.6, 0.8.15, so 0.8.15 is the latest matching version).

[jzaki]

Why does this stop at 30? (2^256 = 10^77-ish) Is it from their code?

EDIT: I noticed that an example show's 5 bits being used for the exponent (2^5 = 32), so it looks to be by design. Most whole ethers and gas values are 10^9 to begin, so another 21 on top of that is a lot of headroom for values being compressed.

[voltrevo]

Yeah it's to make sure the exponent fits into a small space.

If you go above that range, you can still encode it using the VLQ part:

console.log(encodePseudoFloat(ethers.constants.MaxUint256));
// 0x0f81ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f

[jzaki]

Could test the top end of the exponent (10^30, 10^31).
And also 10^77, the most significant digit 1 the rest 0 (base 10).

[voltrevo]

Good idea, will followup 👍.