perf/feature: new serialization format for constraint systems

[gbotrel]

Description

This PR uses integer compression techniques to serialize some constraint systems parts containing ids, offsets, etc and bypass our current cbor choice. (cbor is still used for the rest).

As a bench, using lzss.DecompressionTestCircuit with params landing at 90M constraints (plonk), we go from 9225Mb to 2238Mb size, plus, we serialize (and most importantly deserialize) way faster:

benchmark                                      old ns/op        new ns/op      delta
BenchmarkSerialization/serialize_cbor-96       54962357901      3086254450     -94.38%
BenchmarkSerialization/serialize_cbor-96       41877637150      3117870148     -92.55%
BenchmarkSerialization/deserialize_cbor-96     100910397720     4267472009     -95.77%
BenchmarkSerialization/deserialize_cbor-96     117815108278     3776636214     -96.79%

[ivokub]

Very nice optimizations! I think in general is good, but in a perfect scenario I would try to figure out if we can work with io.Writer and io.Reader instead of direct byte arrays as imo otherwise we have many copies of very similar structures in memory (constraint system itself, bytes when serializing them and also byte buffer for internal steps a la serializing calldata). But due to applying compression to uints we still cannot get away with all.

And I completely agree that it doesn't make sense for us to include additional compression in the implementation. It should be straightforward to add gzip etc. when the user is interested in it. Otherwise there is too many options (fast or good compression ...)

[ivokub]

Very nice optimizations! I think in general is good, but in a perfect scenario I would try to figure out if we can work with io.Writer and io.Reader instead of direct byte arrays as imo otherwise we have many copies of very similar structures in memory (constraint system itself, bytes when serializing them and also byte buffer for internal steps a la serializing calldata). But due to applying compression to uints we still cannot get away with all.

And I completely agree that it doesn't make sense for us to include additional compression in the implementation. It should be straightforward to add gzip etc. when the user is interested in it. Otherwise there is too many options (fast or good compression ...)

Yep, it makes sense that if we work with []byte then can have parallel processing which is more important than saving a few GB in settings where memory usage is in tens of GBs anyway. Added the test corpus.

Good to merge from my side.