Feature request: canonical (e.g., little-endian) binary archive

[stanford-scs]

This is a feature request for a canonical binary archive, which would be useful in cases where people must cryptographically hash or digitally sign messages, or in which non-cryptographic hashes are used for the purposes of optimizing state reconciliation or otherwise bucketing data structures based on contents.

Currently the portable_binary archive uses a lazy optimization that makes for a different wire format on big and little endian machines. Hence, if a big- and little-endian machine both sign the same data structure serialized with a PortableBinaryOutputArchive, the actual signed bytes will be different. In the best case multiple copies of the same message can be sent around, which is wasteful. In other cases, things like threshold signatures, in which multiple machines of possibly different endianness collaborate to sign a single message, are effectively impossible with cereal.

The best canonical format would probably be an archive that produces and reads little endian format on all architectures.

[c42f]

To add to this: I currently have the need for a canonical big-endian reader: this is to deserialize messages from an embedded system I don't have direct control over. Granted, this may be more of a case of writing a custom archiver, but the embedded system produces very straightforward big endian binary so one might hope to get away with using a standard archiver. The one thing I know I'd need - which is different from the portable endian archiver - is the ability to omit the byte order marker from the start.

[AzothAmmo]

To resolve this I'm going to modify the existing canonical archive to be parameterized on the desired input/output endianness, with conversion happening as required.

[AzothAmmo]

Well I finally got around to doing this - you can now specify the desired input/output endianness in the portable binary archives. The archive is backwards compatible with the prior version and will still output one byte of metadata at the beginning of the serialization.