Skip to content

Latest commit

 

History

History
157 lines (136 loc) · 8.99 KB

README.md

File metadata and controls

157 lines (136 loc) · 8.99 KB
Raw

CPP serializers benchmark

tested libraries

GCC 11 (Ubuntu 20.04 x64)

library test case bin size data size ser time des time
bitsery general 70904B 6913B 1470ms 1524ms
bitsery brief syntax1 70888B 6913B 1416ms 1561ms
bitsery compatibility2 75192B 7113B 1490ms 1291ms
bitsery compression3 70848B 4213B 1927ms 2044ms
bitsery fixed buffer4 53648B 6913B 927ms 1466ms
bitsery stream5 59568B 6913B 1611ms 6180ms
bitsery unsafe read6 66760B 6913B 1352ms 982ms
boost general 279024B 11037B 15126ms 12724ms
cereal general 70560B 10413B 10777ms 9088ms
flatbuffers general 70640B 14924B 8757ms 3361ms
handwritten general7 47936B 10413B 1506ms 1577ms
handwritten unsafe8 47944B 10413B 1616ms 1392ms
iostream general9 53872B 8413B 11956ms 12928ms
msgpack general 89144B 8857B 2770ms 14033ms
protobuf general 2077864B 10018B 19929ms 20592ms
protobuf arena10 2077872B 10018B 10319ms 11787ms
yas general11 61072B 10463B 2286ms 1770ms
yas compression12 65400B 7315B 2770ms 2498ms
yas stream13 56184B 10463B 10871ms 11182ms
zpp_bits general 52192B 8413B 733ms 693ms
zpp_bits fixed buffer 48000B 8413B 620ms 667ms

Clang 12.0.1 (Ubuntu 20.04 x64)

library test case bin size data size ser time des time
bitsery general 53728B 6913B 2128ms 1832ms
bitsery brief syntax1 55320B 6913B 2789ms 2071ms
bitsery compatibility2 54360B 7113B 2195ms 1953ms
bitsery compression3 54688B 4213B 4315ms 4181ms
bitsery fixed buffer4 49248B 6913B 946ms 1941ms
bitsery stream5 54776B 6913B 2047ms 6089ms
bitsery unsafe read6 49688B 6913B 2092ms 1162ms
boost general 237008B 11037B 16011ms 13017ms
cereal general 61480B 10413B 9977ms 8565ms
flatbuffers general 62512B 14924B 9812ms 3472ms
handwritten general7 43112B 10413B 1391ms 1321ms
handwritten unsafe8 43120B 10413B 1393ms 1212ms
iostream general9 48632B 8413B 10992ms 12771ms
msgpack general 77384B 8857B 3563ms 14705ms
protobuf general 2032712B 10018B 18125ms 20211ms
protobuf arena10 2032760B 10018B 9166ms 11378ms
yas general11 51000B 10463B 2114ms 1558ms
yas compression12 51424B 7315B 2874ms 2739ms
yas stream13 54680B 10463B 10624ms 10604ms
zpp_bits general 47128B 8413B 790ms 715ms
zpp_bits fixed buffer 43056B 8413B 605ms 694ms

Additional tests information

  1. deserialization using brief\_syntax, similar to cereal
  2. forward/backward compatibility enabled for Monster
  3. all components of Vec3 is compressed in [-1.0, 1.0] range with precision 0.01
  4. use non-resizable buffer uint8_t[150000] for serialization
  5. use stream input/output adapter, underlying type is std::stringstream
  6. on deserialization do not check for errors
  7. check buffer size on reading, but writing buffer is preallocated std::array<uint8_t, 1000000>
  8. doesn't check for buffer size when reading, buffer: std::array<uint8_t, 1000000>
  9. use std::stringstream's internal std::string
  10. use arena allocator
  11. use yas::mem_<io>stream as buffer
  12. with yas::no_header and yas::compacted
  13. using std::stringstream

NOTE: tests for protobuf and flatbuffers is not 100% fair, because huge amount of CPU cycles goes to converting from generated types, to our defined types.

Why another cpp serializers benchmark

I'm aware that cpp-serializers project already exists, but it's testing set is way too simple and you cannot compile each project to separate executable.

This project contains more realisting data that needs to be serialized.

    enum Color : uint8_t {
        Red,
        Green,
        Blue
    };

    struct Vec3 {
        float x;
        float y;
        float z;
    };

    struct Weapon {
        std::string name;
        int16_t damage;
    };

    struct Monster {
        Vec3 pos;
        int16_t mana;
        int16_t hp;
        std::string name;
        std::vector<uint8_t> inventory;
        Color color;
        std::vector<Weapon> weapons;
        Weapon equipped;
        std::vector<Vec3> path;
    };

All data is random generated, although seed is hard-coded to get predictable results when running same test multiple times.

All projects implement same interface for serialization and deserialization.

struct Buf {
    const uint8_t* ptr;
    size_t bytesCount;
};

class ISerializerTest {
public:
    virtual Buf serialize(const std::vector<MyTypes::Monster>& data) = 0;
    virtual void deserialize(Buf buf, std::vector<MyTypes::Monster>& res) = 0;
    virtual ~ISerializerTest() = default;
};

Testing routine consist of few steps:

  • data generation step, in which monsters are generated (default 50 monsters)
  • warmup step, in which serialization and deserialization is run 5 times, to warmup cpu cache and check if deserialized data equals to original data.
  • measurement step, runs serialization and deserialization multiple times (default 300000 samples), deserialization happens on same object, to avoid costly allocate operations for new object construction each time.

Building & testing

  1. Build project 
    mkdir build && cd build
cmake ..
make
  2. Run tests with ctest -VV OR
  3. Generate testing results (requires nodejs) 
    cd ../tools/
npm install
npm start