Google ProtoBuf for Accel-Sim trace file

[William-An]

Current Accel-Sim trace file is text-based, unnecessarily consuming a large amount of storage space. Although the trace size can be cut to 1/10 with #265 compression trick, another problem with text-based trace format is the complexity of consuming the trace. A parser has to be built to convert the text traces into binary data structures for Accel-Sim or other analysis tools. Considering the fact that most of the time traces are used to drive the simulator rather than examined by humans, it is more convenient to have it in binary form and build a decoder to convert it into text form when needed.

Therefore, I propose using the Goolge ProtoBuf as a substitute for the text trace format. It has the following benefits over current format:

1. Compact in size and support general compression, so the compression PR can also work.
2. Multi-language support with C++, C#, Python, Java, etc. Alleviate the need to write parsers for analysis tools designed to work with our traces
3. Can support trace streaming, so the simulator won't need to read the whole file into memory, reducing runtime memory usage
4. Backward compatibility: with some rules followed, old code and read new trace files just fine, so just need to keep 1 copy of the trace file for all past simulator versions.
5. Used by Gem5 simulator (Trace CPU) and Nsight compute profiler for trace collection, proved to work by existing projects.

[tgrogers]

I like the idea. I have not used protobufs before. Are they stored in binary format on disk?
If so, are there utilities to parse and print the trace so we can still do manual text inspection on demand? It is quite convenient sometimes to just look at the traces and see what they are doing.

[William-An]

1. Yes protobufs are stored in binary format on disk
2. I don't think they provide such an official tool, but it should be easy to build one ourselves, basically just add a bunch of prints to the generated parser.