[WIP] Add post-mortem logging

[AndreiEres]

Description

Fixes #816

What we need

As described here: #816, in the case of a problem, there is no way to get additional information post-mortem. So we have to spend some time recreating the problem. There was mention of the defmt crate for embedded devices, which afaik doesn't work on Linux machines.

What will be changed

I'll try a simple approach. The tracing-subscriber crate allows us to add a new layer that we can use to forward our trace logs. Using the crate tracing-appender we can save them to the disk rolling files every hour. To save disk space, we can compress files and remove old ones.

Current rococo debug logs take ~66 MB for an hour, ~5 MB zipped. Assuming trace logs cost 6 times more, I bet for ~400 MB, ~30 zipped, which is not too much to keep logs for the last 24 hours. And of course I have to check how it affects the node performance.

I assume we can turn off this additional logging in the configuration, just like we do with prometheus metrics.

Checklist

• My PR includes a detailed description as outlined in the "Description" section above
• My PR follows the labeling requirements of this project (at minimum one label for T
  required)
• I have made corresponding changes to the documentation (if applicable)
• I have added tests that prove my fix is effective or that my feature works (if applicable)

[bkchr]

@AndreiEres can you please explain the approach you want to take here?

[AndreiEres]

@AndreiEres can you please explain the approach you want to take here?

Thank you, I updated the PR description.