Store Pre-signatures and Triples using Redis, refactor managers

[volovyks]

This is the setup of Docker in the same container. Some people consider it a bad practice, but it may be a simpler solution for node validators. I have no strong opinion here.

It is expected to be executed with

docker run -d \
  --name my-redis-rust-app \
  -v /path/on/host/redis-data:/var/lib/redis \
  -p 6379:6379 \
  -p 8080:8080 \
  my-redis-rust-app

It will persist the apend file on the VM.

cc @kmaus-near @ppca

[ppca]

@kmaus-near How does your draft redis setup work? Is it different or similar to this?

[kmaus-near]

@kmaus-near How does your draft redis setup work? Is it different or similar to this?

My homelab setup uses a dedicated VM running a Redis container vs. baking it into the multichain image, but other than that similar configuration. @volovyks and myself discussed this a bit yesterday, it's not exactly the "best practice" but if we are trying to simplify this for our partners, this may be a better route. Plus if we are to run it locally, we are then going to be running 3 containers on a single VM if we do not bake it into the image itself, which is not ideal resource wise or best practice wise. In short, both are not ideal, but for different reasons 😂.

I would suggest that we try to keep the code interacting with Redis as decoupled as possible, so if in the future we need to separate Redis from the base image, we can do so easily.

[volovyks]

@kmaus-near all we need in code is the Redis URL, so it is not coupled. If we change how we manage Redis storage, most of the complexity will be on the Terraform side.

[volovyks]

Merged triple storage into this PR:

What was done:

• Removed duplicate storage (DB and memory) approach
• Datastore replaced with Redis
• Refactoring of TripleManager

Other:

• I removed StuckMonitor because it does not fit well into the design with Redis. I assume it was used for testing. I can add a lightweight version in Triple and Presignature managers if requested.

[ChaoticTempest]

now that I see this, it's better for us to just have an extra bool field denoting the ownership of the presignature even though we already maintain a list of mine presignatures in redis. Would simplify the code a bit

[volovyks]

Redis is not designed for such data structures. For example, it will need to deserialize all triples to count mine.

[ChaoticTempest]

#914 should fix the issues you were seeing @volovyks. Too many write locks were being acquired at the same time

[ppca]

LGTM!

[ppca]

@ChaoticTempest was the reason behind removing Arc and RwLock for triple/presignature storage because redis itself can already handle the concurrent read and write with its own locking mechanism?

[ChaoticTempest]

@ppca yup, when sending items to be stored to redis, it's going to be over the wire anyways, so mutability does not matter at all here, thus we don't need RwLock at all. The redis pool struct actually has an internal Arc, so no need for us to manage it ourself