-
Notifications
You must be signed in to change notification settings - Fork 182
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Specify NATS stream replicas #7023
Comments
@kobergj from what I understand, having replicas = 1 does mean that a stream is dead when the nats node, where the stream lives, is gone? |
Isn't that always the case? From what I understand, if you set replicas=3, then 3 copies of the same event will be stored. But if the node is dead it cannot reach any of these 3 copies so the stream is dead too?! Or maybe I just misunderstand... |
See NATS documentation: https://docs.nats.io/nats-concepts/jetstream#persistent-distributed-storage (Section: Stream replication factor)
So we definitely need 3 replicas for every stream when we want to achieve HA. |
But isn't that a setting of the nats js server? We probably don't need it for the single binary, right? I cannot define the value when I publish something because the stream already exists? |
It's a setting of the stream. So when creating a stream, oCIS should specify replicas. Currently it's set to 1 replica. But for some oCIS installations this should be 3 or 5. 1 Replica is fine for the single process use case because we only have one NATS node.
Right, it's a property of the stream. That setting can be changed when creating a stream or |
Ah now I got it. Updating the stream is another thing. Should we add a "create-or-update" logic to go-micro? Or do you see downsides with that? |
Yes, probably this would apply to nats-js caches, events and store implementations, since they all leverage streams.
I would prefer the "create-or-update" logic. At least increasing replicas works flawlessly (when the enough NATS nodes are available). I did not test reducing replicas and setting replicas > number of available nodes. |
For Kubernetes the NACK exists: https://docs.nats.io/running-a-nats-service/configuration/resource_management/configuration_mgmt/kubernetes_controller We also could decide that stream configuration needs to be done outside of oCIS |
working on it in owncloud/ocis-charts#388 |
@mmattel I would consider this as a documentation only task. We should document that oCIS is creating streams with replicas set to 1. If one has a NATS cluster (=multiple nodes), one must ensure to also raise replicas for the streams. We'll add this to the Helm Chart deployment example in owncloud/ocis-charts#388 |
will be supersed if #7272 is implemented |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 10 days if no further activity occurs. Thank you for your contributions. |
Closed by owncloud/ocis-charts#472 |
Is your feature request related to a problem? Please describe.
I would like to have more than 1 replica for the oCIS NATS streams. See also https://docs.nats.io/nats-concepts/jetstream/streams
Describe the solution you'd like
Have a setting to set the replica to a number > 1 I like.
Describe alternatives you've considered
Manually edit the streams in NATS, eg
nats stream edit main-queue --replicas=3
.Additional context
The current stream looks like this:
The text was updated successfully, but these errors were encountered: