How can i get list of my custom created core-nats subscriptions in c#

[YashasAnand]

I basically want to have visibility on how many subscribtions i have created in core nats. As mentioned here https://github.com/nats-io/nats.go/discussions/1682 , we can query nats server using http requirest using system events. Is that possible to do in c#?

Is there anyway i can get to know list of custom created subscriptions for visibility ?

[mtmk]

I basically want to have visibility on how many subscribtions i have created in core nats

yes, if the monitoring is enabled you can just query the server using HTTP as explained in nats-io/nats.go#1682 as you pointed out. Should be fairly straight forward using HttpClient in C#. let me know if you'd like more details.

Is there anyway i can get to know list of custom created subscriptions for visibility ?

I think you can get fairly detailed information using http://localhost:8080/connz?subs=detail for example as described in monitoring page https://docs.nats.io/running-a-nats-service/nats_admin/monitoring#subscription-routing-information

[darkwatchuk]

AFAIR , the HTTP end-point here only reports connection on that server, so if you're in a cluster, you'll have to know the correct server to query, and have HTTP monitoring turned on too. This is also good if you don't have access to the system account.

However, if you do have access, it's a bit easier. You can use req/reply along with PING to get multiple answers from all servers at once, and pick your result. Or, if you know the server you want, ask it directly and just take one response.

So
$SYS.REQ.SERVER.PING.CONNZ
or
$SYS.REQ.SERVER.{serverId}.CONNZ

and pass something like this

JsonSerializer.Serialize(new {
offset = 0,
cid = connectionId,
subscriptions_detail = true
});

[YashasAnand]

Thank you @mtmk , @darkwatchuk for your responses, but i just tried the connz?subs=1 & connz?subs=detailed, only to get a bunch of encoded subscriptions?

"subscriptions_list": [
"_INBOX.otZRBiMlwOkTHrsmcD69yX",
"_INBOX.L8NIFDqGrggQal4T0hvhXY",
"_INBOX.TJRIUcXLyijBEzYioJKwYo.*",
]

is this expected? I also tried the subsz?subs=true & i got many results but some contained my subscriptions
{
"account": "$G",
"subject": "Orders.burgers",
"sid": "2",
"msgs": 3,
"cid": 51
}

Is there some settings i can do to get only user created subscriptions & not internal subscriptions?
Also if i have 50 subscribers on same topic, can i get this information from the server?

[darkwatchuk]

The _INBOX.> subscriptions will probably be as a result of any request/response as your requests will go out with a "Reply to this INBOX", apart from setting your own INBOX so you can track these specifically, there's not much you can do I don't think as you're sending something and then subscribing on the temporary inbox.

Other ones you've seen will be as a result from your subscribe to an a stream/subject (e.g. Orders.burgers). KV watchers will come across as $KV.

You should be able to use the test argument on subsz for getting a list of connections using a topic. Again this works on from http and the system account.

This is good for reference too...

https://docs.nats.io/running-a-nats-service/configuration/sys_accounts

[YashasAnand]

@darkwatchuk , @mtmk , there should be someway i can get list of my own custom subscriptions. I agree what you have suggested in blog works to get all subscribers, but we dont even have ability to assign any uniqueID for a CustomSubscriber.

Currently i have observed that those consumers which have queueGroup property are the ones ive created, but only based on this can i be 100% sure i get only custom created consumers?

Can we attach some metadata like in consumerConfig for jetstream to distingish these as custom created?

public IDictionary<string, string>? Metadata { get; set; }

PS: Also would it be possible to get some extra detail like last activity time on subscriber. this would help a lot too

[DarkPhantom-Dev]

This seems like a valid question , I am working on a similar project where I am trying to gain more visibility for infra teams who mainly do monitoring & alerts. @mtmk @darkwatchuk is it feasible for subscribers to give more info like last message consumed by core-nats subscriber?

Trying to manage this by ourselves in a microservice like environment poses a concern , since we need to keep track of individual subscribers in caches in each microservice & then query each microservice for data (not to mention the overhead for updating the cache on each message consumed ).

Could this be taken as an improvement or its not feasible as per NATS design?

[darkwatchuk]

Yes, I agree. Visibility on Microservices isn't that great. I plan to add it to a monitoring platform I'm writing so have only dipped in to it a bit for viability, but not gone anywhere in depth on it at the moment. I guess that because Microservices is based on core NATS request/reply, and core NATS was built for speed and not tracking/guarantees then this is not really possible (that's a the JetStream layer), but you'd really need to get the NATS guys views on this one!

[mtmk]

do you see this implemented on client side?

[YashasAnand]

Yup, absolutely makes sense @mtmk . But what if we use both jetstream and core nats, as jetstream is built on top of core nats, if we use a mechanism where we persist data for some subjects, and use core nats subscribe for others, all enabled on the same jetstream broker, in such a set up would it be possible for subscribers to give more metrics?

PS: for further clarification, what I mean is 1 jetstream broker supporting both jetstream subjects and core nats subject.

[mtmk]

if I understood your question correctly, strictly speaking jetstream doesn't provide stats for 'subscriptions' (core nats concept) but 'consumers' (which is a jetstream concept). but you can certainly mix jetstream and core nats usage in your application which is very common. one ideal of NATS is to be the sole communications mechanism for an application driven by subject based addressing and varying levels of delivery modes i.e. at-most-once (core nats), at-least-one (jetstream).

sorry it's a bit too high-level and fluffy answer but going back to your concrete example could you describe what exactly you're after as much as you can? Do you want to track all messages in a micro service setup or gather trace metrics for example?

[YashasAnand]

@mtmk , jetstream consumers give metrics like NumWaiting, NumAckWaiting, LastActive (last message consumed time) , all these are given out of the box for jetstream consumers.

I understand metrics like NumWaiting or AckWaiting does not make sense for subscribers, but could we have metrics like last delivered message which can be tracked by each subscriber?

Also i have observed for jetstream that if i have one subject (orders.burgers) & 5 consumers all with same subject filter consuming from same application on 5 different threads, Jetstream is considering this as 1 consumer with 5 clients bound to this consumer. It internally does load-balancing among the threads, but it would be nice if we could get this visibilty too.

Something like NumClientsForConsumer which can give us the number 5.

[mtmk]

it sounds to me these can only be done server side and not really C# specific, probably best discussed on the server discussion page https://github.com/nats-io/nats-server/discussions I could move this discussion there but I'd suggest to open a new discussion with revised points raise, potentially referencing this one.

[mtmk]

for reference: nats-io/nats-server#6276