Cannot send messages, connection error after some period of time. Does not get reestablished. #5944
Closed
1 of 6 tasks
Assignees
Labels
Client
This issue points to a problem in the data-plane of the library.
customer-reported
Issues that are reported by GitHub users external to the Azure organization.
Event Hubs
Service Attention
Workflow: This issue is responsible by Azure service team.
Comments
loarabia
added
Client
|
@Evertvdw , thank you for creating this issue. We are investing this issue and will be back to you with our findings. |
|
@Evertvdw Can you provide more logs? After removing the lines with |
|
Also, if you can log a support ticket, we can also carry out investigations on the service side. Please consider doing that and sharing the ticket number here. |
I'm not sure I understand. Regarding the support ticket, I will do that, thanks for the suggestion! |
|
More logs (cleaner): |
|
@ramya-rao-a The ticket ID for the azure support is: 119103122000087 |
|
Thanks for the logs @Evertvdw and our apologies for not responding sooner. Can you share a code snippet showing how you are sending the messages? |
|
@chradek, Some observations about the sequence of events to aid in investigation while we wait for a response from @Evertvdw
|
|
@ramya-rao-a Yes I can, maybe there is something wrong with the approach I take: I listen to a Kafka topic and on every message I receive there this function is called. So especially on the initial connection a lot of data can come through and this function is called a lot of times. I'm currently not waiting for the function to finish before it can be called again, is that what causes the issue? Anyways, due to some mysterious circumstances after ~3 days of malfunction there is data coming through again now. I did not restart the docker container, so it somehow fixed itself. I will also post some logs of the current state of operation, maybe that helps. |
|
Here is an image of the TSI environment, where you can see that somewhere on 29-10 the data stopped coming in, and started steadily again at around 2-11. |
|
|
@Evertvdw This is because behind the scenes, the first time you call There are a couple things you can do in your code to work around this. One method that will be good overall for performance would be to batch your events so you make fewer Another thing you could do is make 1 send/sendBatch call to create the link, then allow subsequent send/sendBatch calls to occur in parallel. Doing this combined with the method above should give you the best results. As you were hinting, waiting for a send to complete before receiving another event to send should also alleviate the problem, though may also have lower throughput than the above options. |
|
Thank you for the explanation, will close this issue. |
|
Thanks for working with Microsoft on GitHub! Tell us how you feel about your experience using the reactions on this comment. |
6 tasks
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Describe the bug
I'm having trouble sending stuff to an event-hub. At first it worked fine but since yesterday I only get errors on nothing gets through. Restarting the container does not help. The issue looks a lot like #5282 but it's not exactly the same. Does anyone know what is going on here?
To Reproduce
I think somewhere the connection was lost between client and server, but I'm not sure.
Logs
Sample of the logging, the logs preceeded by INFO: are my personal logs, rest comes from the DEBUG mode.
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: