Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Service Bus sdk ServiceUnavailableError #10641

Closed
1 task done
Connor-Wood opened this issue Aug 15, 2020 · 2 comments
Closed
1 task done

Service Bus sdk ServiceUnavailableError #10641

Connor-Wood opened this issue Aug 15, 2020 · 2 comments
Assignees
@richardpark-msft
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Bus

Comments

@Connor-Wood
Copy link

Connor-Wood commented Aug 15, 2020 •
edited
Loading

  • Package Name: @azure/service-bus
  • Package Version: 1.1.7
  • Operating system: mac & Debian
  • nodejs
    • version: 8.11 & 8.16

Describe the bug
After receiving a message via the registerMessageHandler with autoComplete: false, maxMessageAutoRenewLockDurationInSeconds: 3600, and holding onto it/ not settling it, the onError will receive ServiceUnavailableError after ~80 seconds.

Expected behavior
Sdk continues to renew the lock without errors.

ServiceUnavailableError: The request with message_id "cd8a6827-7ca8-234f-b9b1-bf6936fe486d" to "cowoodTopic_cowoodSub/$management" endpoint timed out. Please try again later.
at translate (/Users/connorwood/Desktop/sb-load/node_modules/@azure/amqp-common/dist/index.js:908:17)
at Timeout.actionAfterTimeout [as _onTimeout] (/Users/connorwood/Desktop/sb-load/node_modules/@azure/amqp-common/dist/index.js:1245:31)
at ontimeout (timers.js:498:11)
at tryOnTimeout (timers.js:323:5)
at Timer.listOnTimeout [(timers.js:290:5)](url

logs.txt
)
@ghost ghost added needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. customer-reported Issues that are reported by GitHub users external to the Azure organization. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Aug 15, 2020
@Connor-Wood
Copy link
Author

another side note of things I saw during this but doesn't seem to be in these logs from what I can tell. was ~10 of these logs, not sure if its an issue. We dont use Emitters.

(node:30579) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 message listeners added. Use emitter.setMaxListeners() to increase limit

@ramya-rao-a ramya-rao-a added Client This issue points to a problem in the data-plane of the library. Service Bus labels Aug 16, 2020
@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Aug 16, 2020
@ghost ghost added the needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team label Aug 16, 2020
richardpark-msft added a commit to richardpark-msft/azure-sdk-for-js that referenced this issue Aug 17, 2020
@richardpark-msft
Don't remove the event listener for a message until we've either:
f8bf43e 
- rejected the promise because of an error
- resolved the promise
- timed out and rejected the promise

Otherwise you'll end up in a situation where you have multiple handlers that are constantly removing their listener and missing responses.

Fixes Azure#10641
@richardpark-msft richardpark-msft mentioned this issue Aug 17, 2020
Merged
richardpark-msft added a commit that referenced this issue Aug 18, 2020
@richardpark-msft
[amqp-common] Remove the mgmt link listener only when it matches or t…
15609fc 
…he Promise rejects (#10662)

Mostly just ports, fixing a current customer issue (#10641) where removing the event listener in the mgmt client for messages too aggressively was causing us to miss responses.

While we were in there we also ported #9494 (stray timer can prevent app exit on error).

Fixes #10641
richardpark-msft added a commit to richardpark-msft/azure-sdk-for-js that referenced this issue Aug 18, 2020
@richardpark-msft
Update amqp-common to get the fixes for Azure#10641 (managementlink t…
dfdb3a7 
…iming out when sending parallel requests) and Azure#9287 (management client not properly clearing timer, causing app to hang on exit)
@richardpark-msft richardpark-msft mentioned this issue Aug 18, 2020
Merged
richardpark-msft added a commit that referenced this issue Aug 19, 2020
@richardpark-msft
[service-bus] Update to amqp-common 1.0.0-preview.16 (#10688)
083d38d 
* Update amqp-common to get the fixes for #10641 (managementlink timing out when sending parallel requests) and #9287 (management client not properly clearing timer, causing app to hang on exit)
* Updated external dependency @azure/amqp-common to 1.0.0-preview.16

Fixes #10641
@richardpark-msft
Copy link
Member

@Connor-Wood - the latest version of the service bus package has been published which fixes this issue.

https://www.npmjs.com/package/@azure/service-bus/v/1.1.9

With regards to the warning:

(node:30579) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 message listeners added. Use emitter.setMaxListeners() to increase limit

We're discussing about how to deal with this one but we don't have a fix forthcoming for this. It is cosmetic however.

Closing this issue as it's fixed with the new package but please re-open if you have further questions about the fix.

@ramya-rao-a ramya-rao-a removed the needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team label Sep 12, 2020
openapi-sdkautomation bot pushed a commit to AzureSDKAutomation/azure-sdk-for-js that referenced this issue Dec 30, 2020
@solankisamir @maksimkim
@promoisha @RupengLiu
CodeGen from PR 11549 in Azure/azure-rest-api-specs
245b758 
Api Management - make /tenant endpoints ARM compliant in 2020-06-01-preview version (Azure#11549)

* Adds base for updating Microsoft.ApiManagement from version stable/2019-12-01 to version 2020-06-01-preview

* Updates readme

* Updates API version in new specs and examples

* Add support in API Management for Availability Zones (Azure#10284)

* apim in azs

* fix prettier check

* PATCH should return 200 OK (Azure#10328)

* add support for PATCH returning 200 OK

* CI fixes

prettier fix

CI fixes part 2

* Password no longer a mandatory property when uploading Certificates

* add missing x-ms-odata extension for filter support

* +gatewayhostnameconfiguration protocol changes (Azure#10292)

* [2020-06-01-preview] Update Oauth Server secrets Contract (Azure#10602)

* Oauth server secrets contract

* fix azureMonitor enum

* API Management Service Deleted Services Resource (Azure#10607)

* API Management Service Deleted Services Resource

* Path fix

* Lint + custom-words fixes

* Location URI parameter for deletedservices Resource

* GET for deletedservices by service name

* Remove resourceGroupName from resource path

* fixes

* schema for purge operation

* perttier applied

* 204 response code added

Co-authored-by: REDMOND\glfeokti <glfeokti@microsoft.com>

* OperationNameFormat property added to Diagnostic contract (Azure#10641)

* OperationNameFormat property added to Diagnostic contract

* add azuremonitor to update contract

Co-authored-by: REDMOND\glfeokti <glfeokti@microsoft.com>

* [Microsoft.ApiManagement][2020-06-01-preview] Change Network Status response contract (Azure#10331)

* Change Network Status response contract

* Update examples for network status contract

* ApiManagement - tenant/settings endpoints

* ApiManagement - tenant/settings endpoints fix

* ApiManagement - tenant/settings endpoints fix prettier

* ApiManagement - tenant/settings endpoints fix 3

* ApiManagement - tenant/settings endpoints fix 4

* ApiManagement - tenant/settings endpoints fix 5

Co-authored-by: Samir Solanki <samirsolanki@outlook.com>
Co-authored-by: maksimkim <maksim.kim@gmail.com>
Co-authored-by: promoisha <feoktistovgg@gmail.com>
Co-authored-by: REDMOND\glfeokti <glfeokti@microsoft.com>
Co-authored-by: RupengLiu <rliu1211@terpmail.umd.edu>
Co-authored-by: vfedonkin <vifedo@microsoft.com>
@github-actions github-actions bot locked and limited conversation to collaborators Apr 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@richardpark-msft richardpark-msft

Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that Service Bus
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ramya-rao-a @Connor-Wood @richardpark-msft