-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: disable concurrency when downloading azure files #12532
Conversation
workaround for Azure/azure-sdk-for-go#22156 Signed-off-by: Roel Arents <roel.arents@kadaster.nl>
This sounds like it's going to be fixed soon upstream? So maybe we should just wait for the fix and update the dep rather than adding and removing a workaround. I would think the Azure SDK would have a release before Argo as well |
Co-authored-by: Anton Gilgur <4970083+agilgur5@users.noreply.github.com> Signed-off-by: Roel Arents <2691308+roelarents@users.noreply.github.com>
True, if they fix it before the next argo release then this is obsolete. If not, I do suggest this temporary workaround. Because there is another issue (that we also encountered in production) in the azure blob sdk since 1.0 (which is in argo 3.5). That has been fixed upstream already. I bumped the sdk to 1.2.1 for it. But that introduced this issue. (Probably/possibly there is a version in between with the fix for the one and not yet introducing the next. But I think using the latest version is easier.) |
This fix constraints the concurrency to 1, which is not preferable. |
yep, they fixed it. this can be closed. |
Looks like it was fixed in Azure/azure-sdk-for-go#22334 which was released in This PR has been replaced by #12667 as such |
workaround for Azure/azure-sdk-for-go#22156
Motivation
Recently bumped the azure sdk for go for blob storage. But it has an issue with concurrent downloads.
Modifications
Set concurrency when downloading artifacts/files/blobs from azure blob storage to 1 (instead of 5)
Verification
Tested in production with a patch on 3.5.4. We suffered from this issue before (hanging workflow steps) and now not anymore.