AzureStorageExplorer "Preserve Blob Tags" feature fails to detect permission issue, resulting in failed copy/paste across storage accounts

[lmsurpre]

Preflight Checklist

• I have installed the latest version of Storage Explorer.
• I have checked existing resources, including the troubleshooting guide and the release notes.
• I have searched for similar issues.

Storage Explorer Version

1.30.0 and above (reproduced in 1.32.0)

Regression From

1.29.2

Architecture

all (reproduced in x64 and amd64)

Storage Explorer Build Number

20231101.3

Platform

All

OS Version

No response

Bug Description

Copy/paste of blob from one storage account to another results in UnexpectedQuit.

Two variants (depending on the properties of the target storage account:

• UnexpectedQuit (used SAS, discovery not completed)
• UnexpectedQuit (used Oauth, discovery not completed)

The error is similar to #7014 but I think the real issue is that the
Detect value for the "Preserve Blob Tags" setting is not working as designed.

Steps to Reproduce

Given a user that has Storage Blob Data Contributor role (and not Data Owner) on 2 different storage accounts within a single subscription,
and the default AzureStorageExplorer "Preserve Blob Tags" value of "Detect":

1. navigate to a container within storage account A and copy a blob
2. navigate to a container within storage account B and try to paste it

Additionally, in my case, shared keys (aka access keys) are disabled in the source storage account (storage account A above).

Actual Experience

The Activites view shows that the azcopy command has failed with the cryptic "UnexpectedQuit" message.

Expected Experience

If set to 'Detect', AzCopy will only preserve tags if the source and the destination support tags, and if they have the permissions required to preserve tags.

Given that description, I expect the copy/paste to work and for it to:
A. preserve tags if the user has permissions to the get/set tags API
B. not preserve tags if the user does not have this permission

Additionally, if this is expected to fail, then a clear error message that indicates which permission I'm missing would be nice.

Additional Context

No response

[JasonYeMSFT]

@lmsurpre The "Detect" option has limitations when the credential doesn't clearly tell us if it has the necessary permission to paste tags (e.g. OAuth tokens, SAS token with access policy). We don't want to silently drop the tags due to permission issues because it may result in unexpected results for some users. In your case where we cannot tell the permissions, we should have shown a dialog asking you whether you want to attempt preserving tags or not. Please let us know if you didn't see the dialog. Right now the "Detect" option is mostly benefiting people using SAS tokens because it may suppress the dialog after parsing the SAS permissions.

The error reporting is definitely something we should improve upon.

[lmsurpre]

I see that dialog in the case of UnexpectedQuit (**used Oauth**, discovery not completed) but NOT in the case of UnexpectedQuit (used SAS, discovery not completed).
In my case, it seems to use one vs the other depending on whether the TARGET storage account has shared keys enabled.

The dialog in the Oauth case is the only reason I managed to figure out what the heck was going on (because it failed in a similar way to the SAS case with it checked but succeeded if I unchecked it).

[JasonYeMSFT]

What's the permission of your SAS tokens in the case of UnexpectedQuit (used SAS, discovery not completed)?

[lmsurpre]

What's the permission of your SAS tokens in the case of UnexpectedQuit (used SAS, discovery not completed)?

Sorry, but i'm not sure how to check that...i'm just using the Azure Storage Explorer "copy / paste" buttons and its creating the SAS token under the covers.

Here is the azcopy commands it generated:

export AZCOPY_CONCURRENCY_VALUE=AUTO;
./azcopy copy "https://REDACTED.blob.core.windows.net/REDACTED?sv=2023-01-03&st=2023-11-06T22%3A27%3A41Z&se=2023-11-13T22%3A42%3A41Z&skoid=REDACTED&sktid=REDACTED&skt=2023-11-06T22%3A27%3A41Z&ske=2023-11-13T22%3A42%3A41Z&sks=b&skv=2023-01-03&sr=c&sp=rlt&sig=REDACTED" "https://REDACTED.blob.core.windows.net/REDACTED?sv=2023-01-03&se=2023-12-06T22%3A44%3A13Z&sr=c&sp=rwlt&sig=REDACTED" --overwrite=prompt --from-to=BlobBlob --s2s-preserve-access-tier=false --check-length=true --include-directory-stub=false --s2s-preserve-blob-tags=true --recursive --log-level=INFO;
unset AZCOPY_CONCURRENCY_VALUE;

[lmsurpre]

Oh, I guess that sp query param is carrying the permissions. So sp=rlt would be read, list, and tag.
My user account has "Storage Blob Data Contributor" but not "Storage Blob Data Owner", and so I don't think it should be allowed to create a user delegated SAS that includes the tag permission. Maybe the Storage Account API is letting you generate a SAS token that includes the tag permission but then failing when you try to actually use it?

[JasonYeMSFT]

I see what's going on now. User delegation SAS delegates the authorization to the AAD service when the SAS token is used, which might be the issue here. Unfortunately in that case, there is no way for us to gauge whether we can preserve tags or not. We should try to make AzCopy improve error reporting in this scenario to better indicate what's going on.

By the way, the next time you see the "Unexpected Quit" type of error, you may be able to find out more details in the AzCopy log. We offer a "Go to AzCopy logs" action on the activity, which opens AzCopy log directory. If you open it immediately after the failed job, the most recent log file will be for that job.

[JasonYeMSFT]

I should try to see if we can make Storage Explorer detect user delegation SAS as credentials with unknown permissions.

[JasonYeMSFT]

Starting from 1.33.0 Storage Explorer will treat user delegation SAS as credential with unknown permissions and prompt to let user pick whether to preserve tags on demand.