You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
When trying to use start-batch I wanted to copy all files beginning with a prefix. Prefix in this case is how prefix is applied with this command:
az storage blob list --container-name $container --account-name testbullclip --prefix $_.name
This appears to do a performant query against blob storage. I get results in seconds. Start-batch doesn't this. It has --pattern, what --pattern does I couldn't say but it appears to be scanning every single blob name. My container has 1.34M blobs making the implementation of --pattern of no use at all.
Describe the solution you'd like
Use whatever magic the blob list --prefix arg is using with a new arg for start-batch called --prefix.
Describe alternatives you've considered
I've tried using azcopy but it has the same sort of implementation with --include-pattern.
The text was updated successfully, but these errors were encountered:
Hi @worldspawn, thanks for your feedback. Two things to do in next Sprint:
Find a way to add --prefix for az storage blob copy start-batch
Integrate Azcopy v10.3 and add --include/exclude related flags to az storage copy.
We may do both of the above improvements or one of them, but anyway we will provide a way to copy with prefix.
Since Azcopy v10.3 will bring some breaking changes, we plan to add breaking changes information in this sprint in PR #11546 and will do the change in next sprint.
Is your feature request related to a problem? Please describe.
When trying to use start-batch I wanted to copy all files beginning with a prefix. Prefix in this case is how prefix is applied with this command:
This appears to do a performant query against blob storage. I get results in seconds. Start-batch doesn't this. It has
--pattern
, what--pattern
does I couldn't say but it appears to be scanning every single blob name. My container has 1.34M blobs making the implementation of --pattern of no use at all.Describe the solution you'd like
Use whatever magic the blob list --prefix arg is using with a new arg for start-batch called --prefix.
Describe alternatives you've considered
I've tried using azcopy but it has the same sort of implementation with --include-pattern.
The text was updated successfully, but these errors were encountered: