Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blob prefixes not available until you iterate the files #863

Closed
eavonius opened this issue Sep 10, 2022 · 2 comments · Fixed by #866
Closed

Blob prefixes not available until you iterate the files #863

eavonius opened this issue Sep 10, 2022 · 2 comments · Fixed by #866
Assignees
Labels
api: storage Issues related to the googleapis/python-storage API. samples Issues that are directly related to samples. type: question Request for information or clarification. Not an issue.

Comments

@eavonius
Copy link

Environment details

  • OS type and version: Windows 10
  • Python version: 3.9.13
  • pip version: 22.0.4
  • google-cloud-storage version: 2.5.0

Steps to reproduce

  1. Download the sample code for listing files with a prefix.
  2. Comment out the lines 56-58 that print the names of each blob.
  3. Create a google cloud storage bucket with a subfolder, and a file within that subfolder.
  4. Run the sample where it specifies the subfolder as the prefix with slash at the end (as shown) and the delimiter of '/'.
  5. Nothing is displayed (the prefixes are empty).
  6. Un-comment the lines that print the names of each blob.
  7. Run it again.
  8. The prefixes are displayed.

It appears you cannot list the prefixes unless you iterate the files first. This seems strange.

If it's intended, maybe this library should pre-iterate the list so this doesn't happen? Or at least document that this is expected?

@product-auto-label product-auto-label bot added the api: storage Issues related to the googleapis/python-storage API. label Sep 10, 2022
@shaffeeullah shaffeeullah added samples Issues that are directly related to samples. type: question Request for information or clarification. Not an issue. labels Sep 12, 2022
@shaffeeullah shaffeeullah self-assigned this Sep 12, 2022
@cojenco cojenco assigned cojenco and unassigned shaffeeullah Sep 13, 2022
@cojenco
Copy link
Contributor

cojenco commented Sep 13, 2022

Hi eavonius, thank you for the question. You are right about the iterator has to be called first in order to access the list of prefixes. This is intended, as list_blobs operation returns an iterator, and the rpc is done with a response back only when you iterate through the iterator

Another way that works too:

blobs = storage_client.list_blobs(bucket_name, prefix=prefix, delimiter=delimiter)

list(blobs)
print(blobs.prefixes)

I agree this could stand to be clarified in the documentation. I'll update the official documentation and sample notes.

@eavonius
Copy link
Author

@cojenco thanks, makes sense to me. Yep a little note in the documentation would go a long way. Was very confusing at first. Thanks for the help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the googleapis/python-storage API. samples Issues that are directly related to samples. type: question Request for information or clarification. Not an issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants