Skip to content
This repository has been archived by the owner on Nov 14, 2023. It is now read-only.

Commit

Permalink
Fix getting CS content when S3 bucket contains created manually direc…
Browse files Browse the repository at this point in the history
…tories (cvat-ai#6997)

<!-- Raise an issue to propose your change
(https://github.com/opencv/cvat/issues).
It helps to avoid duplication of efforts from multiple independent
contributors.
Discuss your ideas with maintainers to be sure that changes will be
approved and merged.
Read the [Contribution
guide](https://opencv.github.io/cvat/docs/contributing/). -->

<!-- Provide a general summary of your changes in the Title above -->

### Motivation and context
<!-- Why is this change required? What problem does it solve? If it
fixes an open
issue, please link to the issue here. Describe your changes in detail,
add
screenshots. -->
This PR fixes retrieving cloud storage content for an AWS S3 bucket
containing manually created "directories" (AWS ListObjectsV2 API returns
such objects within the content).

![image](https://github.com/opencv/cvat/assets/49038720/af956beb-9a88-4391-aa17-2a9a0629e16c)

A similar problem can be found
[here](https://stackoverflow.com/questions/75620230/aws-s3-listobjectsv2-returns-folder-as-an-object).
In this case, I prefer to simply filter keys from keys with a trailing
slash, rather than filter from objects with 0 size.
### How has this been tested?
Manually
### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable for some reason, then ~~explicitly
strikethrough~~ the whole
line. If you don't do that, GitHub will show incorrect progress for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [x] I submit my changes into the `develop` branch
- [x] I have created a changelog fragment <!-- see top comment in
CHANGELOG.md -->
~~- [ ] I have updated the documentation accordingly~~
~~- [ ] I have added tests to cover my changes~~
~~- [ ] I have linked related issues (see [GitHub docs](

https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))~~
~~- [ ] I have increased versions of npm packages if it is necessary

([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning),

[cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning),

[cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning)
and

[cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))~~

### License

- [x] I submit _my code changes_ under the same [MIT License](
https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the
project.
  Feel free to contact the maintainers if that's a concern.
  • Loading branch information
Marishka17 authored and mikhail-treskin committed Oct 25, 2023
1 parent 93d15f1 commit 084cbf2
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 1 deletion.
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
### Fixed

- Getting CS content when S3 bucket contains manually created directories
(<https://github.com/opencv/cvat/pull/6997>)
2 changes: 1 addition & 1 deletion cvat/apps/engine/cloud_provider.py
Original file line number Diff line number Diff line change
Expand Up @@ -467,7 +467,7 @@ def _list_raw_content_on_one_page(
**({'Prefix': prefix} if prefix else {}),
**({'ContinuationToken': next_token} if next_token else {}),
)
files = [f['Key'] for f in response.get('Contents', [])]
files = [f['Key'] for f in response.get('Contents', []) if not f['Key'].endswith('/')]
directories = [p['Prefix'] for p in response.get('CommonPrefixes', [])]

return {
Expand Down

0 comments on commit 084cbf2

Please sign in to comment.