Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename GCS resource to Bucket resource / cloud storage resource / other catchy name #1423

Closed
ghost opened this issue Oct 14, 2019 · 3 comments

Comments

@ghost
Copy link

ghost commented Oct 14, 2019

Expected Behavior

The GCS resource now supports S3 through the use of a boto configuration file. See #1361 for where this support was added for artifact uploading and comment from Andrea #1258 (comment) describing its use to get the GCS resource speaking to S3.

So the "GCS" resource now supports S3 but is still named "GCS". We should fix this to indicate it supports other bucket types.

@ghost
Copy link
Author

ghost commented Oct 14, 2019

Hm, we should also probably document its usage as well. Will create a separate issue for that.

@ghost
Copy link
Author

ghost commented Oct 15, 2019

We've decided to punt on this for the time being. Lots of churn in PipelineResources and there are cases that the boto config can't handle due to, it seems, problems in the way the s3 implementation is done in gsutil. For example, trying to use Minio in a sidecar and accessing it through localhost doesn't seem to work.

@ghost
Copy link
Author

ghost commented Dec 5, 2019

I'm going to close this issue. Given the work going on with the pipeline resources revamp / prototyping it seems more likely that we'll end up in a place where a dedicated S3 resource type could be introduced and maintained as part of the catalog or something similar instead of messing with the existing GCS resource.

@ghost ghost closed this as completed Dec 5, 2019
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

0 participants