Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[s3] Add client_config configuration setting #1386

Merged
merged 1 commit into from
Apr 25, 2024
Merged

Conversation

jschneier
Copy link
Owner

@jschneier jschneier commented Apr 25, 2024

Closes #1337

@jschneier jschneier merged commit d74a86b into master Apr 25, 2024
19 checks passed
@jschneier jschneier deleted the josh/s3-client-config branch April 25, 2024 03:29
)
self.client_config = self.config

if self.client_config is None:
Copy link

@mgzwarrior mgzwarrior Jun 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking that there is an unintended logical change here that makes self.client_config a required field, defaulted to None. See here for more - #1410

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the logic here.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I am following. The get_default_settings() method here seems to return {} by default.

In our case, we have overridden this method as such:

    def get_default_settings(self):
        return {
            "access_key": setting(f'{self.setting_prefix}_ACCESS_KEY_ID'),
            "secret_key": setting(f'{self.setting_prefix}_SECRET_ACCESS_KEY'),
            ...,
        }

However, we don't have a setting for client_config because this is new. This seems to be the root cause of the error: object has no attribute 'client_config'. One solution I could try is to add a dummy attribute to the above method and default to None, but I shouldn't have to if the setting is not required, right?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't provide backwards compat guarantees on get_default_settings as that is not a public API. If you want this to reliably work I'd recommend something like

settings = super().get_default_settings()
...your changes here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Threaded parallel uploads on S3
2 participants