Force loading provider configuration when using config.as_dict #38793
Conversation
It's not clear to me why the sensitive_config_values excludes providers when config.as_dict is used on the webserver. This has resulted in displaying sensitive provider configurations even when non-sensitive-only is used as the value of [webserver]expose_config. This fixed it but still, it's not clear to me where things went wrong.
| @@ -1439,11 +1439,13 @@ def as_dict( | |||
| ("default", self._default_values), | |||
| ("airflow.cfg", self), | |||
| ] | |||
| # Force the providers configuration to be loaded | |||
| self.configuration_description = retrieve_configuration_description() | |||
There was a problem hiding this comment.
| self.configuration_description = retrieve_configuration_description() | |
| if not self.configuration_description: | |
| self.configuration_description = retrieve_configuration_description() |
perhaps?
There was a problem hiding this comment.
It’s there just that it doesn’t have providers configuration. so this would lead to old results.
where it was set initially, it was set not to load providers configuration.
There was a problem hiding this comment.
Or rather, expanding on the comment you did add 🤦
There was a problem hiding this comment.
That's not the right fix (it will cause much slowe airflow import if we do it this way because importing airflow is initializing configuraiton (this is why by default it does not load provider configuration).
We are lazy-loading provider's configuration by ProvidersManager.iniitialize_providers_configuration(). I have a fix coming with modified tests. I noticed that teh test that was retrieving config for non-sensitive values was not really testing if configuration for non-sensitive values was working - it checked only if as_dict() was called with include_sensitive=False (but it did not really check if sensitive values were replaced).
|
Could we add a test to cover that this doesn't break again please? |
|
Better fix: #38795 |
It's not clear to me why the sensitive_config_values excludes providers when config.as_dict is used on the webserver. This has resulted in displaying sensitive provider configurations even when non-sensitive-only is used as the value of [webserver]expose_config.
This fixed it but still, it's not clear to me where things went wrong.
cc @potiuk