-
Notifications
You must be signed in to change notification settings - Fork 59
Turning on strictMode for propeller and manager config #505
base: master
Are you sure you want to change the base?
Conversation
Signed-off-by: pmahindrakar-oss <prafulla.mahindrakar@gmail.com>
036bf04
to
7790414
Compare
@pmahindrakar-oss I do not see |
yes i agree @hamersaw. I was trying to address the one where i saw this initially . But i wanted to understand the reason we don't have this turned on by default for all flyte repos. By definition
The way this was discovered was flytepropeller accepted a config like this one though the key
So turning on this helps as it will immediately throw an error once we turn on strictMode to show something like this This seems to be right error to catch earlier than to find issues with running flyte instances. The reason we could want non-strict behavior i assume is in case of maintaining backward compatible configs in which we delete deprecated keys in the structure but we expect to support reading older configs without erroring out which i find error prone too or i might be missing something. |
@pmahindrakar-oss had a chance to take a deeper look into this. I can see the benefit but I am concerned that this will break existing deployments in a number of scenarios. For example, propeller manager requires additional configuration. This is currently added to the propeller configmap. In this case I think it would be VERY nice to have this feature in all of our repos, but I can't see a way to enable it with backwards compatibility. Do you have any ideas? |
@hamersaw i see your point. I think flyteadmin does deal with similar issue but it can handle this problem since it treats flyteadmin configuration as one single entity and scheduler, clusterresourcecontroller etc as subconfigurations. So when flyteadmin, scheduler or clusterresourcecontroller loads the configurations , it loads the entire flyteadmin config for the app even though it might just use a section of it.
So if we turn on strictMode on all the components will still continue to function correctly I think if we define the way flyteadmin does it then that should work and also be backward compatible . Let me know what you think. |
This won't work with single binary though right? Because we have the configuration for all of the components in a single file. |
yeah you are right @hamersaw that we wont be able to change for single binary but we can keep loading the config in non strict mode for it https://github.com/flyteorg/flyte/blob/master/cmd/single/root.go#L67 The current change will only affect propeller when its run in an independent mode |
Signed-off-by: pmahindrakar-oss prafulla.mahindrakar@gmail.com
TL;DR
Type
Are all requirements met?
Complete description
How did you fix the bug, make the feature etc. Link to any design docs etc
Tracking Issue
fixes flyteorg/flyte#3113
Follow-up issue
NA