Replies: 3 comments
-
It's an interesting question. Who is responsible for "how the experiment should run?". Is it the experiment author or the operator? There is a We could also support an env based approach. |
Beta Was this translation helpful? Give feedback.
-
Yeah I think there is a balance here. Where my thoughts on this came from is lets say hypothetically I am at an enterprise org using GitHub enterprise. My platform team provides a GitHub action to run They don't want to have to define different actions for each flag combination. Similarly they don't want to clutter the action with lots of conditionals that allow X or Y flags to be enabled. So my thoughts here are anything which you can define as a command line switch or flag should also be able to be defined in configuration too. That way I can define the behaviour I want in my experiment like It also means I could share the experiment with someone and they could just run it without having to know exactly what flags I used in the CLI. My caveat to that is CLI flags would always win. So if I have an experiment which has configuration on runtime behaviour and I just run Allowing CLI flags to override runtime config has two benefits, Firstly it keeps the solution backwards compatible which is command line flags dictate the runtime behaviour so we don't break any existing setup. Secondly it allows the experiment operator the ability to take an experiment and reuse it changing only what they need dynamically at run time. This allows much better reuse of experiments without having to copy them to change only a single flag etc. This idea follows the ansible style flow of precedence where command line always wins. This is the opposite of terraform where configuration is king and config is opinionated and if defined then you cant change it. I think for this project the ansible approach provides much more flexibility and reuse of experiments. An example of this might be I want to define an experiment. For safety reasons I want to set the fail flag by default any time the experiment runs. This is to protect my production environments (I believe that you should always write your base case for production and then extend it for any other env). So when it runs in prod and there is an issue it will fail immediately. However I might want to run that same experiment in my dev env but I want to let it keep failing to see if they system recovers and enters a state of stability after 5 minutes. So here I would want to override the experiments fail-fast flag and let it keep failing in dev. So i can still have a single experiment which will have default run time config defined in the experiment but I can override that config explicitly by passing the flags at the command line. |
Beta Was this translation helpful? Give feedback.
-
If I'm not mistaken the CLI flags already take precedence over the experiment ones. |
Beta Was this translation helpful? Give feedback.
-
As a devops engineer I am normally running my workload through a CICD pipeline. My pipeline needs to be generic and flexible enough to run a variety of workloads. While it is possible, sometimes dynamically passing command line flags or switches with bash doesnt work so easy. And you then need to build out arrays an have if statements to push flags on the array etc.
Life would be much simpler if all the available command line flags and switches could also be provide programmatically, through config files or env vars or both.
This gives the added value that I can then store all my configuration under source control and not have the person or pipeline running the experiment having to be aware of all the flags or options I expect it to run with.
Beta Was this translation helpful? Give feedback.
All reactions