remote-config: rename FEATURES test to ASM_FEATURES #565
Conversation
|
If i'm not wrong, these scnearios are not in the CI, is it expected ? |
|
It looks like the file |
It looks like it is not used, an artifact from when we didn't have multiple test scenarios. I'll make another PR to remove it. |
@cbeauchesne - We don't want disabling APPSEC to cause issues with other tests but I think everything is fine since we isolated the remote config tests to independent executions, if I'm understanding "scenarios" correctly. |
Scenario runs are totally isolated (it's their purpose :) ) My point is that those two scenario are never executed in the system tests ci, meaning that we can break them anyday without knowing it. Can you add them in |
I think you forgot a "not" 😄 |
indeed |
|
Can definitely do this, but might require some more discussion. We have a single "scenario" but multiple flavors of it. (Basically one for each product, and then for each product the "cached" and "no cached" version) We can't run multiple flavors together because of the stateful nature of remote config. Thus to test everything we'd need this block for all possible "flavors": - name: Run REMOTE_CONFIG_MOCKED_BACKEND_<PRODUCT>_<FLAVOR> scenario
run: ./run.sh REMOTE_CONFIG_MOCKED_BACKEND_<PRODUCT>_<FLAVOR>
env:
DD_API_KEY: ${{ secrets.DD_API_KEY }}Does that sound right/ok to you? If so I'll draw this up in another PR. |
|
Hello, |
|
@ameske how many product/flavor do you have ? |
ameske
force-pushed
the
kyle.ames/features-to-asm-features
branch
from
f6b2e90 to
0219948
Compare
|
I just cleaned up the merge conflict, assuming CI/CD passes here let's go ahead and merge this so that discussion around adding the scenarios to the CI doesn't slow down progress. I'll open a PR with the naive route of registering scenarios with CI/CD and we can take a look and see if there's a better way. |
|
Trying to help move this along, but I'm not sure what's going on with all the failures at the moment. I'm trying to reproduce locally but not having any success - the runner is just hanging. |
|
I'll have a look |
The product FEATURES was renamed to ASM_FEATURES prior to going prod, so this just updates all references of the old name so that new tracers can test against the product that will go live.
Within tracers, appsec being enabled via the DD_APPSEC_ENABLED environment variable disables remote configuration. In order to be able to leverage these tests, we need that environment variable set to false.
Co-authored-by: simon-id <simon.id@datadoghq.com>
Co-authored-by: simon-id <simon.id@datadoghq.com>
ameske
force-pushed
the
kyle.ames/features-to-asm-features
branch
from
43d5b2f to
02cb3eb
Compare
|
Seems the current failures all stem from an issue on main, and not this PR. @cbeauchesne - we're all set with this specific change so we're just needing a system-tests core review now. |
Co-authored-by: simon-id <simon.id@datadoghq.com>
|
Status update: the file |
|
Status update: the ruby lib crashes when passed an empty string throught |
The product FEATURES was renamed to ASM_FEATURES prior to going prod, so this just updates all references of the old name so that new tracers can test against the product that will go live.
Closes: #558
Closes: #559
Description
Please include a short summary of the change
Check list
Your PR is not ready to be reviewed? Please save it as a draft 🙏
Yes to all? Feel free to merge it whenever you want ❤️