feat(helm): set readOnlyRootFilesystem on CNI, more explicit templates

[michaelbeaumont]

Also make the runAsNonRoot: false more visible.

Checklist prior to review

• Link to relevant issue as well as docs and UI issues --
• This will not break child repos: it doesn't hardcode values (.e.g "kumahq" as a image registry) and it will work on Windows, system specific functions like syscall.Mkfifo have equivalent implementation on the other OS --
• Tests (Unit test, E2E tests, manual test on universal and k8s) --
• Do you need to update UPGRADE.md? --
• Does it need to be backported according to the backporting policy? --
• Do you need to explicitly set a > Changelog: entry here or add a ci/ label to run fewer/more tests?

[bartsmykla]

Hmm. Doesn't readOnlyRootFilesystem prevent CNI installer to put plugin files in appropriate places? I couldn't find any good documentation of this setting.

[michaelbeaumont]

@bartsmykla It's really only about the root filesystem, I think the plugin is installed on the volumes mounted from the host.

[bartsmykla]

Got it