Add v2 migrations steps

[pmalek-sumo]

Description

Add v2 migrations steps

Testing performed

• ci/build.sh
• Redeploy fluentd and fluentd-events pods
• Confirm events, logs, and metrics are coming in

[sumo-drosiek]

please remember to upgrade it when the release 2.0 branch will be created

[pmalek-sumo]

#1174

[sumo-drosiek]

could we test against --reuse-values? AFAIK changes in values.yaml won't be applied

[kasia-kujawa]

Maybe before this step we should add chmod +x upgrade-2.0.0.sh. It is quite obvious but usually this step appears in instructions, e.g. here and here

[kasia-kujawa]

I'm testing on vagrant VM created from release-v1.3 branch, commit: 2fac768.
I see following error:

./upgrade-2.0.0.sh current_values.yaml
[INFO]    Mapping fluentd.image.repository into:
[INFO]    Mapping fluentd.image.tag into:
[INFO]    Mapping image.repository into:
[INFO]    - fluentd.image.repository
[INFO]    - sumologic.setup.job.image.repository
[INFO]    Mapping image.tag into:
[INFO]    - fluentd.image.tag
[INFO]    - sumologic.setup.job.image.tag

[INFO]    Migrating prometheus-config-reloader container to config-reloader in prometheusSpec
Error: unknown flag: --arrays

yq version:

$ yq --version
yq version 3.3.0

[pmalek-sumo]

Good point

[pmalek-sumo]

You'll need yq at least at version 3.4.0

The following handles that at VM provision time but you can run it manually

sumologic-kubernetes-collection/vagrant/provision.sh

Lines 53 to 56 in 63c6cf3

	# install yq with access to file structure
	curl https://github.com/mikefarah/yq/releases/download/3.4.1/yq_linux_amd64 -L -o /usr/local/bin/yq-3.4.1
	chmod +x /usr/local/bin/yq-3.4.1
	ln -s /usr/local/bin/yq-3.4.1 /usr/local/bin/yq

[kasia-kujawa]

I'm testing from release-v1.3 branch, https://github.com/SumoLogic/sumologic-kubernetes-collection/blob/release-v1.3/vagrant/provision.sh#L56

May we mention required yq version in this instruction?

[pmalek-sumo]

Ok, apparently I missed and incorrectly changed the v2 update script

it should be fixed (i.e. user will get a notification about required yq version) via:

https://github.com/SumoLogic/sumologic-kubernetes-collection/pull/1172/files#diff-e010a115e1f3b27c1977c58d40e6aad5860cb28d7beb501161fb7d4a9358e6cc

[kasia-kujawa]

As 2.0.0 release is not available yet. I tested with dev version - 2.0.0-dev.0-133-g6e362ae, so I added dev helm repo:

$ helm repo list
NAME         	URL
sumologic    	https://sumologic.github.io/sumologic-kubernetes-collection
sumologic-dev	https://sumologic.github.io/sumologic-kubernetes-collection/dev

and I see following error:

$ helm upgrade collection sumologic-dev/sumologic --version=2.0.0-dev.0-133-g6e362ae -f new_values.yaml -n sumologic
Error: UPGRADE FAILED: cannot patch "collection-fluent-bit" with kind DaemonSet: DaemonSet.apps "collection-fluent-bit" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"collection", "app.kubernetes.io/name":"fluent-bit"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable && cannot patch "collection-sumologic-fluentd-events" with kind StatefulSet: StatefulSet.apps "collection-sumologic-fluentd-events" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden && cannot patch "collection-sumologic-fluentd-metrics" with kind StatefulSet: StatefulSet.apps "collection-sumologic-fluentd-metrics" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden && cannot patch "collection-sumologic-fluentd-logs" with kind StatefulSet: StatefulSet.apps "collection-sumologic-fluentd-logs" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden

[pmalek-sumo]

So this is most likely connected to labels and/or statefulset's volume claim when fluentd.persistence is enabled.

sumologic-kubernetes-collection/deploy/helm/sumologic/templates/statefulset.yaml

Lines 151 to 161 in d1ed616

	{{- if .Values.fluentd.persistence.enabled }}
	volumeClaimTemplates:
	- metadata:
	name: buffer
	spec:
	accessModes: [{{ .Values.fluentd.persistence.accessMode }}]
	storageClassName: {{ .Values.fluentd.persistence.storageClass }}
	resources:
	requests:
	storage: {{ .Values.fluentd.persistence.size }}
	{{- end }}

I'll add warning to upgrade 2.0.0 script, make it leave persistence as disabled if user set it as such and add short doc about how to enable it.