Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use env for prometheus metrics namespace #676

Closed
wants to merge 3 commits into from

Conversation

sumo-drosiek
Copy link
Contributor

Description

Get namespace dynamically

Testing performed
  • ci/build.sh
  • Redeploy fluentd and fluentd-events pods
  • Confirm events, logs, and metrics are coming in

@sumo-drosiek sumo-drosiek added this to the v1.1 milestone May 25, 2020
@sumo-drosiek
Copy link
Contributor Author

It's strong candidate for backporting. Should I cherry-pick that on the release-1.0/0.17?

Copy link
Contributor

@perk-sumo perk-sumo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM except for one comment about autogenerated files.

<match prometheus.metrics.apiserver**>
@type sumologic
@id sumologic.endpoint.metrics.apiserver
sumo_client "k8s_1.0.0"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this is deleted?
Would you mind rebasing on current origin/master and deleting autogenerated commits (so that they can be regenerated once more)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think something with rebase goes wrong

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

solved in #672

Copy link
Contributor Author

@sumo-drosiek sumo-drosiek May 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<<<<<<< HEAD
=======

above is more interesting

@sumo-drosiek sumo-drosiek force-pushed the drosiek-dynamic-metrics-namespace branch 2 times, most recently from da735c1 to f5fbbe2 Compare May 26, 2020 08:58
Copy link
Contributor

@frankreno frankreno left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, assuming you tested locally and the prometheus config loaded correctly with the thanos sidecar replacing all the environment variables

@sumo-drosiek
Copy link
Contributor Author

Yeah, it works, but I re-tested it with random collector name and I will update app label selector also

@sumo-drosiek
Copy link
Contributor Author

I believe we should create new labels for matching the apps for prometheus, because now it won't be possible to generate it same way like in kubernetes templates:

https://github.com/SumoLogic/sumologic-kubernetes-collection/blob/master/deploy/helm/sumologic/templates/_helpers.tpl#L13-L16

@sumo-drosiek
Copy link
Contributor Author

I set the label for default configuration as static value.

*/}}
{{- define "sumologic.labels.app" -}}
{{- template "sumologic.fullname" . }}
{{- printf "collection-sumologic" }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there any upgrade risks with this change and someone who has used a different release/namespace? I believe labels are immutable and one thing I could see is helm upgrade requiring a --force flag. If so, it would be good to document in 1.1.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it requires --force flag during upgrade

Error: UPGRADE FAILED: StatefulSet.apps "falling-zorse-sumologic-fluentd-events" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden && StatefulSet.apps "falling-zorse-sumologic-fluentd-metrics" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden && StatefulSet.apps "falling-zorse-sumologic-fluentd-logs" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the template is used for all app labels, it shouldn't affect anything (not sure about changed servicemonitors definitions)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make sure we add a release note with this change and error and flag.

Copy link
Contributor Author

@sumo-drosiek sumo-drosiek May 26, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created #680 to not forget about it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

sumo-drosiek pushed a commit that referenced this pull request May 27, 2020
@sumo-drosiek
Copy link
Contributor Author

Closing due to #685 -> more clean and safe way

@sumo-drosiek sumo-drosiek deleted the drosiek-dynamic-metrics-namespace branch May 27, 2020 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants