Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kube-prometheus plugin re-apply still shows change #231

Closed
IronCore864 opened this issue Feb 25, 2022 · 2 comments · Fixed by #234
Closed

kube-prometheus plugin re-apply still shows change #231

IronCore864 opened this issue Feb 25, 2022 · 2 comments · Fixed by #234
Assignees
@daniel-hutao
Labels
bug Something isn't working

Comments

@IronCore864
Copy link
Member

Describe the Bug

Apply kube-prometheus a second time still shows change even though it's installed and running well.

How to Reproduce

Config:

---
tools:
- name: kube-prometheus-dev
  plugin:
    kind: kube-prometheus
    version: 0.2.0
  options:
    create_namespace: true
    repo:
      name: prometheus-community
      url: https://prometheus-community.github.io/helm-charts
    chart:
      chart_name: prometheus-community/kube-prometheus-stack
      release_name: dev
      namespace: monitoring
      wait: true
      timeout: 5m
      upgradeCRDs: true

then run dtm apply. kube-prometheus is installed successfully, and the state is:

kube-prometheus-dev_kube-prometheus:
  name: kube-prometheus-dev
  plugin:
    kind: kube-prometheus
    version: 0.2.0
  options:
    chart:
      chart_name: prometheus-community/kube-prometheus-stack
      namespace: monitoring
      release_name: dev
      timeout: 5m
      upgradeCRDs: true
      wait: true
    create_namespace: true
    repo:
      name: prometheus-community
      url: https://prometheus-community.github.io/helm-charts
  resource:
    workflows: |
      deployments:
        - name: dev-grafana
          ready: true
        - name: dev-kube-prometheus-stack-operator
          ready: true
        - name: dev-kube-state-metrics
          ready: true
      daemonsets:
        - name: dev-prometheus-node-exporter
          ready: true
      statefulsets:
        - name: alertmanager-dev-kube-prometheus-stack-alertmanager
          ready: true
        - name: prometheus-dev-kube-prometheus-stack-prometheus
          ready: true

then run dtm -d apply again (with debugging log). Part of the log:

2022-02-25 11:10:06 ℹ [INFO]  The deployment dev-grafana is ready.
2022-02-25 11:10:06 ℹ [INFO]  The deployment dev-kube-prometheus-stack-operator is ready.
2022-02-25 11:10:06 ℹ [INFO]  The deployment dev-kube-state-metrics is ready.
2022-02-25 11:10:06 ℹ [INFO]  The daemonset dev-prometheus-node-exporter is ready.
2022-02-25 11:10:06 ℹ [INFO]  The statefulset alertmanager-dev-kube-prometheus-stack-alertmanager is ready.
2022-02-25 11:10:06 ℹ [INFO]  The statefulset prometheus-dev-kube-prometheus-stack-prometheus is ready.
2022-02-25 11:10:06 ℹ [DEBUG]  All resources read ready.
2022-02-25 11:10:06 ℹ [DEBUG]    map[string]interface{}{
- 	"workflows": string("{}\n"),
+ 	"workflows": string(
+ 		"""
+ 		deployments:
+ 		  - name: dev-grafana
+ 		    ready: true
+ 		  - name: dev-kube-prometheus-stack-operator
+ 		    ready: true
+ 		  - name: dev-kube-state-metrics
+ 		    ready: true
+ 		daemonsets:
+ 		  - name: dev-prometheus-node-exporter
+ 		    ready: true
+ 		statefulsets:
+ 		  - name: alertmanager-dev-kube-prometheus-stack-alertmanager
+ 		    ready: true
+ 		  - name: prometheus-dev-kube-prometheus-stack-prometheus
+ 		    ready: true
+ 		"""
+ 	),
  }

2022-02-25 11:10:06 ℹ [DEBUG]  Tool kube-prometheus-dev kube-prometheus resource drifted from state.
2022-02-25 11:10:06 ℹ [INFO]  Change added: kube-prometheus-dev -> Update
@IronCore864 IronCore864 added the bug Something isn't working label Feb 25, 2022
@daniel-hutao
Copy link
Member

config.yaml

---
tools:
  - name: kube-prometheus-dev
    plugin:
      kind: kube-prometheus
      version: 0.2.0
    options:
      create_namespace: true
      repo:
        name: prometheus-community
        url: https://prometheus-community.github.io/helm-charts
      chart:
        chart_name: prometheus-community/kube-prometheus-stack
        release_name: dev
        namespace: monitoring
        wait: false
        timeout: 5m
        upgradeCRDs: true

command

./dtm-darwin-arm64 apply -y -f config-kube-prometheus.yaml --debug

then, the devstream.state is

kube-prometheus-dev_kube-prometheus:
  name: kube-prometheus-dev
  plugin:
    kind: kube-prometheus
    version: 0.2.0
  options:
    chart:
      chart_name: prometheus-community/kube-prometheus-stack
      namespace: monitoring
      release_name: dev
      timeout: 5m
      upgradeCRDs: true
      wait: false
    create_namespace: true
    repo:
      name: prometheus-community
      url: https://prometheus-community.github.io/helm-charts
  resource:
    workflows: |
      deployments:
        - name: dev-grafana
          ready: true
        - name: dev-kube-prometheus-stack-operator
          ready: true
        - name: dev-kube-state-metrics
          ready: true
      daemonsets:
        - name: dev-prometheus-node-exporter
          ready: true
      statefulsets:
        - name: alertmanager-dev-kube-prometheus-stack-alertmanager
          ready: true
        - name: prometheus-dev-kube-prometheus-stack-prometheus
          ready: true

the real resources state:

# kgpo -n monitoring
NAME                                                    READY   STATUS    RESTARTS   AGE
alertmanager-dev-kube-prometheus-stack-alertmanager-0   2/2     Running   0          7m
dev-grafana-c6b75795d-8ldw7                             3/3     Running   0          7m1s
dev-kube-prometheus-stack-operator-569bb84865-jmpx9     1/1     Running   0          7m1s
dev-kube-state-metrics-657b8d74d5-8gs8s                 1/1     Running   0          7m1s
dev-prometheus-node-exporter-nqncp                      1/1     Running   0          7m1s
prometheus-dev-kube-prometheus-stack-prometheus-0       2/2     Running   0          7m

apply again

# ./dtm-darwin-arm64 apply -y -f config-kube-prometheus.yaml --debug

2022-02-25 13:05:35 ℹ [INFO]  Log level is: debug.
2022-02-25 13:05:35 ℹ [INFO]  Apply started.
2022-02-25 13:05:35 ℹ [DEBUG]  Config file: 
---
tools:
  - name: kube-prometheus-dev
    plugin:
      kind: kube-prometheus
      version: 0.2.0
    options:
      create_namespace: true
      repo:
        name: prometheus-community
        url: https://prometheus-community.github.io/helm-charts
      chart:
        chart_name: prometheus-community/kube-prometheus-stack
        release_name: dev
        namespace: monitoring
        wait: false
        timeout: 5m
        upgradeCRDs: true


2022-02-25 13:05:35 ℹ [INFO]  Using dir <.devstream> to store plugins.
2022-02-25 13:05:35 ℹ [DEBUG]  The global manager m is not initialized.
2022-02-25 13:05:35 ℹ [DEBUG]  Got a state from the backend: kube-prometheus-dev_kube-prometheus -> {kube-prometheus-dev {kube-prometheus 0.2.0} map[chart:map[chart_name:prometheus-community/kube-prometheus-stack namespace:monitoring release_name:dev timeout:5m upgradeCRDs:true wait:false] create_namespace:true repo:map[name:prometheus-community url:https://prometheus-community.github.io/helm-charts]] map[workflows:deployments:
  - name: dev-grafana
    ready: true
  - name: dev-kube-prometheus-stack-operator
    ready: true
  - name: dev-kube-state-metrics
    ready: true
daemonsets:
  - name: dev-prometheus-node-exporter
    ready: true
statefulsets:
  - name: alertmanager-dev-kube-prometheus-stack-alertmanager
    ready: true
  - name: prometheus-dev-kube-prometheus-stack-prometheus
    ready: true
]}.
2022-02-25 13:05:36 ℹ [INFO]  The deployment dev-grafana is ready.
2022-02-25 13:05:36 ℹ [INFO]  The deployment dev-kube-prometheus-stack-operator is ready.
2022-02-25 13:05:36 ℹ [INFO]  The deployment dev-kube-state-metrics is ready.
2022-02-25 13:05:36 ℹ [INFO]  The daemonset dev-prometheus-node-exporter is ready.
2022-02-25 13:05:36 ℹ [INFO]  The statefulset alertmanager-dev-kube-prometheus-stack-alertmanager is ready.
2022-02-25 13:05:36 ℹ [INFO]  The statefulset prometheus-dev-kube-prometheus-stack-prometheus is ready.
2022-02-25 13:05:36 ℹ [DEBUG]  All resources read ready.
2022-02-25 13:05:36 ℹ [DEBUG]    map[string]interface{}{
-       "workflows": string("{}\n"),
+       "workflows": string(
+               """
+               deployments:
+                 - name: dev-grafana
+                   ready: true
+                 - name: dev-kube-prometheus-stack-operator
+                   ready: true
+                 - name: dev-kube-state-metrics
+                   ready: true
+               daemonsets:
+                 - name: dev-prometheus-node-exporter
+                   ready: true
+               statefulsets:
+                 - name: alertmanager-dev-kube-prometheus-stack-alertmanager
+                   ready: true
+                 - name: prometheus-dev-kube-prometheus-stack-prometheus
+                   ready: true
+               """
+       ),
  }

...
...
...

@daniel-hutao
Copy link
Member

I've got it.

@daniel-hutao daniel-hutao mentioned this issue Feb 25, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@daniel-hutao daniel-hutao

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix: helm-style plugins Read() don't return the state with ready workflows
2 participants
@daniel-hutao @IronCore864