Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flake: ScyllaDBMonitoring [It] should setup monitoring stack #1478

Open
rzetelskik opened this issue Oct 16, 2023 · 13 comments
Open

Flake: ScyllaDBMonitoring [It] should setup monitoring stack #1478

rzetelskik opened this issue Oct 16, 2023 · 13 comments
Labels
kind/flake Categorizes issue or PR as related to a flaky test. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete.

Comments

@rzetelskik
Copy link
Member

rzetelskik commented Oct 16, 2023 •
edited
Loading 

• [FAILED] [2800.181 seconds]
ScyllaDBMonitoring [It] should setup monitoring stack
github.com/scylladb/scylla-operator/test/e2e/set/scylladbmonitoring/scylladbmonitoring.go:33
...
  [FAILED] Timed out after 300.000s.
  The function passed to Eventually failed at github.com/scylladb/scylla-operator/test/e2e/set/scylladbmonitoring/scylladbmonitoring.go:150 with:
  Unexpected error:
      <*url.Error | 0xc000ba7440>: 
      Get "https://10.96.192.8:9142/api/v1/targets": EOF
      {
          Op: "Get",
          URL: "https://10.96.192.8:9142/api/v1/targets",
          Err: <*errors.errorString | 0x34a8000>{s: "EOF"},
      }
  occurred
  In [It] at: github.com/scylladb/scylla-operator/test/e2e/set/scylladbmonitoring/scylladbmonitoring.go:184 @ 10/13/23 16:13:20.552

  Full Stack Trace
    github.com/scylladb/scylla-operator/test/e2e/set/scylladbmonitoring.glob..func1.1()
    	github.com/scylladb/scylla-operator/test/e2e/set/scylladbmonitoring/scylladbmonitoring.go:184 +0x1504

https://github.com/scylladb/scylla-operator/actions/runs/6509877307/job/17682537281#step:3:5678
https://github.com/scylladb/scylla-operator/suites/17202489789/artifacts/983189835
@rzetelskik rzetelskik added the kind/flake Categorizes issue or PR as related to a flaky test. label Oct 16, 2023
@scylla-operator-bot scylla-operator-bot bot added the needs-priority Indicates a PR lacks a `priority/foo` label and requires one. label Oct 16, 2023
@rzetelskik
Copy link
Member Author

/cc @tnozicka

@rzetelskik rzetelskik changed the title [Flake] ScyllaDBMonitoring [It] should setup monitoring stack Flake: ScyllaDBMonitoring [It] should setup monitoring stack Oct 16, 2023
@tnozicka
Copy link
Member

tnozicka commented Oct 16, 2023 •
edited
Loading

it seems like registry issues or cpu starvation

Oct 13 16:13:20.581: INFO: At 2023-10-13 16:07:11 +0000 UTC - event for prometheus-basic-szsv9-0: {kubelet fv-az1109-718} Failed: Error: ErrImagePull
  Oct 13 16:13:20.581: INFO: At 2023-10-13 16:07:11 +0000 UTC - event for prometheus-basic-szsv9-0: {kubelet fv-az1109-718} Failed: Failed to pull image "quay.io/prometheus-operator/prometheus-config-reloader:v0.61.1": rpc error: code = Unknown desc = reading manifest v0.61.1 in quay.io/prometheus-operator/prometheus-config-reloader: received unexpected HTTP status: 502 Bad Gateway

@tnozicka tnozicka added priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. and removed needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Oct 16, 2023
@tnozicka tnozicka mentioned this issue Nov 9, 2023
Merged
@tnozicka
Copy link
Member

tnozicka commented Nov 9, 2023

https://prow.scylla-operator.scylladb.com/view/gs/scylla-operator-prow/pr-logs/pull/scylladb_scylla-operator/1532/pull-scylla-operator-e2e-gke-parallel/1722576840066863104#1:test-build-log.txt%3A696

@rzetelskik rzetelskik mentioned this issue Dec 22, 2023
Closed
@zimnx
Copy link
Collaborator

zimnx commented Jan 19, 2024

https://prow.scylla-operator.scylladb.com/view/gs/scylla-operator-prow/pr-logs/pull/scylladb_scylla-operator/1614/pull-scylla-operator-master-e2e-gke-parallel-clusterip/1748305604050423808

@zimnx zimnx mentioned this issue Jan 19, 2024
Open
2 tasks
@rzetelskik
Copy link
Member Author

https://prow.scylla-operator.scylladb.com/view/gs/scylla-operator-prow/pr-logs/pull/scylladb_scylla-operator/1702/pull-scylla-operator-master-e2e-gke-parallel-clusterip/1750469061491298304#1:test-build-log.txt%3A2377

@rzetelskik rzetelskik mentioned this issue Jan 25, 2024
Merged
@rzetelskik
Copy link
Member Author

https://prow.scylla-operator.scylladb.com/view/gs/scylla-operator-prow/pr-logs/pull/scylladb_scylla-operator/1810/pull-scylla-operator-v1.11-e2e-gke-parallel/1767207087982514176#1:test-build-log.txt%3A1717

@rzetelskik rzetelskik mentioned this issue Mar 11, 2024
Merged
@rzetelskik
Copy link
Member Author

https://prow.scylla-operator.scylladb.com/view/gs/scylla-operator-prow/pr-logs/pull/scylladb_scylla-operator/1861/pull-scylla-operator-master-e2e-gke-parallel/1772754205728051200#1:test-build-log.txt%3A1834

@rzetelskik rzetelskik mentioned this issue Mar 27, 2024
Merged
@tnozicka
Copy link
Member

tnozicka commented Apr 4, 2024

https://prow.scylla-operator.scylladb.com/view/gs/scylla-operator-prow/pr-logs/pull/scylladb_scylla-operator/1877/pull-scylla-operator-master-e2e-gke-parallel/1775796980677611520#1:test-build-log.txt%3A2293

@tnozicka tnozicka mentioned this issue Apr 4, 2024
Merged
@scylla-operator-bot scylla-operator-bot
Copy link
Contributor

The Scylla Operator project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 30d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle stale
  • Close this issue with /close
  • Offer to help out

/lifecycle stale

@scylla-operator-bot scylla-operator-bot bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 7, 2024
@tnozicka
Copy link
Member

tnozicka commented Jul 8, 2024

closing as stale flake

@tnozicka tnozicka closed this as completed Jul 8, 2024
@tnozicka
Copy link
Member

tnozicka commented Aug 5, 2024

https://prow.scylla-operator.scylladb.com/view/gs/scylla-operator-prow/pr-logs/pull/scylladb_scylla-operator/2053/pull-scylla-operator-master-e2e-gke-parallel/1819469733397794816#1:test-build-log.txt%3A3043

@tnozicka tnozicka reopened this Aug 5, 2024
@tnozicka tnozicka mentioned this issue Aug 5, 2024
Merged
@tnozicka
Copy link
Member 

[FAILED] Timed out after 600.000s.
  The function passed to Eventually failed at github.com/scylladb/scylla-operator/test/e2e/set/scylladbmonitoring/scylladbmonitoring.go:256 with:
  Unexpected error:
      <*url.Error | 0xc000571c20>: 
      Get "https://admin/:***@10.112.190.200:9142/api/search?limit=1000&page=1&type=dash-db": EOF
      {
          Op: "Get",
          URL: "https://admin/:***@10.112.190.200:9142/api/search?limit=1000&page=1&type=dash-db",
          Err: <*errors.errorString | 0x4f779a0>{s: "EOF"},
      }
  occurred
  In [It] at: github.com/scylladb/scylla-operator/test/e2e/set/scylladbmonitoring/scylladbmonitoring.go:258 @ 08/19/24 06:26:49.851
  Full Stack Trace
    github.com/scylladb/scylla-operator/test/e2e/set/scylladbmonitoring.init.func1.1()
    	github.com/scylladb/scylla-operator/test/e2e/set/scylladbmonitoring/scylladbmonitoring.go:258 +0x2184

https://prow.scylla-operator.scylladb.com/view/gs/scylla-operator-prow/logs/ci-scylla-operator-latest-e2e-gke/1825412597013286912#1:test-build-log.txt%3A2726

@scylla-operator-bot scylla-operator-bot
Copy link
Contributor

The Scylla Operator project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

  • After 30d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue as fresh with /remove-lifecycle rotten
  • Close this issue with /close
  • Offer to help out

/lifecycle rotten

@scylla-operator-bot scylla-operator-bot bot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 18, 2024
@scylla-operator-bot scylla-operator-bot bot added the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/flake Categorizes issue or PR as related to a flaky test. lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@zimnx @tnozicka @rzetelskik