Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow resource limits overrides #726

Closed
mncf9 opened this issue Jan 8, 2024 · 0 comments · Fixed by #789
Closed

Allow resource limits overrides #726

mncf9 opened this issue Jan 8, 2024 · 0 comments · Fixed by #789
Assignees

Comments

@mncf9
Copy link

mncf9 commented Jan 8, 2024

Currently, Tempo Stack controls how resources are distributed amongst the Tempo deployments. This might lead to the resource wasting. While some of the deployments have overcommitted limits, the other struggle to get more. Just as in my case:
image

You can clearly see that the Ingester does not need such high resources reserved while the Compactor practically gets OOM Killed all the time. Of course I have scaled up to a bigger instance to keep compactor alive, but then I have tons of resources being wasted.

How about Tempo Stack allowed overriding components limits? E.g like that:

apiVersion: tempo.grafana.com/v1alpha1
kind: TempoStack
metadata:
  name: stack
  namespace: tempons
spec:
  resources:
    total:
      limits:
        memory: 24Gi
        cpu: 1000m
  template:
    compactor:
      resources:
        limits:
          cpu: 500m
          memory: 16Gi
    queryFrontend:
      resources:
        limits:
          cpu: 100m
          memory: 256Mi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants