Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add log rotation for Loki #525

Closed
mnaser opened this issue Aug 11, 2023 · 2 comments · Fixed by #1765
Closed

Add log rotation for Loki #525

mnaser opened this issue Aug 11, 2023 · 2 comments · Fixed by #1765
Assignees
@ricolin

Comments

@mnaser
Copy link
Member

mnaser commented Aug 11, 2023

We need to setup a log rotation in place to setup a tail like 365d logging or so.
@ricolin ricolin self-assigned this Aug 21, 2023
@ricolin
Copy link
Member

ricolin commented Aug 30, 2023 •
edited
Loading

One solution I found is to set kubelet configs containerLogMaxSize and containerLogMaxFiles which also recommanded in loki document https://grafana.com/docs/loki/latest/clients/promtail/logrotation/#kubernetes
But that's gonna limit rotation by size instead of duration.
Is that a acceptable approach here?

Also didn't find anything log rotate parameter or method in grafana-agent-operator, so I think this might be the only straightforward solution here.

@yaguangtang
Copy link
Contributor

I think we can enable loki compactor for indexing rotation, https://grafana.com/docs/loki/latest/operations/storage/retention/

yaguangtang added a commit that referenced this issue Aug 7, 2024
@yaguangtang
Support loki log rotation
aa45dc6 
Fixes #525

limits_config:
  retention_period: 10d # Keep 10 days
compactor:
  delete_request_cancel_period: 10m # don't wait 24h before processing the delete_request
  retention_enabled: true # actually do the delete
  retention_delete_delay: 2h # wait 2 hours before actually deleting stuff
yaguangtang added a commit that referenced this issue Aug 9, 2024
@yaguangtang
Support loki log rotation
e8d7063 
Fixes #525

limits_config:
  retention_period: 30d # Keep 30 days
compactor:
  delete_request_cancel_period: 10m # don't wait 24h before processing the delete_request
  retention_enabled: true # enable log retention
  retention_delete_delay: 2h # wait 2 hours before actually deleting stuff
@yaguangtang yaguangtang mentioned this issue Aug 9, 2024
Merged
yaguangtang added a commit that referenced this issue Aug 9, 2024
@yaguangtang
Support loki log rotation
250afc4 
Fixes #525

limits_config:
  retention_period: 30d # Keep 30 days
compactor:
  delete_request_cancel_period: 10m # don't wait 24h before processing the delete_request
  retention_enabled: true # enable log retention
  retention_delete_delay: 2h # wait 2 hours before actually deleting stuff
yaguangtang added a commit that referenced this issue Aug 29, 2024
@yaguangtang
Support loki log rotation
34fdfda 
Enable loki log rotation by default, the default retention period
is 30 days, but can be changed with an ansible variable
loki_log_retention_period.

Fixes #525
yaguangtang added a commit that referenced this issue Aug 29, 2024
@yaguangtang
Support loki log rotation
047c3ec 
Enable loki log rotation by default, the default retention period
is 30 days, but can be changed with an ansible variable
loki_log_retention_period.

Fixes #525
yaguangtang added a commit that referenced this issue Aug 30, 2024
@yaguangtang
Support loki log rotation
995592b 
Enable loki log rotation by default, the default retention period
is 30 days.

Fixes #525
yaguangtang added a commit that referenced this issue Sep 10, 2024
@yaguangtang
Support loki log rotation
702432d 
Enable loki log rotation by default, the default retention period
is 30 days.

Fixes #525
yaguangtang added a commit that referenced this issue Oct 2, 2024
@yaguangtang
Support loki log rotation
c3e20d3 
Enable loki log rotation by default, the default retention period
is 30 days.

Fixes #525
yaguangtang added a commit that referenced this issue Oct 2, 2024
@yaguangtang
Support loki log rotation
3019321 
Enable loki log rotation by default, the default retention period
is 30 days.

Fixes #525
yaguangtang added a commit that referenced this issue Oct 2, 2024
@yaguangtang
Support loki log rotation
c8e952e 
Enable loki log rotation by default, the default retention period
is 30 days.

Fixes #525
yaguangtang added a commit that referenced this issue Oct 2, 2024
@yaguangtang
Support loki log rotation
c7dce03 
Enable loki log rotation by default, the default retention period
is 30 days.

Fixes #525
This was referenced Oct 2, 2024
Merged
Merged
Merged
Merged
yaguangtang added a commit that referenced this issue Oct 4, 2024
@yaguangtang
Support loki log rotation
365a6cb 
Enable loki log rotation by default, the default retention period
is 30 days.

Fixes #525
@atmosphere-ci atmosphere-ci bot closed this as completed in 06b1ff5 Oct 4, 2024
yaguangtang added a commit that referenced this issue Oct 4, 2024
@yaguangtang
Support loki log rotation
1e6e324 
Enable loki log rotation by default, the default retention period
is 30 days.

Fixes #525
yaguangtang added a commit that referenced this issue Oct 4, 2024
@yaguangtang
Support loki log rotation
799bc95 
Enable loki log rotation by default, the default retention period
is 30 days.

Fixes #525
yaguangtang added a commit that referenced this issue Oct 4, 2024
@yaguangtang
Support loki log rotation
7ee6e15 
Enable loki log rotation by default, the default retention period
is 30 days.

Fixes #525
atmosphere-ci bot pushed a commit that referenced this issue Oct 4, 2024
@yaguangtang
[ATMOSPHERE-465] add log rotation for loki (#1944)
cbc982b 
Enable loki log rotation by default, the default retention period is 30 days.
Fixes #525
atmosphere-ci bot pushed a commit that referenced this issue Oct 4, 2024
@yaguangtang
[ATMOSPHERE-464] Support loki log rotation (#1945)
3534f5f 
Enable loki log rotation by default, the default retention period is 30 days.
Fixes #525
atmosphere-ci bot pushed a commit that referenced this issue Oct 4, 2024
@yaguangtang
[ATMOSPHERE-466] Support loki log rotation (#1943)
4dce9a9 
Enable loki log rotation by default, the default retention period is 30 days.
Fixes #525
atmosphere-ci bot pushed a commit that referenced this issue Oct 4, 2024
@yaguangtang
[ATMOSPHERE-467] Support loki log rotation (#1942)
5dd627a 
Enable loki log rotation by default, the default retention period is 30 days.
Fixes #525
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ricolin ricolin

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[ATMOSPHERE-116] Support loki log rotation vexxhost/atmosphere
3 participants
@mnaser @yaguangtang @ricolin