Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Promtail-friendly one line logs #37

Merged
merged 6 commits into from
Jan 7, 2020
Merged

Conversation

raneq
Copy link
Collaborator

@raneq raneq commented Dec 27, 2019

Print log entries as single lines, also in error case. This implies wrapping the execution of commands in case they fail, and printing their output flattened (\n escaped) in that case.

3 examples:

Fail Output Example #1

2019-12-23 15:13:01.982148376+00:00 | INFO | Create temporal directory
2019-12-23 15:13:01.985607707+00:00 | INFO | Postgres DB dump
2019-12-23 15:13:02.278978954+00:00 | INFO | Compress assets data
2019-12-23 15:13:02.289908150+00:00 | INFO | Restic backup
2019-12-23 15:13:03.107826457+00:00 | ERROR | Fatal: unable to open repo at b2:some-bucket:/: b2.NewClient: b2_authorize_account: 401:

Fail Output Example #2

2019-12-23 15:43:01.162415961+00:00 | INFO | Create temporal directory
2019-12-23 15:43:01.165357643+00:00 | INFO | Postgres DB dump
2019-12-23 15:43:01.471210237+00:00 | INFO | Compress assets data
2019-12-23 15:43:01.481858818+00:00 | INFO | Restic backup
2019-12-23 15:43:03.052475062+00:00 | ERROR | Fatal: unable to open repo at b2:non-existing-bucket:/: Bucket: non-existing-bucket: bucket not found

Success Output

2019-12-27 09:27:01.222474096+00:00 | INFO | Create temporal directory
2019-12-27 09:27:01.229189669+00:00 | INFO | Postgres DB dump
2019-12-27 09:27:02.238105299+00:00 | INFO | Compress assets data
2019-12-27 09:27:02.252841960+00:00 | INFO | Restic backup
2019-12-27 09:27:18.586371329+00:00 | INFO | Restic forget
2019-12-27 09:27:24.489343955+00:00 | INFO | Restic check
2019-12-27 09:27:34.028960454+00:00 | INFO | Backup complete

I also create a template config file for promtail that I have checked it works. However, it's not backups-role purpose to install and configure promtail, so we don't do anything with that config file. I leave it there more as reference. We could move it to a wiki page or /docs dir.

log prefixes datetime in standard format and loglevel
run evals a command, saves its stderr, and if the return code means error, prints that stderr through log, to the script stdout
@raneq raneq changed the title Promtail one line logs WIP Promtail one line logs Dec 27, 2019
It works, but we don't use it still from tasks. We have to decide how to combine many promtail jobs that we could need from a webapp, e.g.: backups + app logs + system logs.

We leave this config here for the record.
@raneq raneq changed the title WIP Promtail one line logs Promtail one line logs Dec 27, 2019
@raneq raneq changed the title Promtail one line logs Promtail-friendly one line logs Dec 27, 2019
@raneq raneq requested a review from danypr92 December 27, 2019 12:12
CHANGELOG.md Outdated
@@ -6,6 +6,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

- Print promtail-friendly logs. See #37
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to have the link to the PR as well. In the vast majority of cases we'll be reading the Changelog from our computers and having the link will come in handy.

@raneq raneq requested a review from sauloperez January 7, 2020 12:33
@sauloperez sauloperez merged commit e6340a8 into master Jan 7, 2020
@sauloperez sauloperez deleted the promtail-one-line-logs branch January 7, 2020 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants