Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

journal: make current log file have a fixed name #7112

Merged
merged 2 commits into from
Aug 26, 2021

Conversation

lanzafame
Copy link
Contributor

Resolves #7050.

@lanzafame lanzafame requested a review from a team as a code owner August 18, 2021 06:00
Copy link
Member

@raulk raulk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this will lead to slightly different semantics on the naming of journal files. Right now we stamp journals with the timestamp at which they were opened. This will stamp them with the date at which they were closed/rotated.

I'd honestly prefer to have a date range in there, even if it results in an excessively long filename.

Also to consider is the behaviour on restart -- this will stamp the existing journal with the timestamp of the restart. Currently we start a new journal with the date of the restart, which I think it's more intuitive.

journal/fs.go Outdated
nfi, err := os.Create(filepath.Join(f.dir, fmt.Sprintf("lotus-journal-%s.ndjson", build.Clock.Now().Format(RFC3339nocolon))))
// check if journal file exists
if _, err := os.Stat(filepath.Join(f.dir, "lotus-journal.json")); err == nil {
err := os.Rename(filepath.Join(f.dir, "lotus-journal.json"), filepath.Join(f.dir, fmt.Sprintf("lotus-journal-%s.ndjson", build.Clock.Now().Format(RFC3339nocolon))))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: that's a long line, can we break it out in local vars?

journal/fs.go Outdated
@@ -109,9 +109,17 @@ func (f *fsJournal) rollJournalFile() error {
_ = f.fi.Close()
}

nfi, err := os.Create(filepath.Join(f.dir, fmt.Sprintf("lotus-journal-%s.ndjson", build.Clock.Now().Format(RFC3339nocolon))))
// check if journal file exists
if _, err := os.Stat(filepath.Join(f.dir, "lotus-journal.json")); err == nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically, this could be a directory, but even if it is we want to rename it, so it's fine that we don't use the stat output.

@lanzafame
Copy link
Contributor Author

@raulk All these comments should be resolved. I also realised that I was using .json instead of .ndjson for the 'current' log file.

@Stebalien Stebalien requested a review from raulk August 25, 2021 16:34
journal/fs.go Outdated
current := filepath.Join(f.dir, "lotus-journal.ndjson")
rolled := filepath.Join(f.dir, fmt.Sprintf(
"lotus-journal-%s-%s.ndjson",
f.start.Format(RFC3339nocolon), end.Format(RFC3339nocolon),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This breaks when we restart, because f.start will no longer have the original start timestamp, so the stamping would be inaccurate. We could stamp the file with the start timestamp, and then append the end timestamp on rotation, but that would make the current filename dynamic again. I'm OK just accepting the semantic change to stamp the rotated file with the end date.

journal/fs.go Outdated

nfi, err := os.Create(filepath.Join(f.dir, fmt.Sprintf("lotus-journal-%s.ndjson", build.Clock.Now().Format(RFC3339nocolon))))
nfi, err := os.Create(filepath.Join(f.dir, "lotus-journal.json"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
nfi, err := os.Create(filepath.Join(f.dir, "lotus-journal.json"))
nfi, err := os.Create(filepath.Join(f.dir, "lotus-journal.ndjson"))

@raulk
Copy link
Member

raulk commented Aug 25, 2021

@lanzafame Apologies for back and forth, but I can't imagine a simple way of remembering the original start date on restart. I'm happy to accept the original semantic change where the timestamp is now the end/rotation timestamp. This is also how I believe frameworks like log4j handle rotation.

Also, caught a small bug.

@raulk raulk changed the title journal: current log file does not contain dynamic date journal: make current log file have a fixed named Aug 26, 2021
@raulk raulk merged commit 15667db into filecoin-project:master Aug 26, 2021
@raulk
Copy link
Member

raulk commented Aug 26, 2021

@lanzafame Thank you!

@raulk raulk changed the title journal: make current log file have a fixed named journal: make current log file have a fixed name Aug 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make current journal log file distinguishable from rotated journal log files
2 participants