Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add csv logging from daemon #432

Merged
merged 7 commits into from
Oct 27, 2022
Merged

Add csv logging from daemon #432

merged 7 commits into from
Oct 27, 2022

Conversation

JakeRoggenbuck
Copy link
Owner

fixes #359

@JakeRoggenbuck JakeRoggenbuck changed the title Add all new log for csv things Add csv logging from daemon Oct 26, 2022
@JakeRoggenbuck
Copy link
Owner Author

image

@JakeRoggenbuck JakeRoggenbuck requested review from Camerooooon and Shuzhengz and removed request for Camerooooon October 26, 2022 21:52
Copy link
Collaborator

@Shuzhengz Shuzhengz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should have a time / size limit on the csv to not murder the ram (also FAT16B has a 2GiB max file size limit, and FAT32 is 4GiB)

Edit: also if anyone is still using fat12 for some reason the max size would be 32mb

src/daemon.rs Outdated
Comment on lines 216 to 221

if let Some(name) = &self.settings.csv_file {
if !Path::new(name).exists() {
File::create(name).expect("CSV file could not be created.");
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should have an option to just override it on startup so u dont have to go in and delete the previous one every time

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New folders for each day? We could create a new CSV file hourly. We don't want massive individual files.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep sounds good, we can also just use the timestamp as the file name

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point about the size limit. It just writes out the current cpu objects with the to_csv to format them as strings then write it to a file. It does not store the whole thing in memory. I think a size limit would be very good so it does not break a hard drive.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file was getting multiple kilobytes in a few seconds

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea, also the size limits are acutally all in mebibytes instead of megabites

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea

src/daemon.rs Outdated Show resolved Hide resolved
src/cpu.rs Outdated Show resolved Hide resolved
@JakeRoggenbuck
Copy link
Owner Author

The only thing I have not written yet is the "die after 5mb" for log file size

@JakeRoggenbuck
Copy link
Owner Author

@Shuzhengz Should I create the file size check in this pr or not?

@Shuzhengz
Copy link
Collaborator

yeah i think it should be included in this pr, but we can also just make a new issue and deal with it later
the main concern is causing system to crash when exceed max file size, but that's pretty unlikely in modern systems

@JakeRoggenbuck
Copy link
Owner Author

Yea, it's a very good thing to consider. I'll also check the permissions needed to create the csv file. I'll probably do it in this PR.

Copy link
Collaborator

@Shuzhengz Shuzhengz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@JakeRoggenbuck
Copy link
Owner Author

Looks like i need all change requesters to review. Probably a good thing

@JakeRoggenbuck
Copy link
Owner Author

Cool, it has one review and I fixed all the requested changes, so i'm gonna merge it.

@JakeRoggenbuck JakeRoggenbuck merged commit 43f6ecd into main Oct 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CSV Logging
3 participants