Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement paged data serialization #132

Merged
merged 5 commits into from
Sep 26, 2020
Merged

Implement paged data serialization #132

merged 5 commits into from
Sep 26, 2020

Conversation

michaelwoerister
Copy link
Member

@michaelwoerister michaelwoerister commented Sep 18, 2020 •
edited by wesleywiser
Loading

This PR implements the new paging-based data format, roughly as described in #128. However, the format has been tweaked to allow variable sized pages.

This is a work in progress:

  • Make SerializationSink paging-based
  • Write unit tests for SerializationSink
  • Add documentation about the data format
  • Add more explanatory comments throughout the serialization module
  • Add a proper file header to the monolithic file in addition to the three stream headers
  • Run local perf.rlo tests with a rustc using this version of measureme.

Closes #128

@michaelwoerister
Turn serialization::BackingStorage into enum instead of a trait with …
ce63252 
…two impls.
@michaelwoerister
Implement paged serialization approach.
1825a06 
This allows to store all data in a single file instead of three separate ones.
@wesleywiser wesleywiser self-assigned this Sep 20, 2020
wesleywiser
wesleywiser reviewed Sep 24, 2020
View reviewed changes
Copy link
Member

@wesleywiser wesleywiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good to me!

measureme/src/serialization.rs Outdated Show resolved Hide resolved
measureme/src/serialization.rs Outdated Show resolved Hide resolved
analyzeme/src/profiling_data.rs Show resolved Hide resolved
@michaelwoerister michaelwoerister changed the title [wip] Implement paged data serialization Implement paged data serialization Sep 25, 2020
@michaelwoerister michaelwoerister marked this pull request as ready for review September 25, 2020 11:40
@wesleywiser
Copy link
Member

This looks great to me! Do you have any feeling for the performance for this approach? Is it on par with the previous implementation?

@michaelwoerister
Copy link
Member Author

Yeah, I expect performance to be the same as the proof of concept implementation.

@wesleywiser wesleywiser merged commit ad0b58d into rust-lang:master Sep 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wesleywiser wesleywiser wesleywiser approved these changes

Assignees

@wesleywiser wesleywiser

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Single file on disk format
2 participants
@michaelwoerister @wesleywiser