Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JsonBufferBuilder. #51

Merged
merged 4 commits into from
Aug 29, 2024
Merged

Conversation

davidmorgan
Copy link
Contributor

New alternative implementation for JsonBuffer, still WIP, will send an email about status.

@davidmorgan davidmorgan mentioned this pull request Aug 26, 2024
pkgs/dart_model/lib/src/json_buffer/closed_map.dart Outdated Show resolved Hide resolved
pkgs/dart_model/lib/src/json_buffer/closed_map.dart Outdated Show resolved Hide resolved
pkgs/dart_model/lib/src/json_buffer/closed_map.dart Outdated Show resolved Hide resolved
pkgs/dart_model/lib/src/json_buffer/closed_map.dart Outdated Show resolved Hide resolved
pkgs/dart_model/lib/src/json_buffer/closed_map.dart Outdated Show resolved Hide resolved
pkgs/dart_model/lib/src/json_buffer/typed_map.dart Outdated Show resolved Hide resolved
pkgs/dart_model/lib/src/json_buffer/typed_map.dart Outdated Show resolved Hide resolved
@@ -0,0 +1,353 @@
// Copyright (c) 2024, the Dart project authors. Please see the AUTHORS file
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran out of steam a bit by the time I got to this one admittedly, but I can take a closer look later 🤣

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a happy coincidence as I need to take another pass at this one :) will do so next, so please expect another commit with additions/tweaks here after these comments.

@jakemac53
Copy link
Contributor

Definitely liking the general direction here - lots of areas to squeeze out more performance later on too if desired.

I think my main concern overall is we are doing so much work lazily we are hiding a lot of costs and moving them to places that aren't necessarily expected, in the name of "free" deserialization. We should probably be thinking about benchmarks that actually use these deserialized objects in "reasonable" ways, including reading the same fields multiple times, etc.

Copy link
Contributor Author

@davidmorgan davidmorgan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks :)

All replied, and pushed an additional commit instead of breaking everything this time.

You commented that there aren't enough comments on TypedMap and I agree, I didn't get around to taking another look at TypedMap either, I will do that next :)

Feel free to wait for next commit before taking another look, or to look at the comments+updates I'm sending now, as you like.

pkgs/dart_model/lib/src/json_buffer/closed_map.dart Outdated Show resolved Hide resolved
pkgs/dart_model/lib/src/json_buffer/closed_map.dart Outdated Show resolved Hide resolved
pkgs/dart_model/lib/src/json_buffer/closed_map.dart Outdated Show resolved Hide resolved
pkgs/dart_model/lib/src/json_buffer/closed_map.dart Outdated Show resolved Hide resolved
pkgs/dart_model/lib/src/json_buffer/entry_map.dart Outdated Show resolved Hide resolved
pkgs/dart_model/lib/src/json_buffer/typed_map.dart Outdated Show resolved Hide resolved
pkgs/dart_model/lib/src/json_buffer/typed_map.dart Outdated Show resolved Hide resolved
pkgs/dart_model/lib/src/json_buffer/typed_map.dart Outdated Show resolved Hide resolved
/// Set to an instance of [Explanations] to turn on logging, reset to null to
/// turn off logging. If set, [JsonBufferBuilder#toString] will print
/// additional information for each byte.
Explanations? explanations;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could do it with an environment variable so it is const - and then if tests fail you could just set the environment variable to get better output (and maybe we would suggest doing this)?

I don't mind a global if it isn't mutable :)

pkgs/dart_model/lib/src/json_buffer/type.dart Outdated Show resolved Hide resolved
@davidmorgan davidmorgan marked this pull request as ready for review August 28, 2024 10:25
Made any classes and methods that can be private, private.
Copy link
Contributor Author

@davidmorgan davidmorgan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Addressed comments, and finished cleaning up TypedMap and adding comments.

Elsewhere I tidied up a bit, mostly improving comments, making things private.

PTAL, I think this is now about ready :)

/// Set to an instance of [Explanations] to turn on logging, reset to null to
/// turn off logging. If set, [JsonBufferBuilder#toString] will print
/// additional information for each byte.
Explanations? explanations;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Environment const is a good idea, thanks, done; moved it to a member of JsonBufferBuilder and made everything private.

I couldn't get the bool to work right with default "dart test", only "dart test -c source", so I put that in the instructions.

///
/// Then, [JsonBufferBuilder#toString] prints additional information for
/// each byte.
final _Explanations? _explanations =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be static yeah?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Class local is a bit neater for the tests, and it doesn't seem to make a difference in the benchmarks, i.e. performance seems to be the same.

/// Then, [JsonBufferBuilder#toString] prints additional information for
/// each byte.
final _Explanations? _explanations =
const bool.fromEnvironment('debug_json_buffer') ? _Explanations() : null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was originally thinking system environment variables not -D ones, but I see now that the former can't be const.

We could file an issue on package:test to support these, fwiw. Not sure we would actually implement it, there could be some pushback, but it might be worth thinking about.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, filed dart-lang/test#2272

@davidmorgan davidmorgan merged commit 66dc6a5 into dart-lang:main Aug 29, 2024
43 checks passed
@davidmorgan davidmorgan deleted the json-buffer-builder branch August 29, 2024 08:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants