Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fair play with JSON #335

Merged
merged 10 commits into from
May 10, 2021
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,8 +200,16 @@ Testing base64 encoding/decoding of the large blob into the newly allocated buff

Testing parsing and simple calculating of values from a big JSON file.

NOTE: D implementations except Mir-based (like Ion and Asdf) have
[inaccurate number parsing](https://issues.dlang.org/show_bug.cgi?id=20967).
NOTE: DAW JSON Link, gason, 'fast', and D implementations except Mir-based (Ion and Asdf)
have inaccurate IEEE incompatible number parsing.
- [DAW JSON Link's number parsing code](https://github.com/beached/daw_json_link/blob/v2.9.2/include/daw/json/impl/daw_json_parse_real.h),
nuald marked this conversation as resolved.
Show resolved Hide resolved
- [gason's number parsing code](https://github.com/vivkin/gason/blob/v1.0.0/src/gason.cpp#L73)
- ['fast' number parsing code](https://github.com/mleise/fast/blob/v0.3.5/source/fast/parsing.d#L130)
- [D stdlib number parsing issue](https://issues.dlang.org/show_bug.cgi?id=20967)

NOTE: gason mutates input strings.

NOTE: simdjson and 'fast' (D) require input strings with batch of trailing zeros: a special zero padding for SIMD instructions.

[Json](json)

Expand All @@ -215,8 +223,8 @@ NOTE: D implementations except Mir-based (like Ion and Asdf) have
| C++/g++ (simdjson DOM) | 0.148<sub>±0.004</sub> | 109.88<sub>±00.04</sub> + 176.60<sub>±00.00</sub> | 3.69<sub>±00.23</sub> |
| C++/g++ (gason) | 0.159<sub>±0.001</sub> | 109.21<sub>±00.01</sub> + 97.17<sub>±00.03</sub> | 3.02<sub>±00.09</sub> |
| C++/g++ (RapidJSON) | 0.221<sub>±0.006</sub> | 109.24<sub>±00.01</sub> + 128.82<sub>±00.00</sub> | 4.56<sub>±00.48</sub> |
| D/ldc2 (Mir Amazon's Ion) | 0.231<sub>±0.003</sub> | 109.38<sub>±00.07</sub> + 16.11<sub>±00.01</sub> | 4.86<sub>±00.12</sub> |
| D/ldc2 (Mir Asdf) | 0.238<sub>±0.005</sub> | 109.45<sub>±00.04</sub> + 57.83<sub>±00.00</sub> | 4.92<sub>±00.27</sub> |
| D/ldc2 (Mir Amazon's Ion DOM) | 0.231<sub>±0.003</sub> | 109.38<sub>±00.07</sub> + 16.11<sub>±00.01</sub> | 4.86<sub>±00.12</sub> |
| D/ldc2 (Mir Asdf DOM) | 0.238<sub>±0.005</sub> | 109.45<sub>±00.04</sub> + 57.83<sub>±00.00</sub> | 4.92<sub>±00.27</sub> |
| C++/g++ (Boost.JSON) | 0.498<sub>±0.009</sub> | 109.81<sub>±00.03</sub> + 435.70<sub>±00.00</sub> | 10.00<sub>±00.67</sub> |
| Java | 0.508<sub>±0.007</sub> | 253.80<sub>±00.21</sub> + 70.03<sub>±01.09</sub> | 13.87<sub>±00.33</sub> |
| C++/g++ (RapidJSON SAX) | 0.530<sub>±0.008</sub> | 109.44<sub>±00.02</sub> + 0.00<sub>±00.00</sub> | 11.80<sub>±00.31</sub> |
Expand Down
4 changes: 2 additions & 2 deletions json/generate_json.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@

524_288.times do
h = {
'x' => rand,
'y' => rand,
'x' => rand * -10e-30,
'y' => rand * 10e+30,
'z' => rand,
'name' => "#{('a'..'z').to_a.sample(6).join} #{rand(10_000)}",
'opts' => { '1' => [1, true] }
Expand Down
2 changes: 1 addition & 1 deletion json/test_dawjsonlink.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ coordinate_t calc(const string& text) {
auto x = 0.0, y = 0.0, z = 0.0;
auto len = 0;

using range_t = daw::json::json_array_range<coordinate_t, daw::json::NoCommentSkippingPolicyUnchecked>;
nuald marked this conversation as resolved.
Show resolved Hide resolved
using range_t = daw::json::json_array_range<coordinate_t>;
auto rng = range_t(json_sv, "coordinates");

for (auto c : rng) {
Expand Down
2 changes: 1 addition & 1 deletion json/test_mir_asdf.d
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ void main()
{
validate!calc;
auto text = cast(string) "/tmp/1.json".read;
"D/ldc2 (Mir Asdf)".notifyStart;
"D/ldc2 (Mir Asdf DOM)".notifyStart;
auto coordinate = calc(text);
notifyStop;
coordinate.print;
Expand Down
2 changes: 1 addition & 1 deletion json/test_mir_ion.d
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ void main()
{
validate!calc;
auto text = cast(string) "/tmp/1.json".read;
"D/ldc2 (Mir Amazon's Ion)".notifyStart;
"D/ldc2 (Mir Amazon's Ion DOM)".notifyStart;
auto coordinate = calc(text);
notifyStop;
coordinate.print;
Expand Down
2 changes: 1 addition & 1 deletion json/test_mir_ion_file.d
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import mir.ion.deser.json : deserializeJsonFile;

void main() @safe @nogc
{
"D/ldc2 (Mir Amazon's Ion, file input)".notifyStart;
"D/ldc2 (Mir Amazon's Ion DOM, file input)".notifyStart;
auto coordinate = "/tmp/1.json".deserializeJsonFile!Avg.coordinates.avg;
notifyStop;
coordinate.print;
Expand Down
2 changes: 1 addition & 1 deletion json/test_rapid.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ string read_file(const string& filename) {

coordinate_t calc(const string& text) {
Document jobj;
jobj.Parse(text);
jobj.Parse<kParseFullPrecisionFlag>(text);
nuald marked this conversation as resolved.
Show resolved Hide resolved

const Value& coordinates = jobj["coordinates"];
auto len = coordinates.Size();
Expand Down
2 changes: 1 addition & 1 deletion json/test_rapid_sax.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ void calc(stringstream& ss, const TCallback& callback) {
IStreamWrapper isw(ss);
Reader reader;
CoordinateHandler handler(callback);
reader.Parse(isw, handler);
reader.Parse<kParseFullPrecisionFlag>(isw, handler);
nuald marked this conversation as resolved.
Show resolved Hide resolved
}

int main() {
Expand Down