Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: bson/json comparison benchmarks #2799

Merged
merged 3 commits into from
Apr 11, 2024
Merged

chore: bson/json comparison benchmarks #2799

merged 3 commits into from
Apr 11, 2024

Conversation

tychoish
Copy link
Contributor

I modified the #2974 benchmarks to include the same test but with BSON
encoding.

BSON is faster than JSON in most/many csaes, with much more stable
perofrmance patterns, though tthe larger globs-cases are curious, and
may be a sign that the test hits some larger limitation of the
hardware.

This could be interesting fodder--with tests run other than on my
laptop--for a blog post or similar about enconding impacts on
performance.

---------------------------------------------------------------------------------------------------------- benchmark: 24 tests -----------------------------------------------------------------------------------------------------------
Name (time in ms)                                      Min                     Max                    Mean                 StdDev                  Median                    IQR            Outliers     OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_bson_function[c16.r256]                      510.4455 (1.0)          709.4622 (1.0)          567.7784 (1.0)          80.4747 (1.0)          535.9142 (1.0)          62.7712 (1.0)           1;1  1.7613 (1.0)           5           1
test_json_function[c16.r256-read_json]          3,159.1614 (6.19)       6,474.4995 (9.13)       3,848.5652 (6.78)      1,468.4337 (18.25)      3,191.3783 (5.96)        896.9773 (14.29)         1;1  0.2598 (0.15)          5           1
test_json_function[c16.r256-read_ndjson]        3,239.7357 (6.35)       8,110.2017 (11.43)      5,439.2703 (9.58)      2,472.8914 (30.73)      4,463.3695 (8.33)      4,800.2773 (76.47)         2;0  0.1838 (0.10)          5           1

test_bson_function[c16.r512]                      579.2455 (1.13)       1,522.4248 (2.15)         859.2683 (1.51)        410.8683 (5.11)         601.0137 (1.12)        540.5734 (8.61)          1;0  1.1638 (0.66)          5           1
test_json_function[c16.r512-read_json]         12,946.2755 (25.36)     28,673.7928 (40.42)     16,488.0264 (29.04)     6,827.0157 (84.83)     13,616.5120 (25.41)     4,673.7290 (74.46)         1;1  0.0607 (0.03)          5           1
test_json_function[c16.r512-read_ndjson]       12,733.5076 (24.95)     24,246.5132 (34.18)     17,969.9877 (31.65)     4,966.2403 (61.71)     17,908.8948 (33.42)     8,817.5478 (140.47)        2;0  0.0556 (0.03)          5           1

test_bson_function[c32.r256]                    1,921.5829 (3.76)       2,442.6062 (3.44)       2,073.9255 (3.65)        216.3419 (2.69)       1,980.5387 (3.70)        245.4824 (3.91)          1;0  0.4822 (0.27)          5           1
test_json_function[c32.r256-read_json]         13,797.7979 (27.03)     16,052.3017 (22.63)     14,552.5215 (25.63)       917.1318 (11.40)     14,309.2267 (26.70)     1,200.4637 (19.12)         1;0  0.0687 (0.04)          5           1
test_json_function[c32.r256-read_ndjson]       12,411.8242 (24.32)     32,970.0620 (46.47)     21,950.0160 (38.66)     8,832.0592 (109.75)    18,016.6833 (33.62)    14,692.9570 (234.07)        2;0  0.0456 (0.03)          5           1

test_bson_function[c32.r512]                    2,181.5750 (4.27)       7,000.0344 (9.87)       4,531.6169 (7.98)      2,262.6077 (28.12)      4,306.0612 (8.03)      4,351.9662 (69.33)         2;0  0.2207 (0.13)          5           1
test_json_function[c32.r512-read_json]         53,522.8613 (104.86)    90,312.0904 (127.30)    62,884.7446 (110.76)   15,555.4531 (193.30)    57,240.7821 (106.81)   13,818.5718 (220.14)        1;1  0.0159 (0.01)          5           1
test_json_function[c32.r512-read_ndjson]       53,220.0279 (104.26)    61,396.9645 (86.54)     59,007.9780 (103.93)    3,323.5944 (41.30)     60,523.2753 (112.93)    3,054.5214 (48.66)         1;0  0.0169 (0.01)          5           1

test_bson_function[glob.n16]                    1,206.8886 (2.36)       5,805.5513 (8.18)       2,138.7193 (3.77)      2,049.8427 (25.47)      1,225.5290 (2.29)      1,156.5005 (18.42)         1;1  0.4676 (0.27)          5           1
test_json_function[glob.n16-read_json]         16,888.6769 (33.09)     25,854.9869 (36.44)     20,025.6336 (35.27)     3,483.2713 (43.28)     19,284.4107 (35.98)     3,835.3617 (61.10)         1;0  0.0499 (0.03)          5           1
test_json_function[glob.n16-read_ndjson]       14,361.7829 (28.14)     27,842.9034 (39.25)     18,084.3947 (31.85)     5,727.1944 (71.17)     14,969.5425 (27.93)     6,397.9288 (101.92)        1;0  0.0553 (0.03)          5           1

test_bson_function[glob.n64]                    4,511.0605 (8.84)       5,792.7155 (8.16)       5,011.1584 (8.83)        472.5563 (5.87)       4,892.1433 (9.13)        395.0379 (6.29)          2;1  0.1996 (0.11)          5           1
test_json_function[glob.n64-read_json]         34,398.9564 (67.39)     48,915.9772 (68.95)     41,650.4574 (73.36)     6,002.0668 (74.58)     40,901.4151 (76.32)    10,158.6490 (161.84)        2;0  0.0240 (0.01)          5           1
test_json_function[glob.n64-read_ndjson]       13,666.7455 (26.77)     46,893.1358 (66.10)     21,892.1260 (38.56)    14,293.3494 (177.61)    14,201.4302 (26.50)    13,511.0962 (215.24)        1;0  0.0457 (0.03)          5           1

test_bson_function[glob.n256]                  17,942.7219 (35.15)     21,417.1342 (30.19)     19,036.9317 (33.53)     1,440.5947 (17.90)     18,418.8785 (34.37)     1,836.9645 (29.26)         1;0  0.0525 (0.03)          5           1
test_json_function[glob.n256-read_json]       110,671.9929 (216.81)   125,233.4375 (176.52)   118,422.8545 (208.57)    5,688.4685 (70.69)    120,446.3032 (224.75)    8,176.9406 (130.27)        2;0  0.0084 (0.00)          5           1
test_json_function[glob.n256-read_ndjson]      14,976.3305 (29.34)     26,482.2308 (37.33)     19,886.8290 (35.03)     4,688.5536 (58.26)     21,014.2603 (39.21)     6,996.9872 (111.47)        2;0  0.0503 (0.03)          5           1

test_bson_function[glob.n512]                  35,058.7739 (68.68)     37,264.6268 (52.53)     35,736.1031 (62.94)       937.9202 (11.65)     35,214.9876 (65.71)     1,220.2517 (19.44)         1;0  0.0280 (0.02)          5           1
test_json_function[glob.n512-read_json]       187,815.8671 (367.95)   239,793.4055 (337.99)   213,641.4891 (376.28)   24,750.8322 (307.56)   215,358.7825 (401.85)   48,084.3203 (766.02)        2;0  0.0047 (0.00)          5           1
test_json_function[glob.n512-read_ndjson]      16,084.7495 (31.51)     44,574.1158 (62.83)     28,615.9287 (50.40)    10,704.3141 (133.01)    28,637.9175 (53.44)    13,887.8985 (221.25)        2;0  0.0349 (0.02)          5           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

@tychoish tychoish requested review from talagluck and scsmithr March 20, 2024 11:24
@tychoish tychoish merged commit 3ac9482 into main Apr 11, 2024
26 checks passed
@tychoish tychoish deleted the tycho/bson-comparison branch April 11, 2024 22:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants