[DNM] cramjam support

[crusaderky]

Blocked by

• cramjam 2.7.0 final release
• A/B tests

[crusaderky]

DNM

[github-actions]

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

       25 files   -        1         25 suites   - 1   14h 42m 55s ⏱️ - 1h 29m 52s
  3 608 tests +     27    3 499 ✔️ +     29     105 💤 ±0  4 ❌  - 2 
42 399 runs   - 2 941  40 223 ✔️  - 2 946  2 172 💤 +9  4 ❌  - 4 

For more details on these failures, see this check.

Results for commit 9b488a7. ± Comparison against base commit 09ea0f5.

♻️ This comment has been updated with latest results.

[crusaderky]

The whole decompress_into hack would become redundant if we could get rid of sharding.
XREFS:

• Deserialization of compressed data is sluggish and causes memory flares #7433
• Can we get rid of sharding? #7824

[crusaderky]

The A/B tests show a >2x slowdown of cramjam.lz4 vs. plain lz4. Either that, or one of the test modules hangs forever. Did not investigate yet.
https://github.com/coiled/benchmarks/actions/runs/4938359937/jobs/8828024130

[milesgranger]

The A/B tests show a >2x slowdown of cramjam.lz4 vs. plain lz4.

oi, got me nervous. Ran benchmarks again on cramjam's end and all seems normal, as well as your example comparison and got:

=== lz4 ===
0.5676321685314178
75.5 ms ± 574 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
44.8 ms ± 509 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
=== cramjam.lz4 ===
0.5676321685314178
64.3 ms ± 156 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
24.8 ms ± 309 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
9.74 ms ± 22.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Nothing immediately stands out to me here why it'd be slower though, but weird edge cases happen. 🤔

[crusaderky]

It's more likely that one test hanged

…

On Thu, 11 May 2023, 06:16 Miles, ***@***.***> wrote: The A/B tests show a >2x slowdown of cramjam.lz4 vs. plain lz4. oi, got me nervous. Ran benchmarks again on cramjam's end and all seems normal, as well as your example comparison and got: === lz4 ===0.567632168531417875.5 ms ± 574 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)44.8 ms ± 509 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)=== cramjam.lz4 ===0.567632168531417864.3 ms ± 156 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)24.8 ms ± 309 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)9.74 ms ± 22.1 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) Nothing immediately stands out to me here why it'd be slower though, but weird edge cases happen. 🤔 — Reply to this email directly, view it on GitHub <#7823 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABPM4MGR6SQT5BVEOPMN3VDXFRR2XANCNFSM6AAAAAAX4PYTNU> . You are receiving this because you were assigned.Message ID: ***@***.***>