avoid intermediate array for lz4/zlib

[aminnj]

When decompressing, we do something like

@view(uncomp_data[fufilled+1:fufilled+uncompbytes]) = <output of decompression algo>

for LZ4, this is actually making a temporary array out_buffer which is then copied into uncomp_data.

function lz4_decompress(
        input::Union{Vector{UInt8},Base.CodeUnits{UInt8}},
        expected_size::Integer=length(input) * 2
    )
    out_buffer = Vector{UInt8}(undef, expected_size)
    out_size = LZ4_decompress_safe(pointer(input), pointer(out_buffer), length(input), expected_size)
    resize!(out_buffer, out_size)
end

By feeding LZ4_decompress_safe a pointer to uncomp_data, we can go faster! 🚀
And the same exercise was repeated for ZLIB to eliminate the intermediate storage. I copied almost exactly what ROOT has.

LZ4

before

julia> @btime sum(tf.nMuon)
  600.881 ms (2650 allocations: 870.01 MiB)
0x0000000008e67ad8

after

julia> @btime sum(tf.nMuon)
  472.534 ms (2339 allocations: 580.06 MiB)
0x0000000008e67ad8

ZLIB

before

julia> @btime sum(tf.nMuon)
  1.326 s (15170 allocations: 848.43 MiB)
0x0000000008e67ad8


julia> @time sum.(tf.Muon_pt)
  8.802347 seconds (22.05 k allocations: 4.399 GiB, 6.27% gc time)

after

julia> @btime sum(tf.nMuon)
  1.107 s (12162 allocations: 544.53 MiB)
0x0000000008e67ad8

julia> @time sum.(tf.Muon_pt)
  7.474018 seconds (18.92 k allocations: 3.216 GiB, 5.11% gc time)

......so ~15% speedups for zlib and more for lz4.

[codecov]

Codecov Report

Merging #128 (6d800ed) into master (c3cf8fb) will decrease coverage by 0.28%.
The diff coverage is 84.37%.

@@            Coverage Diff             @@
##           master     #128      +/-   ##
==========================================
- Coverage   92.45%   92.17%   -0.29%     
==========================================
  Files          11       11              
  Lines        1392     1418      +26     
==========================================
+ Hits         1287     1307      +20     
- Misses        105      111       +6     

Impacted Files	Coverage Δ
src/types.jl	88.34% <84.37%> (-3.86%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c3cf8fb...6d800ed. Read the comment docs.

[aminnj]

For future reference, this is how ROOT does zlib for example, but the others are in there as well:

https://github.com/root-project/root/blob/87a998d48803bc207288d90038e60ff148827664/core/zip/src/RZip.cxx#L392

[Moelf]

do we want to wrap this whole if-else logic into a function? since technically there's another place we use this:
https://github.com/tamasgal/UnROOT.jl/blob/35d00838067d68e74ede4ebddfb298c4eac4f813/src/streamers.jl#L85-L87

[aminnj]

in principle, yes, we can consolidate them, though I don't know if it's worth it because the streamers are only unpacked once and they're small. Maybe once we're happy with the inplace versions (and perhaps also have done if for the remaining two algos as well)

[aminnj]

And for reference, the updated zlib decompression function works with zlib-ng still. Eg

cp ~/julia-1.7.0-beta4/lib/julia/libz.so.1.2.11 bck.so # backup
# compile zlib-ng
cp ~/zlib-ng/libz.so.1.2.11.zlib-ng ~/julia-1.7.0-beta4/lib/julia/libz.so.1.2.11

and gives another 10% improvement in decompression speed itself. So when that gets incorporated, we get another speedboost.