Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a 'sink' parameter to the LazyTree() functions to support other table/dataframe types #282

Merged
merged 8 commits into from
Oct 12, 2023

Conversation

grasph
Copy link
Member

@grasph grasph commented Oct 12, 2023

closes #280

@codecov
Copy link

codecov bot commented Oct 12, 2023

Codecov Report

All modified lines are covered by tests ✅

Files Coverage Δ
src/iteration.jl 85.54% <100.00%> (+0.52%) ⬆️

📢 Thoughts on this report? Let us know!.

@Moelf
Copy link
Member

Moelf commented Oct 12, 2023

according to documentation

which wraps any Tables.AbstractColumns-compatible object that has already had copies of its columns made, and are thus safe for the columns-consumer to assume ownership of (this is because DataFrames.jl, by default makes copies of all columns upon construction). In both cases, individual columns are collected in Vector{AbstractVector}s by calling Tables.getcolumn(x, nm) for each column name.

are you sure our getcolumn doesn't allocated?

@Moelf
Copy link
Member

Moelf commented Oct 12, 2023

Looks good to me

this PR

julia> @time LazyTree("./test/samples/NanoAODv5_sample.root", "Events", r"Muon.*");
  0.041865 seconds (394.89 k allocations: 39.741 MiB)

julia> @time LazyTree("./test/samples/NanoAODv5_sample.root", "Events", r"Muon.*"; sink=DataFrame);
  0.041879 seconds (394.97 k allocations: 39.748 MiB)

main

julia> @time LazyTree("./test/samples/NanoAODv5_sample.root", "Events", r"Muon.*");
  0.045695 seconds (394.95 k allocations: 39.757 MiB)

julia> @time DataFrame(LazyTree("./test/samples/NanoAODv5_sample.root", "Events", r"Muon.*"); copycols=false);
  0.045971 seconds (395.03 k allocations: 39.766 MiB)

julia> @time DataFrame(LazyTree("./test/samples/NanoAODv5_sample.root", "Events", r"Muon.*"); copycols=true);
  0.066453 seconds (398.93 k allocations: 43.543 MiB)

@Moelf Moelf requested a review from tamasgal October 12, 2023 21:35
@Moelf
Copy link
Member

Moelf commented Oct 12, 2023

@grasph I hope you're okay with the tests and cosmetics changes I made, I think we can safely merge this and make a patch release.

Minor version Y (x.Y.z | x > 0) MUST be incremented if new, backwards compatible functionality is introduced to the public API.

we are pre 1.0 (x == 0) so I say we don't bump the 10 in v0.10.z

@Moelf Moelf merged commit 2583c34 into JuliaHEP:main Oct 12, 2023
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

nanoAOD_ttbar latency
2 participants