-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Threading failure with @batch
#133
Comments
julia> @time @batch for evt in t ; _ = evt.Muon_pt; end
0.501378 seconds (149.80 k allocations: 3.466 GiB, 11.44% gc time, 12.94% compilation time)
julia> @time @threads for evt in t ; _ = evt.Muon_pt; end
0.413486 seconds (24.45 k allocations: 3.385 GiB, 18.05% gc time, 4.27% compilation time)
julia> @time @batch for (i,evt) in enumerate(t) ; _ = evt.Muon_pt ; end
2.157369 seconds (61.70 M allocations: 8.052 GiB, 56.97% gc time, 4.21% compilation time)
julia> @time @threads for (i,evt) in enumerate(t) ; _ = evt.Muon_pt ; end
2.276005 seconds (61.57 M allocations: 7.970 GiB, 60.74% gc time, 1.21% compilation time) Whether we use And in the future it would be nice if we could get |
Actually, this is an easy fix. -Base.getindex(e::Iterators.Enumerate{LazyTree{T}}, row::Int) where T = (row, first(iterate(e.itr, row)))
+Base.getindex(e::Iterators.Enumerate{LazyTree{T}}, row::Int) where T = (row, LazyEvent(innertable(e.itr), row)) and then... julia> @time @threads for evt in t ; _ = evt.Muon_pt ; end
0.420194 seconds (24.55 k allocations: 3.460 GiB, 16.40% gc time, 0.00% compilation time)
julia> @time @threads for (i,evt) in enumerate(t) ; _ = evt.Muon_pt ; end
0.401146 seconds (27.84 k allocations: 3.460 GiB, 16.16% gc time, 4.41% compilation time) and then we drop polyester. |
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
(nondeterministic) MWE:
but note that it only fails 30% of the time :( with something like this:
@Moelf suggests it's the customization in
UnROOT/src/polyester.jl
. Commenting that out seems to eliminate these failures and@batch
still works because of the recent PR that madeLazyTree <: AbstractVector{LazyEvent{T}}
.So, since
@threads
/@batch
have pretty much the same overhead when considering our problem/batch size [1], if we can do all of the following,then we can drop all the polyester stuff.
[1]
Double muon file H->ZZ->4mu benchmark with
polyester.jl
commented outThe text was updated successfully, but these errors were encountered: