-
Notifications
You must be signed in to change notification settings - Fork 368
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Resolve type instability issues in aggregation #2424
Comments
In the following functions, the first two allocate, but not the last one: function fill_row2!(row, col::AbstractVector,
i::Integer,
colnames::NTuple{N, Symbol}) where N
@inbounds for j in 1:length(colnames)
cn = colnames[j]
val = row[cn]
col[i] = val
end
return nothing
end
function fill_row2b!(row, col::AbstractVector,
i::Integer,
colnames::NTuple{N, Symbol}) where N
@inbounds for cn in colnames
val = row[cn]
col[i] = val
end
return nothing
end
function fill_row3!(row, col::AbstractVector,
i::Integer,
colname::Symbol) where N
val = row[colname]
col[i] = val
return nothing
end A possible workaround is to require column names to be always in the same order: then we could index using an integer, which I just checked doesn't allocate. |
This is a bug. We have to fix it and insist that column names always have the same order. I will fix it. Otherwise we have:
so it is hard to predict the column order (it will depend on the first row processed, which in threading context is problematic) |
Also this is what we require in other places:
|
See https://gist.github.com/nalimilan/b801d48ab2931b6a671f05081fd94ba3 and in particular
https://gist.github.com/nalimilan/b801d48ab2931b6a671f05081fd94ba3#file-splitapplycombine-jl-mem-L1402
and
https://gist.github.com/nalimilan/b801d48ab2931b6a671f05081fd94ba3#file-splitapplycombine-jl-mem-L1438
and
https://gist.github.com/nalimilan/b801d48ab2931b6a671f05081fd94ba3#file-splitapplycombine-jl-mem-L1439
The text was updated successfully, but these errors were encountered: