-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
countlines slower on master than v0.4 #16822
Comments
I don't see a slowdown on my system, but I also don't have access to that data file, so it's hard to tell. Can you try this redefinition of function Base.countlines(io::IO, eol::Char='\n')
isascii(eol) || throw(ArgumentError("only ASCII line terminators are supported"))
aeol = UInt8(eol)
a = Array{UInt8}(8192)
nl = 0
while !eof(io)
nb = readbytes!(io, a)
@simd for i=1:nb
@inbounds nl += a[i] == aeol
end
end
nl
end
Base.countlines(f::AbstractString, eol::Char='\n') = open(io->countlines(io,eol), f)::Int |
Now the speed of It seems the slowdown on master before is caused by Thanks. @StefanKarpinski is this similar pattern can be optimized by the compiler since |
I had originally written the function that way but @nalimilan convinced me that I didn't need to manually do this transformation, but that seems to be inconsistent across different systems, which is concerning. Does anyone have any ideas as to why that optimization would kick in on some systems and not others? |
In some confiurations LLVM doesn't seem to do this optimization Fix #16822
Compare
countlines
on master and v0.4On master:
On Jula v0.4:
But the speed of
readall/readstring
seems close between master and v0.4.compare the speed of
readall/readstring
on master and v0.4version info
My Julia master is the latest now:
Julia v0.4 version is:
The text was updated successfully, but these errors were encountered: