- Sponsor
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Don't permalloc the pkgimgs unnecessarily #48215
Conversation
I think we still hit this codepath with |
Yeah, annoyingly we don't have that information from there. Do you find it reasonable to check options there? |
@gbaraldi I pushed a fix. Can I ask you to write some tests? |
So I was thinking about the test. Do we spawn a process with pkgimgs=no and load some package? Maybe a stdlib? |
Co-authored-by: Valentin Churavy <v.churavy@gmail.com>
8eaa163
to
ac5aed8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bool/true/false is C++, this should use int/1/0
Error in testset precompile:
Error During Test at /cache/build/default-amdci4-3/julialang/julia-master/julia-bdc77d5c19/share/julia/test/precompile.jl:339
Test threw exception
Expression: begin
m = Base._require_from_serialized(Base.PkgId(Foo), cachefile, ocachefile)
#= /cache/build/default-amdci4-3/julialang/julia-master/julia-bdc77d5c19/share/julia/test/precompile.jl:341 =# @test m isa Module
end
Error reading package image file.
Doesn't |
I mentioned this privately to valentin. But the test failure we see there happens because we specifically load the same package image twice. Though since we are dlopening it, instead of mmaping a different copy, we just get the same thing, that means we have already performed relocations over it. So that test is doing something kinda illegal here. Though we might want to add a check that we've already loaded a sysimg before. @KristofferC Do we have a check to see if package was loaded beforehand? Or does it happen on a higher level than Lines 1763 to 1773 in 57101cf
|
It loads the same package image twice, which triggers a checksum failure.
The check is typically at Line 1569 in 87f8958
require directly then that is skipped and the existing module is overwritten.
|
@nanosoldier |
Your package evaluation job has completed - possible new issues were detected. |
@vchuravy I was talking with @vtjnash and maybe we want to move the data section back to the .ji files. Two reasons 1, we might want to load the same file twice in order to inspect it before relocations, which |
Sure that sounds reasonable. I won't have time to tackle it though. |
Doesn't make sense really for this to be on the milestone imo. |
It was on the milestone because it was initially a small fix, but on further discussion it's probably not so simple. So I think we can release 1.9 without it. If the fix isn't too difficult we can probably land it on a 1.9 patch |
From the HPC call: This currently breaks PkgInspector. So the idea is to add an opt-out env variable that users of PkgInspector can set to get the copy back. |
It seems we were permallocing the memory for the sysimg even though the OS already does that more efficiently.