Don't permalloc the pkgimgs unnecessarily

[gbaraldi]

It seems we were permallocing the memory for the sysimg even though the OS already does that more efficiently.

[vchuravy]

I think we still hit this codepath with --pkgimages=no where we still would need to do that?

[gbaraldi]

Yeah, annoyingly we don't have that information from there. Do you find it reasonable to check options there?

[vchuravy]

@gbaraldi I pushed a fix. Can I ask you to write some tests?

[gbaraldi]

So I was thinking about the test. Do we spawn a process with pkgimgs=no and load some package? Maybe a stdlib?

[vtjnash]

bool/true/false is C++, this should use int/1/0

Error in testset precompile:
Error During Test at /cache/build/default-amdci4-3/julialang/julia-master/julia-bdc77d5c19/share/julia/test/precompile.jl:339
  Test threw exception
  Expression: begin
    m = Base._require_from_serialized(Base.PkgId(Foo), cachefile, ocachefile)
    #= /cache/build/default-amdci4-3/julialang/julia-master/julia-bdc77d5c19/share/julia/test/precompile.jl:341 =# @test m isa Module
end
  Error reading package image file.

[gbaraldi]

Doesn't stdbool.h add bool/true/false?

[gbaraldi]

I mentioned this privately to valentin. But the test failure we see there happens because we specifically load the same package image twice. Though since we are dlopening it, instead of mmaping a different copy, we just get the same thing, that means we have already performed relocations over it. So that test is doing something kinda illegal here. Though we might want to add a check that we've already loaded a sysimg before.

@KristofferC Do we have a check to see if package was loaded beforehand? Or does it happen on a higher level than

julia/base/loading.jl

Lines 1763 to 1773 in 57101cf

	function _require_from_serialized(uuidkey::PkgId, path::String, ocachepath::Union{String, Nothing})
	@lock require_lock begin
	set_pkgorigin_version_path(uuidkey, nothing)
	newm = _tryrequire_from_serialized(uuidkey, path, ocachepath)
	newm isa Module || throw(newm)
	insert_extension_triggers(uuidkey)
	# After successfully loading, notify downstream consumers
	run_package_callbacks(uuidkey)
	return newm
	end
	end

[KristofferC]

Do we have a check to see if package was loaded beforehand?

The check is typically at

julia/base/loading.jl

Line 1569 in 87f8958

if !root_module_exists(uuidkey)

but if you call require directly then that is skipped and the existing module is overwritten.

[vchuravy]

@nanosoldier runtests()

[nanosoldier]

Your package evaluation job has completed - possible new issues were detected.
A full report can be found here.

[gbaraldi]

@vchuravy I was talking with @vtjnash and maybe we want to move the data section back to the .ji files. Two reasons 1, we might want to load the same file twice in order to inspect it before relocations, which dlopen doesn't allow. The second thing is that controlling how we mmap the files means we can maybe use huge pages in their load to save a nice amount of time, since julia boot time is mostly page faults.

[vchuravy]

Sure that sounds reasonable. I won't have time to tackle it though.

[KristofferC]

Doesn't make sense really for this to be on the milestone imo.

[gbaraldi]

It was on the milestone because it was initially a small fix, but on further discussion it's probably not so simple. So I think we can release 1.9 without it. If the fix isn't too difficult we can probably land it on a 1.9 patch

[vchuravy]

From the HPC call: This currently breaks PkgInspector. So the idea is to add an opt-out env variable that users of PkgInspector can set to get the copy back.