Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upgrade selected GAP packages at startup #3222

Merged
merged 3 commits into from
Jan 26, 2024

Conversation

ThomasBreuer
Copy link
Member

resolves #3219

@lgoettgens
Copy link
Member

I'll try to explain the Aqua issue reported here: A packages __init__() function gets called during precompilation of its dependents. In OSCAR's case (with this PR), GAP.Packages.install is called which creates a Downloads.Downloader object. This keeps the connection open for an extended period of time, which hinders the precompilation of OSCAR to finish (but only if OSCAR is a dependency of something).
A fix for something similar in Polymake.jl is oscar-system/Polymake.jl#466 by @benlorenz. There also is a "real" fix in JuliaLang/Downloads.jl#234, but this will only be available in 1.11+.

@benlorenz
Copy link
Member

benlorenz commented Jan 23, 2024

That Downloads.jl fix will also be in 1.10.1.

Edit:
But I am not totally sure if this will fix this issue on 1.10 as there might be some other changes in the atexit code on nightly.

Also there might be something else involved here, since that Downloads.jl issue should be fixed and included on the latest julia nightly (via JuliaLang/julia#52885).
But it did error even on nightly.

@lgoettgens
Copy link
Member

Hm, I don't see it in JuliaLang/julia#52755 or am I just blind?

@benlorenz
Copy link
Member

The commit is there but it doesn't seem to appear in the list: JuliaLang/julia@aaaf5de (#52755)
Maybe because it was added to that branch via a separate PR: JuliaLang/julia#52886

@ThomasBreuer
Copy link
Member Author

@lgoettgens @benlorenz Thanks for the information, but I do not understand what the proposal is.
The failing tests with Julia 1.10 are run with Julia 1.10.0, which does not know Downloads.default_downloader.
Thus the cited fix from Polymake.jl would not help there. Or what am I missing?
And if I understand @benlorenz' comment right then the failure in Julia nightly (also in the Aqua.jl tests) indicates another (?) problem.

@benlorenz
Copy link
Member

1.10 should have that function, it has an exclamation mark at the end:

julia> Downloads.default_downloader!
default_downloader! (generic function with 2 methods)

I am indeed confused why this also happens on nightly which is supposed to clean up these downloaders automatically. Especially since the following seems to work, even though it is not really nice:

  GAP.Packages.Downloads.default_downloader!(GAP.Packages.Downloads.Downloader(grace=0.01))
  for (pkg, version) in [
     ("repsn", "3.1.1"),
     ]
    GAP.Packages.install(pkg, version, interactive = false, quiet = false)
  end
  GAP.Packages.Downloads.default_downloader!()

Note that this would need some extra if statement to skip these calls for 1.9 and older.

Another option would be to add a custom Downloader in the GAP packages.jl file like this:

   dlhelper = Downloads.Downloader(; grace=0.1)
...
        buffer = Downloads.download(String(url), IOBuffer(); downloader=dlhelper)

The dlhelper can probably be created in the init_packagemanager function and be used in both lambda functions.

@ThomasBreuer
Copy link
Member Author

Thanks @benlorenz.

Logically, the right place to fix the problem is in GAP.jl, as you suggest.
I have created a pull request for that.

This does not solve the problem in Oscar until the next release of GAP.jl. I am going to add the "not really nice" solution sketched above to the current pull request.

Copy link

codecov bot commented Jan 25, 2024

Codecov Report

Merging #3222 (b83b4e6) into master (e5a2cdf) will increase coverage by 0.05%.
Report is 14 commits behind head on master.
The diff coverage is 100.00%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3222      +/-   ##
==========================================
+ Coverage   81.62%   81.68%   +0.05%     
==========================================
  Files         546      546              
  Lines       72877    73059     +182     
==========================================
+ Hits        59489    59677     +188     
+ Misses      13388    13382       -6     
Files Coverage Δ
src/Oscar.jl 62.02% <100.00%> (+2.56%) ⬆️

... and 26 files with indirect coverage changes

src/Oscar.jl Outdated
@@ -80,9 +80,21 @@ function __init__()
GAP.Globals.BindGlobal(GapObj("Oscar"), Oscar)
GAP.Globals.SetPackagePath(GAP.Obj("OscarInterface"), GAP.Obj(joinpath(@__DIR__, "..", "gap", "OscarInterface")))
GAP.Globals.LoadPackage(GAP.Obj("OscarInterface"))
# Work around a problem in Julia 1.10 (see https://github.com/oscar-system/Oscar.jl/pull/3222).
isdefined(GAP.Packages.Downloads, :default_downloader!) && GAP.Packages.Downloads.default_downloader!(GAP.Packages.Downloads.Downloader(grace=0.01))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have just release GAP.jl 0.10.2 so you could remove this and update Project.toml instead.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great. Thanks.

@fingolfin fingolfin enabled auto-merge (squash) January 26, 2024 09:16
@fingolfin fingolfin merged commit cc01f7b into oscar-system:master Jan 26, 2024
20 checks passed
@ThomasBreuer ThomasBreuer deleted the TB_upgrade_repsn branch January 26, 2024 10:06
HechtiDerLachs pushed a commit to HechtiDerLachs/Oscar.jl that referenced this pull request Jan 26, 2024
ooinaruhugh pushed a commit to ooinaruhugh/Oscar.jl that referenced this pull request Feb 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Get GAP package repsn 3.1.1 or newer into Oscar
4 participants