add support for package extensions #3264

KristofferC · 2022-11-24T16:02:31Z

Requires JuliaLang/julia#47695

Make `Pkg.precompile` work with glue packages

KristofferC · 2022-11-25T08:51:24Z

Tweaked the printing for glue pkgs in precompilation:

KristofferC · 2022-11-25T11:11:59Z

Status output:

Shows you glue packages and glue dependencies and which ones are loaded (in green).

IanButterworth · 2022-11-25T12:37:51Z

Awesome. Should there be a + before the enabled glue packages or something, so that it's not just color

KristofferC · 2022-11-25T13:45:49Z

Yeah, I can add that

docs/src/creating-packages.md

src/API.jl

Co-authored-by: Ian Butterworth <i.r.butterworth@gmail.com>

Seelengrab · 2022-11-26T08:52:44Z

docs/src/creating-packages.md

+  callback only when glue packages are not supported
+  ```julia
+  # This symbol is only defined on Julia versions that support glue packages
+  if isdefined(Base, :get_gluepkg)


Suggested change

if isdefined(Base, :get_gluepkg)

if !isdefined(Base, :get_gluepkg)

If I understand correctly?

Yes, thanks :)

halleysfifthinc · 2022-11-28T17:55:48Z

Should this also include a REPL command to add glue packages, or is this experimental enough that the accessibility of a convenient REPL mode isn't desired?

Something like ] add --glue PkgName [Glue($PkgName)] [glue/PkgName] where ungiven optional glue package name and location arguments would be assumed to be "Glue$PkgName" and ProjectDir/glue/PkgName.jl or ProjectDir/glue/PkgName/PkgName.jl, respectively?

KristofferC · 2022-11-28T18:11:53Z

It could be added but it isn't rely necessary for the core functionality so maybe in another PR.

quinnj · 2022-11-29T07:29:31Z

docs/src/creating-packages.md

+objects from a wide variety of different Julia packages.
+Adding all those different Julia packages as dependencies
+could be expensive since they would end up getting loaded even if they were never used.
+Instead, these code required to plot objects for specific packages can be put into separate files


Suggested change

Instead, these code required to plot objects for specific packages can be put into separate files

Instead, the code required to plot objects for specific packages can be put into separate files

docs/src/creating-packages.md

quinnj · 2022-11-29T07:48:05Z

src/Registry/registry_instance.jl

@@ -53,6 +54,12 @@ struct PkgInfo

    # Deps.toml
    deps::Dict{VersionRange, Dict{String, UUID}}
+
+    # WeakCompat.toml
+    glue_compat::Dict{VersionRange, Dict{String, VersionSpec}}


So there will be new registry files written for these then? i.e. we'll need to update RegistryTools.jl to support these as well?

Yes, there already is a branch on LocalRegistry.jl supporting this.

quinnj · 2022-11-29T07:54:21Z

src/Operations.jl

@@ -1127,7 +1179,7 @@ function rm(ctx::Context, pkgs::Vector{PackageSpec}; mode::PackageMode)
    # only declare `compat` for remaining direct or `extra` dependencies
    # `julia` is always an implicit direct dependency
    filter!(ctx.env.project.compat) do (name, _)
-        name == "julia" || name in keys(ctx.env.project.deps) || name in keys(ctx.env.project.extras)
+        name == "julia" || name in keys(ctx.env.project.deps) || name in keys(ctx.env.project.extras) || name in keys(ctx.env.project.gluedeps)


I tend to prefer haskey(dict, name) than name in keys(dict)

Yeah, I just kept the existing way of writing it but that is better indeed.

quinnj · 2022-11-29T07:55:16Z

src/Operations.jl

    new = download_source(ctx)
+    fixup_glue!(ctx.env, pkgs)


Is it possible to call fixup_glue! from download_source? Or are there cases where we download, but don't want to fixup glue?

Maybe. Or have another function that calls both of them.

Before this PR we could create the manifest without even having to download the packages. Now, we need to look up the glue packages in the downloaded packages project file since that information is not stored in the registry.

An alternative to this is to not store the glue package mapping in the manifest at all and just look it up in the package's project file in code loading. But we have so far tried to avoid that opting for storing everything required for code loading in the manifest...

IanButterworth

LGTM. Just some minor formatting and wording etc.

docs/src/creating-packages.md

src/Pkg.jl

src/Types.jl

src/Operations.jl

It is possible to think that `Registry` may be a standalone Pkg command with the previous docstring (e.g. a command that constructs or displays registry info).

Co-authored-by: Lilith Hafner <Lilith.Hafner@gmail.com>

`APIOptions` is defined as an alias for `Dict{Symbol, Any}`. Consequently, the definition of a constructor for `APIOptions` introduced a new constructor for Dict. This commit removes that constructor.

`PackageToken` is a Union of multiple struct types. This commit removes a constructor for `PackageToken` and replaces it with a function `packagetoken`. Furthermore, this commit removes `PackageIdentifier` as an alias for String and replaces that with a struct type, which wraps a String containing a package identifier.

add support for glue packages

e0954b9

KristofferC mentioned this pull request Nov 24, 2022

Add support for "package extensions" to code loading JuliaLang/julia#47695

Merged

add some general explanation comments

2d6adbb

IanButterworth mentioned this pull request Nov 25, 2022

Make Pkg.precompile work with glue packages #3265

Merged

1 task

IanButterworth and others added 2 commits November 25, 2022 00:26

precompile glue packages too

83ad655

add support for glue packages

1a69736

KristofferC force-pushed the kc/glue branch from e0954b9 to 1a69736 Compare November 25, 2022 07:32

KristofferC added 3 commits November 25, 2022 08:39

Merge pull request #3265 from IanButterworth/ib/glue_precompile

8ae5d35

Make `Pkg.precompile` work with glue packages

fix order of sections in Project file

041e9f0

tweak printing of glue pkgs in precompilation

f2376f8

KristofferC added 2 commits November 25, 2022 10:30

add docs for backwards compat

6eb01e1

add a status option for glue

b591e39

IanButterworth reviewed Nov 25, 2022

View reviewed changes

docs/src/creating-packages.md Outdated Show resolved Hide resolved

src/API.jl Outdated Show resolved Hide resolved

src/API.jl Outdated Show resolved Hide resolved

KristofferC added 3 commits November 25, 2022 15:57

fixup precompile

e23c49e

add tests for glue packages retrieved from a registry

941998b

add small note

2bfd2ab

IanButterworth reviewed Nov 25, 2022

View reviewed changes

src/API.jl Outdated Show resolved Hide resolved

KristofferC and others added 5 commits November 25, 2022 19:14

Update docs/src/creating-packages.md

abc9e21

Co-authored-by: Ian Butterworth <i.r.butterworth@gmail.com>

Update src/API.jl

92bed4c

Co-authored-by: Ian Butterworth <i.r.butterworth@gmail.com>

Update src/API.jl

49cd28a

Co-authored-by: Ian Butterworth <i.r.butterworth@gmail.com>

fixup fixup_glue

a9e5ffa

weak -> glue

c52bf27

Seelengrab reviewed Nov 26, 2022

View reviewed changes

fix tests passing when dep warnings are errors (as they are on Julia CI)

f1a47ed

KristofferC force-pushed the kc/glue branch from 243f72d to f1a47ed Compare November 27, 2022 14:18

quinnj reviewed Nov 29, 2022

View reviewed changes

docs/src/creating-packages.md Outdated Show resolved Hide resolved

quinnj reviewed Nov 29, 2022

View reviewed changes

docs/src/creating-packages.md Outdated Show resolved Hide resolved

quinnj reviewed Nov 29, 2022

View reviewed changes

KristofferC added 3 commits November 29, 2022 16:35

gluedeps -> weakdeps

169e128

ignore packages both in weakdeps and deps for backwards compat

fa6dbfe

glue -> extension

abe7000

KristofferC changed the title ~~add support for glue packages~~ add support for package extensions Dec 3, 2022

KristofferC mentioned this pull request Dec 3, 2022

support weak dependencies JuliaRegistries/RegistryTools.jl#74

Merged

IanButterworth approved these changes Dec 7, 2022

View reviewed changes

KristofferC force-pushed the kc/glue branch from a6d7450 to 511dc0b Compare December 7, 2022 18:18

KristofferC and others added 6 commits December 7, 2022 19:38

fix REPL for extension option

5085eb7

Clarify RegistrySpec docstring (#3155)

644a176

It is possible to think that `Registry` may be a standalone Pkg command with the previous docstring (e.g. a command that constructs or displays registry info).

slightly simplify API internals (#3156)

50a0938

Co-authored-by: Lilith Hafner <Lilith.Hafner@gmail.com>

REPLMode: Remove constructor for APIOptions

0c2bafd

`APIOptions` is defined as an alias for `Dict{Symbol, Any}`. Consequently, the definition of a constructor for `APIOptions` introduced a new constructor for Dict. This commit removes that constructor.

review fixes

a2aef89

KristofferC force-pushed the kc/glue branch from 511dc0b to a2aef89 Compare December 7, 2022 18:39

KristofferC merged commit 5d8b9dd into master Dec 7, 2022

KristofferC deleted the kc/glue branch December 7, 2022 19:25

IanButterworth mentioned this pull request Jan 2, 2023

WIP: Support for weak dependencies #3216

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add support for package extensions #3264

add support for package extensions #3264

KristofferC commented Nov 24, 2022 •

edited

Loading

KristofferC commented Nov 25, 2022

KristofferC commented Nov 25, 2022

IanButterworth commented Nov 25, 2022

KristofferC commented Nov 25, 2022

Seelengrab Nov 26, 2022

KristofferC Nov 26, 2022

halleysfifthinc commented Nov 28, 2022

KristofferC commented Nov 28, 2022 •

edited

Loading

quinnj Nov 29, 2022

quinnj Nov 29, 2022

KristofferC Nov 29, 2022

quinnj Nov 29, 2022

KristofferC Nov 29, 2022

quinnj Nov 29, 2022

KristofferC Nov 29, 2022

IanButterworth left a comment

	if isdefined(Base, :get_gluepkg)
	if !isdefined(Base, :get_gluepkg)

	Instead, these code required to plot objects for specific packages can be put into separate files
	Instead, the code required to plot objects for specific packages can be put into separate files

add support for package extensions #3264

add support for package extensions #3264

Conversation

KristofferC commented Nov 24, 2022 • edited Loading

KristofferC commented Nov 25, 2022

KristofferC commented Nov 25, 2022

IanButterworth commented Nov 25, 2022

KristofferC commented Nov 25, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

halleysfifthinc commented Nov 28, 2022

KristofferC commented Nov 28, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

IanButterworth left a comment

Choose a reason for hiding this comment

KristofferC commented Nov 24, 2022 •

edited

Loading

KristofferC commented Nov 28, 2022 •

edited

Loading