Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allows users to set a dependency mirror #563

Merged
merged 1 commit into from
Apr 22, 2024

Conversation

TisVictress
Copy link
Contributor

@TisVictress TisVictress commented Apr 4, 2024

Implements dependency-mirrors RFC

A new release of each buildpack will need to be cut with the updated version of packit containing this change.

Reason for this change:
While there is already support for dependency mappings, this adds support for dependency mirrors. The difference is the user would not need to specify a mapping for each individual dependency. Dependency mirrors are more efficient because the user can set one mirror for all or multiple dependencies.

The mirror can be set via an environment variable or a binding of type dependency-mirror. In addition, multiple mirror URLs can be set, a default mirror with extra specific hostname mirrors. For example, if a user wants to download all dependencies from the same place except for dependencies that originated in github (see below). The default mirror can contain a placeholder, {originalHost}, to support mirrors that host many different repositories, which in some cases include the original hostname in the path. If only specific hostname mirrors are set, the appropriate dependencies will be download via those mirrors while everything else will be downloaded through the original URLs in the buildpack's metadata.

Example mirrors via environment variables:

BP_DEPENDENCY_MIRROR              https://mirror.example.org/{originalHost}
BP_DEPENDENCY_MIRROR_GITHUB_COM   https://mirror.example.org/public-github
BP_DEPENDENCY_MIRROR_NODEJS_ORG   https://mirror.example.org/node-dist

Example of mirrors via binding

/platform
    └── bindings
        └── dependency-mirror
            ├── default                https://mirror.example.org/{originalHost}
            ├── github.com             https://mirror.example.org/public-github
            ├── nodejs.org             https://mirror.example.org/node-dist
            └── type                   dependency-mirror

If a dependency mapping and a dependency mirror is set, an error will be generated. Dependency mappings and mirrors cannot be used simultaneously.

If a binding is used to set dependency mirrors, there can only be one binding of type dependency-mirror, containing multiple Entries for each mirror URL.

If both methods, environment variable(s) and binding, are used to set a dependency mirror, the mirror(s) from the environment variable(s) will be chosen, as specified in the above RFC.

@TisVictress
Copy link
Contributor Author

@sophiewigmore I've updated the description. Let me know if you have any questions.

postal/service.go Outdated Show resolved Hide resolved
@sophiewigmore
Copy link
Member

Do we need to add anything to deal with the username/password parts of the URI? and what about the file scheme (https or file). Can the postal.Deliver method deal with both of these things already? Might need tests for these scenarios too if we need to change anything

@TisVictress
Copy link
Contributor Author

Do we need to add anything to deal with the username/password parts of the URI?

If a username and password are included in the url, it's captured by url.Parse which is a go-builtin function.

@TisVictress
Copy link
Contributor Author

Can the postal.Deliver method deal with both of these things already?

@sophiewigmore I'll double check and get back to you.

@TisVictress
Copy link
Contributor Author

@sophiewigmore cargo.Transport handles both https and file.

I did some adhoc/integration testing with the nginx buildpack, inserting user:pass@ before the beginning of the uri and after the uri scheme. For example,

https://user:pass@artifacts.paketo.io/nginx/nginx_1.24.0_linux_x64_jammy_fa1dfe46.tgz

Usernames and passwords are handled by cargo.Transport implicitly because it does not tamper with the uri before passing it directly to the http request. (The username and password are put in the Authorization header of the request)

I've updated the PR to address all the above suggestions.

postal/service.go Outdated Show resolved Hide resolved
@sophiewigmore sophiewigmore added the semver:minor A change requiring a minor version bump label Apr 22, 2024
@sophiewigmore sophiewigmore merged commit 4c9f338 into paketo-buildpacks:v2 Apr 22, 2024
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver:minor A change requiring a minor version bump
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants