Skip to content

Commit

Permalink
Merge #364
Browse files Browse the repository at this point in the history 
364: AutoMerge should not crash if the registry consistency tests fail r=DilumAluthge a=DilumAluthge

Currently, if the registry tests fail, AutoMerge crashes ungracefully. This is not very helpful.

Instead, this PR makes the registry consistency tests into a regular AutoMerge check just like everything else.

Co-authored-by: Dilum Aluthge <dilum@aluthge.com>
  • Loading branch information
@bors @DilumAluthge
bors[bot] and DilumAluthge authored Mar 14, 2021
2 parents e180573 + 962ad9d commit 7268a0e
Show file tree
Hide file tree
Showing 5 changed files with 119 additions and 97 deletions.
2 changes: 1 addition & 1 deletion Project.toml
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name = "RegistryCI"
uuid = "0c95cc5f-2f7e-43fe-82dd-79dbcba86b32"
authors = ["Dilum Aluthge <dilum@aluthge.com>", "Fredrik Ekre <ekrefredrik@gmail.com>", "contributors"]
version = "6.8.1"
version = "6.8.2"

[deps]
Base64 = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"
Expand Down
16 changes: 16 additions & 0 deletions src/AutoMerge/guidelines.jl
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,21 @@
import HTTP

const guideline_registry_consistency_tests_pass =
Guideline("Registy consistency tests",
data -> meets_registry_consistency_tests_pass(data.registry_head,
data.registry_deps))

function meets_registry_consistency_tests_pass(registry_head::String,
registry_deps::Vector{String})
try
RegistryCI.test(registry_head; registry_deps = registry_deps)
return true, ""
catch ex
@error "" exception=(ex, catch_backtrace())
end
return false, "The registry consistency tests failed"
end

const guideline_compat_for_all_deps =
Guideline("Compat (with upper bound) for all dependencies",
data -> meets_compat_for_all_deps(data.registry_head,
Expand Down
145 changes: 75 additions & 70 deletions src/AutoMerge/new-package.jl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,53 +10,55 @@ end

function pull_request_build(data::GitHubAutoMergeData, ::NewPackage; check_license)::Nothing
# Rules:
# 1. Only changes a subset of the following files:
# - `Registry.toml`,
# - `E/Example/Compat.toml`
# - `E/Example/Deps.toml`
# - `E/Example/Package.toml`
# - `E/Example/Versions.toml`
# 2. TODO: implement this check. When implemented, this check will make sure that the changes to `Registry.toml` only modify the specified package.
# 3. Normal capitalization
# - name should match r"^[A-Z]\w*[a-z]\w*[0-9]?$"
# - i.e. starts with a capital letter, ASCII alphanumerics only, contains at least 1 lowercase letter
# 4. Not too short
# - at least five letters
# - you can register names shorter than this, but doing so requires someone to approve
# 5. Meets julia name check
# - does not include the string "julia" with any case
# - does not start with "Ju"
# 6. DISABLED. Standard initial version number - one of 0.0.1, 0.1.0, 1.0.0, X.0.0
# - does not apply to JLL packages
# 7. Repo URL ends with /$name.jl.git where name is the package name. We only apply this check if the package is not a subdirectory package.
# 8. Compat for all dependencies
# - there should be a [compat] entry for Julia
# - all [deps] should also have [compat] entries
# - all [compat] entries should have upper bounds
# - dependencies that are standard libraries do not need [compat] entries
# - dependencies that are JLL packages do not need [compat] entries
# 9. (only applies to JLL packages) The only dependencies of the package are:
# - Pkg
# - Libdl
# - other JLL packages
# 10. Package's name is sufficiently far from existing package names in the registry
# - We exclude JLL packages from the "existing names"
# - We use three checks:
# - that the lowercased name is at least 1 away in Damerau Levenshtein distance from any other lowercased name
# - that the name is at least 2 away in Damerau Levenshtein distance from any other name
# - that the name is sufficiently far in a visual distance from any other name
# 11. Package's name has only ASCII characters
# 12. Version can be installed
# - given the proposed changes to the registry, can we resolve and install the new version of the package?
# - i.e. can we run `Pkg.add("Foo")`
# 13. Package repository contains only OSI-approved license(s) in the license file at toplevel in the version being registered
# 14. Version can be loaded
# - once it's been installed (and built?), can we load the code?
# - i.e. can we run `import Foo`
# 15. Package UUID doesn't conflict with an UUID in the provided
# list of package registries. The exception is if also the
# package name *and* package URL matches those in the other
# registry, in which case this is a valid co-registration.
# 1. Registry consistency tests pass
# 2. Only changes a subset of the following files:
# - `Registry.toml`,
# - `E/Example/Compat.toml`
# - `E/Example/Deps.toml`
# - `E/Example/Package.toml`
# - `E/Example/Versions.toml`
# 3. TODO: implement this check. When implemented, this check will make sure that the changes to `Registry.toml` only modify the specified package.
# 4. Normal capitalization
# - name should match r"^[A-Z]\w*[a-z]\w*[0-9]?$"
# - i.e. starts with a capital letter, ASCII alphanumerics only, contains at least 1 lowercase letter
# 5. Not too short
# - at least five letters
# - you can register names shorter than this, but doing so requires someone to approve
# 6. Meets julia name check
# - does not include the string "julia" with any case
# - does not start with "Ju"
# 7. DISABLED. Standard initial version number - one of 0.0.1, 0.1.0, 1.0.0, X.0.0
# - does not apply to JLL packages
# 8. Repo URL ends with /$name.jl.git where name is the package name. We only apply this check if the package is not a subdirectory package.
# 9. Compat for all dependencies
# - there should be a [compat] entry for Julia
# - all [deps] should also have [compat] entries
# - all [compat] entries should have upper bounds
# - dependencies that are standard libraries do not need [compat] entries
# - dependencies that are JLL packages do not need [compat] entries
# 10. (only applies to JLL packages) The only dependencies of the package are:
# - Pkg
# - Libdl
# - other JLL packages
# 11. Package's name is sufficiently far from existing package names in the registry
# - We exclude JLL packages from the "existing names"
# - We use three checks:
# i. that the lowercased name is at least 1 away in Damerau Levenshtein distance from any other lowercased name
# ii. that the name is at least 2 away in Damerau Levenshtein distance from any other name
# iii. that the name is sufficiently far in a visual distance from any other name
# 12. Package's name has only ASCII characters
# 13. Version can be installed
# - given the proposed changes to the registry, can we resolve and install the new version of the package?
# - i.e. can we run `Pkg.add("Foo")`
# 14. Package repository contains only OSI-approved license(s) in the license file at toplevel in the version being registered
# 15. Version can be loaded
# - once it's been installed (and built?), can we load the code?
# - i.e. can we run `import Foo`
# 16. Dependency confusion check
# - Package UUID doesn't conflict with an UUID in the provided
# list of package registries. The exception is if also the
# package name *and* package URL matches those in the other
# registry, in which case this is a valid co-registration.
this_is_jll_package = is_jll_name(data.pkg)
@info("This is a new package pull request",
pkg = data.pkg,
Expand All @@ -77,29 +79,32 @@ function pull_request_build(data::GitHubAutoMergeData, ::NewPackage; check_licen
# with the results so far before continuing to the following
# guidelines.
guidelines =
[(guideline_pr_only_changes_allowed_files, true), # 1
#(guideline_only_changes_specified_package, true), # 2 (unimplemented)
(guideline_normal_capitalization,
!this_pr_can_use_special_jll_exceptions), # 3
(guideline_name_length,
!this_pr_can_use_special_jll_exceptions), # 4
(guideline_julia_name_check, true), # 5
#(guideline_standard_initial_version_number,
# !this_pr_can_use_special_jll_exceptions), # 6 (deactivated)
(guideline_repo_url_requirement, true), # 7
(guideline_compat_for_all_deps, true), # 8
(guideline_allowed_jll_nonrecursive_dependencies,
this_is_jll_package), # 9
(guideline_distance_check, true), # 10
(guideline_name_ascii, true), # 11
(:update_status, true),
(guideline_version_can_be_pkg_added, true), # 12
# `guideline_version_has_osi_license` must be run after
# `guideline_version_can_be_pkg_added` so that it can use the downloaded code!
(guideline_version_has_osi_license, check_license), # 13
(guideline_version_can_be_imported, true), # 14
(:update_status, true),
(guideline_dependency_confusion, true)] # 15
[
(guideline_registry_consistency_tests_pass, true), # 1
(guideline_pr_only_changes_allowed_files, true), # 2
# (guideline_only_changes_specified_package, true), # 3 (unimplemented)
(guideline_normal_capitalization, # 4
!this_pr_can_use_special_jll_exceptions),
(guideline_name_length, # 5
!this_pr_can_use_special_jll_exceptions),
(guideline_julia_name_check, true), # 6
# (guideline_standard_initial_version_number, # 7 (deactivated)
# !this_pr_can_use_special_jll_exceptions),
(guideline_repo_url_requirement, true), # 8
(guideline_compat_for_all_deps, true), # 9
(guideline_allowed_jll_nonrecursive_dependencies, # 10
this_is_jll_package),
(guideline_distance_check, true), # 11
(guideline_name_ascii, true), # 12
(:update_status, true),
(guideline_version_can_be_pkg_added, true), # 13
# `guideline_version_has_osi_license` must be run after
# `guideline_version_can_be_pkg_added` so that it can use the downloaded code!
(guideline_version_has_osi_license, check_license), # 14
(guideline_version_can_be_imported, true), # 15
(:update_status, true),
(guideline_dependency_confusion, true), # 16
]


checked_guidelines = Guideline[]
Expand Down
50 changes: 27 additions & 23 deletions src/AutoMerge/new-version.jl
View file
Original file line number Diff line number Diff line change
@@ -1,31 +1,32 @@
function pull_request_build(data::GitHubAutoMergeData, ::NewVersion; check_license)::Nothing
# Rules:
# 1. Only changes a subset of the following files:
# 1. Registry consistency tests pass
# 2. Only changes a subset of the following files:
# - `E/Example/Compat.toml`
# - `E/Example/Deps.toml`
# - `E/Example/Versions.toml`
# 2. Sequential version number
# 3. Sequential version number
# - if the last version was 1.2.3 then the next can be 1.2.4, 1.3.0 or 2.0.0
# - does not apply to JLL packages
# 3. Compat for all dependencies
# 4. Compat for all dependencies
# - there should be a [compat] entry for Julia
# - all [deps] should also have [compat] entries
# - all [compat] entries should have upper bounds
# - dependencies that are standard libraries do not need [compat] entries
# - dependencies that are JLL packages do not need [compat] entries
# 4. If it is a patch release, then it does not narrow the Julia compat range
# 5. (only applies to JLL packages) The only dependencies of the package are:
# 5. If it is a patch release, then it does not narrow the Julia compat range
# 6. (only applies to JLL packages) The only dependencies of the package are:
# - Pkg
# - Libdl
# - other JLL packages
# 6. Version can be installed
# 7. Version can be installed
# - given the proposed changes to the registry, can we resolve and install the new version of the package?
# - i.e. can we run `Pkg.add("Foo")`
# 7. Package repository contains only OSI-approved license(s) in the license file at toplevel in the version being registered
# 8. Version can be loaded
# 8. Package repository contains only OSI-approved license(s) in the license file at toplevel in the version being registered
# 9. Version can be loaded
# - once it's been installed (and built?), can we load the code?
# - i.e. can we run `import Foo`

this_is_jll_package = is_jll_name(data.pkg)
@info("This is a new package pull request",
pkg = data.pkg,
Expand All @@ -46,20 +47,23 @@ function pull_request_build(data::GitHubAutoMergeData, ::NewVersion; check_licen
# with the results so far before continuing to the following
# guidelines.
guidelines =
[(guideline_pr_only_changes_allowed_files, true), # 1
(guideline_sequential_version_number,
!this_pr_can_use_special_jll_exceptions), # 2
(guideline_compat_for_all_deps, true), # 3
(guideline_patch_release_does_not_narrow_julia_compat,
!this_pr_can_use_special_jll_exceptions), # 4
(guideline_allowed_jll_nonrecursive_dependencies,
this_is_jll_package), # 5
(:update_status, true),
(guideline_version_can_be_pkg_added, true), # 6
# `guideline_version_has_osi_license` must be run after
# `guideline_version_can_be_pkg_added` so that it can use the downloaded code!
(guideline_version_has_osi_license, check_license), # 7
(guideline_version_can_be_imported, true)] # 8
[
(guideline_registry_consistency_tests_pass, true), # 1
(guideline_pr_only_changes_allowed_files, true), # 2
(guideline_sequential_version_number, # 3
!this_pr_can_use_special_jll_exceptions),
(guideline_compat_for_all_deps, true), # 4
(guideline_patch_release_does_not_narrow_julia_compat, # 5
!this_pr_can_use_special_jll_exceptions),
(guideline_allowed_jll_nonrecursive_dependencies, # 6
this_is_jll_package),
(:update_status, true),
(guideline_version_can_be_pkg_added, true), # 7
# `guideline_version_has_osi_license` must be run after
# `guideline_version_can_be_pkg_added` so that it can use the downloaded code!
(guideline_version_has_osi_license, check_license), # 8
(guideline_version_can_be_imported, true), # 9
]

checked_guidelines = Guideline[]

Expand Down
3 changes: 0 additions & 3 deletions src/AutoMerge/public.jl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,6 @@ function run(env = ENV,

registry_head = directory_of_cloned_registry(cicfg; env=env)

# Run tests on the registry (these are very quick)
RegistryCI.test(registry_head; registry_deps = registry_deps)

# Figure out what type of build this is
run_pr_build = conditions_met_for_pr_build(cicfg; env=env, master_branch=master_branch)
run_merge_build = conditions_met_for_merge_build(cicfg; env=env, master_branch=master_branch)
Expand Down

2 comments on commit 7268a0e

@DilumAluthge
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JuliaRegistrator register

@JuliaRegistrator
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registration pull request created: JuliaRegistries/General/31938

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a v6.8.2 -m "<description of version>" 7268a0efbf4c9e0890d97ff893463e6fc164c807
git push origin v6.8.2

Please sign in to comment.