Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raise an error when using include_dependency with non-existent file #52105

Closed
wants to merge 2 commits into from

Conversation

longqian95
Copy link
Contributor

Since depending on a non-existent file or directory is strange, raise an error instead of silently triggering precompilation.

Fixes #52063

Since depending on a non-existent file or directory is strange, raise an error instead of silently triggering precompilation.

Fixes JuliaLang#52063
@@ -1677,6 +1677,7 @@ function _include_dependency(mod::Module, _path::AbstractString; track_content=t
push!(_require_dependencies,
(mod, path, filesize(path), open(_crc32c, path, "r"), -1.0))
else
@assert ispath(path) "'$path': No such file or directory"
Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good but assert is the wrong choice here. Just check and throw a meaningful error type

@@ -1677,6 +1677,7 @@ function _include_dependency(mod::Module, _path::AbstractString; track_content=t
push!(_require_dependencies,
(mod, path, filesize(path), open(_crc32c, path, "r"), -1.0))
else
@assert ispath(path) "'$path': No such file or directory"
Copy link
Member

@stevengj stevengj Nov 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
@assert ispath(path) "'$path': No such file or directory"
(isfile(path) && uperm(path) & 0x04 == 0x04) ||
throw(ArgumentError("$(repr(path)) is not a readable file"))

(It has to be a readable file — not just a path, e.g. not a directory — for include_dependency, because Julia will compute a CRC hash of the contents IIRC. And repr is the right way to convert the path to a quoted string.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The perm check should be moved to the other branch, because incluce_dependencys are only tracked by mtime, or does that also require read permissions?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mtime doesn't require read permissions, sorry. But I guess #51798 will affect this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#51798 is concerned with the other branch. If the file does not exist there, then the error is thrown through open().

@stevengj stevengj added the needs tests Unit tests are required for this change label Nov 15, 2023
@stevengj
Copy link
Member

You should also add a test to test/precompile.jl, e.g. something like:

@test_throws ArgumentError include_dependency("i_do_not_exist")
@test_throws ArgumentError include_dependency(@__DIR__)

@fatteneder
Copy link
Member

This test here is failing now

@test map(x -> x.filename, deps) == [ Foo_file, joinpath(dir, "foo.jl"), joinpath(dir, "bar.jl") ]
because the include_dependency() files foo.jl, bar.jl are never written to disk.

I would suggest to update the test so that the files actually exist before precompilation. And then add another test as suggested by @stevengj above.

@fatteneder fatteneder added this to the 1.11 milestone Feb 7, 2024
@vtjnash vtjnash closed this Apr 16, 2024
vchuravy pushed a commit that referenced this pull request Jun 21, 2024
… or directory (#53286)

Replaces #52105
Fixes #52063

There is a question about whether the `ispath & uperm` check should be
moved inside the `_track_dependencies[]` check (like it was in #52105),
which would make it such that any errors are thrown only during
precompilation.

---------

Co-authored-by: Qian Long <longqian95@gmail.com>
Co-authored-by: Jameson Nash <vtjnash@gmail.com>
KristofferC pushed a commit that referenced this pull request Jul 23, 2024
… or directory (#53286)

Replaces #52105
Fixes #52063

There is a question about whether the `ispath & uperm` check should be
moved inside the `_track_dependencies[]` check (like it was in #52105),
which would make it such that any errors are thrown only during
precompilation.

---------

Co-authored-by: Qian Long <longqian95@gmail.com>
Co-authored-by: Jameson Nash <vtjnash@gmail.com>
(cherry picked from commit 71fa11f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs tests Unit tests are required for this change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Using include_dependency with a non-existent file triggers precompilation at every startup
5 participants