Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix GCExt test #47699

Merged
merged 6 commits into from
Nov 27, 2022
Merged

Fix GCExt test #47699

merged 6 commits into from
Nov 27, 2022

Conversation

vchuravy
Copy link
Member

Noticed while working on #47407

This makes me wonder if gcext is run on CI.

@fingolfin
Copy link
Contributor

Ugh! Thank you!

src/gc.c Outdated Show resolved Hide resolved
@brenhinkeller brenhinkeller added multithreading Base.Threads and related functionality GC Garbage collector labels Nov 25, 2022
@vchuravy vchuravy changed the title Fix GCExt test with resizable threadpool Fix GCExt test Nov 25, 2022
@vchuravy vchuravy added the backport 1.9 Change should be backported to release-1.9 label Nov 25, 2022
@@ -307,6 +307,7 @@ static size_t gc_alloc_size(jl_value_t *val)

int internal_obj_scan(jl_value_t *val)
{
// FIXME: `jl_gc_internal_obj_base_ptr` is not allowed to be called from outside GC
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rbehrends we already register a task scanner and a root scanner callback in gcext, which exercise jl_gc_internal_obj_base_ptr. So could we just remove internal_obj_scan? (I guess if we did that, a major part of gcext.c and LocalTest.jl would end up dead code and could be removed...)

Or do you think it's still valuable? If so, perhaps the code @vchuravy added here in a previous revision of the PR (and which is still visible in https://github.com/JuliaLang/julia/pull/47407/files) which sets/restores gc_n_threads and gc_all_tls_states, could simply be inserted into internal_obj_scan ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, the purpose of that part of the test was that all internal object references are properly recognized, so ideally we'd want to keep it around. It's to protect against some future evolution of the GC where the logic might not work for some reason or another.

As I said, I haven't had an opportunity yet to look at exactly what the problem was, but if we could just add code to internal_obj_scan() to make it possible to call jl_gc_internal_obj_base_ptr() from there, that would indeed be the ideal solution.

An alternative would be to check internal_obj_scan() inside mark_stack() and mark_stack_data(). This would require maintaining the failure counter at the C level, but might ultimately be the less hacky solution.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is more that you are looking at data that is managed by the GC without holding the GC "lock". So the data might be mutated underneath you. So if we could move this to inside the mark_stack that would be better.

@vchuravy
Copy link
Member Author

@rbehrends when you have time it would be good to take a look at the test and re-engineer it so that it does what you need to test. But at least we are now running the tests as art of CI instead of ignoring them

KristofferC pushed a commit that referenced this pull request Nov 28, 2022
* Add test/gcext to out-of-tree
* Disable gcext test that uses jl_gc_internal_obj_base_ptr

(cherry picked from commit 5495b8d)
KristofferC pushed a commit that referenced this pull request Nov 28, 2022
* Add test/gcext to out-of-tree
* Disable gcext test that uses jl_gc_internal_obj_base_ptr

(cherry picked from commit 5495b8d)
@KristofferC KristofferC mentioned this pull request Dec 14, 2022
26 tasks
@KristofferC KristofferC removed the backport 1.9 Change should be backported to release-1.9 label Dec 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
GC Garbage collector multithreading Base.Threads and related functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants