Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

disallow setindex on immutable values #34176

Merged
merged 1 commit into from
Dec 27, 2019
Merged

Conversation

vtjnash
Copy link
Member

@vtjnash vtjnash commented Dec 21, 2019

It's generally not a great idea to break the object model, even with the best of intentions, since you're inherently then relying on the compiler to make specific optimization choices and avoid others. This was written fairly carefully to be safe at the time, assuming it was not improperly optimized. But others are not as careful when copying this code. And it is just better not to break the object model and attempt to mutate constant values.

I started implementing equivalent optimizations of the old code ("SmallArray"), but then surmised that those seemed unlikely to be actually necessary.

@vtjnash
Copy link
Member Author

vtjnash commented Dec 21, 2019

@nanosoldier runbenchmarks(ALL, vs=":master")

@nanosoldier
Copy link
Collaborator

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

@vtjnash vtjnash changed the title RFC: disallow setindex on immutable values disallow setindex on immutable values Dec 23, 2019
@vtjnash vtjnash force-pushed the jn/no-immutable-setindex branch from 0f63778 to 77624c0 Compare December 23, 2019 18:46
@vtjnash
Copy link
Member Author

vtjnash commented Dec 23, 2019

OK, added some extra checks and tests for those. We can never entirely stop you from shooting yourself in the foot, but we can still try.

src/datatype.c Outdated
@@ -923,6 +924,10 @@ static void init_struct_tail(jl_datatype_t *type, jl_value_t *jv, size_t na)
JL_DLLEXPORT jl_value_t *jl_new_structv(jl_datatype_t *type, jl_value_t **args, uint32_t na)
{
jl_ptls_t ptls = jl_get_ptls_states();
if (!jl_is_datatype(type) || type->layout == NULL)
jl_type_error("new", (jl_value_t*)jl_datatype_type, (jl_value_t*)type);
if ((type->ninitialized > na && !type->mutabl) || na > jl_datatype_nfields(type))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we weren't doing this check at all before it doesn't really matter, but doesn't the ninitialized check make sense for both mutable and immutable structs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, yeah, probably. I was concerned about causing too much breakage of otherwise mostly valid code, but it seems that jl_new_struct_uninit is usually what's needed when someone wants an incomplete mutable type, so I've gone ahead and removed the extra check here.

This was written fairly carefully to be safe, assuming it was not improperly optimized.
But others are not as careful when copying this code. And it is just better not to break the object model and attempt to mutate constant values.
@vtjnash vtjnash force-pushed the jn/no-immutable-setindex branch from 77624c0 to f115b14 Compare December 23, 2019 19:10
@JeffBezanson JeffBezanson merged commit acb7bd9 into master Dec 27, 2019
@JeffBezanson JeffBezanson deleted the jn/no-immutable-setindex branch December 27, 2019 21:20
@maleadt
Copy link
Member

maleadt commented Dec 28, 2019

There should probably have been run a PkgEval here?
From acb7bd9#commitcomment-36599963, (at least) the following new failures seem related:

@JeffBezanson
Copy link
Member

Those all look expected to me. This is designed to prevent using ccall to mutate an immutable.

@timholy
Copy link
Member

timholy commented Dec 29, 2019

The failures in LoweredCodeUtils and Revise appear to flow directly from JuliaInterpreter, so those three can basically be collapsed down to 1.

For JuliaInterpreter, we use this to evaluate new expressions: https://github.com/JuliaDebug/JuliaInterpreter.jl/blob/5611e3c91785e3bf1ccad7f6fca20c501eb1ac38/src/interpret.jl#L365-L368. It's amusing that @vtjnash both suggested this strategy (JuliaDebug/ASTInterpreter2.jl#37 (comment)) and wrote the Julia commit that broke it 😛. Not that I expect perfect consistency, I change my mind all the time.

Any thoughts about alternatives? If we Core.eval the %new expressions, it's about a 2x performance hit on

julia> nt = (a=1,)
(a = 1,)

julia> @interpret pairs(nt)

That can dominate real-world test cases, e.g., as mentioned here running Julia's subarray tests under the interpreter is mostly a question of how fast you can evaluate %new expressions.

@timholy
Copy link
Member

timholy commented Dec 29, 2019

Ah, nvm, this PR shows how to circumvent it. For the benefit of the other packages, one just has to prepare the args in advance and ccall(:jl_new_structv, ...).

timholy added a commit to JuliaDebug/JuliaInterpreter.jl that referenced this pull request Dec 29, 2019
timholy added a commit to JuliaDebug/JuliaInterpreter.jl that referenced this pull request Dec 29, 2019
KristofferC pushed a commit that referenced this pull request Apr 11, 2020
This was written fairly carefully to be safe, assuming it was not improperly optimized.
But others are not as careful when copying this code. And it is just better not to break the object model and attempt to mutate constant values.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants