Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vector2.__new__ violates its interface contract #146

Closed
wants to merge 3 commits into from

Conversation

nbraud
Copy link
Collaborator

@nbraud nbraud commented Apr 11, 2019

While working on #144, I discovered that instances of Vector2 cannot be copied (using copy.{copy,deepcopy}), and I believe this is because we are violating the interface contract for __new__

nbraud added a commit to nbraud/ppb-vector that referenced this pull request Apr 11, 2019
@nbraud
Copy link
Collaborator Author

nbraud commented Apr 11, 2019

@astronouth7303 Assigning this to you because I do not know well-enough the internals of the Python object model :3

@nbraud nbraud mentioned this pull request Apr 12, 2019
7 tasks
@nbraud
Copy link
Collaborator Author

nbraud commented Apr 12, 2019

According to __new__'s spec. __init__ is called automatically afterwards, so I moved more of the logic back in it (with the added benefit that subclasses will get the non-copying behaviour “for free”).
It looks like this resulted in some progress:

>>> import copy
>>> copy.copy(Vector2(0, 0))                                                                 
---------------------------------------------------------------------------
FrozenInstanceError                       Traceback (most recent call last)
<ipython-input-6-7fbabe237cc4> in <module>
----> 1 copy.copy(Vector2(0, 0))

/usr/lib/python3.7/copy.py in copy(x)
    104     if isinstance(rv, str):
    105         return x
--> 106     return _reconstruct(x, None, *rv)
    107 
    108 

/usr/lib/python3.7/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
    290             if slotstate is not None:
    291                 for key, value in slotstate.items():
--> 292                     setattr(y, key, value)
    293 
    294     if listiter is not None:

<string> in __setattr__(self, name, value)

FrozenInstanceError: cannot assign to field 'x'

Clearly something is still broken, though. I checked, and regular dataclasses are copyable without issue:

>>> from dataclasses import dataclass                                                        

>>> @dataclass(frozen=True) 
>>> class Foo: 
>>>     bar: str 
>>>     weight: float 

>>> copy.copy(Foo("blah", 0.5))                                                              
Foo(bar='blah', weight=0.5)

@nbraud nbraud mentioned this pull request Apr 14, 2019
nbraud added a commit to nbraud/ppb-vector that referenced this pull request Apr 15, 2019
ppb_vector/vector2.py Outdated Show resolved Hide resolved
@nbraud
Copy link
Collaborator Author

nbraud commented Apr 16, 2019

pings @astronouth7303

I ended up merging this branch into #144, because I ended up running into some of the issues fixed here, but I'd prefer to get this reviewed and merged first.

The specific issue with pickle and copy was fixed when you implemented __reduce__, but I think protocol violations are worth fixing (especially since this one manifested itself as hard-to-debug wonkyness).

@nbraud
Copy link
Collaborator Author

nbraud commented May 22, 2019

Rebased to deal with a merge conflict.

@AstraLuma
Copy link
Member

I'm just not buying that an argumentless __new__ is part of the contract.

@nbraud
Copy link
Collaborator Author

nbraud commented May 24, 2019

How come that pickle (and possibly other modules) make that assumption, then?

I guess I should split this into several PRs, and only keep the controversial __new__/__init__ change here. (Update: done, 664a287 is all that is left in this branch)

This fixes a violation of the [protocol] for object construction:

> If __new__ returns an instance of cls, then the new instance’s __init__()
> method will be invoked like __init__(self[, ...]), where self is the new
> instance and the remaining arguments are the same as were passed to __new__.

[protocol]: https://docs.python.org/3/reference/datamodel.html#object.__new__
@nbraud
Copy link
Collaborator Author

nbraud commented May 24, 2019

PS: Since I extracted the fix for __reduce__ to #157, this fails in hilarious ways.
I will pull #157 into this branch, but please merge it first.

$157 was extracted from ppb#146, but this branch still depends on it.
@AstraLuma
Copy link
Member

AstraLuma commented May 24, 2019

It appears that when a custom __reduce__/__reduce_ex__/etc (there's like 5 possible dunder methods?) is not defined, pickle defaults to saving the __dict__, calling argless __new__, and restoring the __dict__.

This is fine for many standard Python classes (eg, the engine, systems, scenes, and sprites), this assumption falls apart for a slots-based immutable class.

Keep in mind that pickle is standard library, not core interpreter. It's doing a lot of magic, but is making some assumptions while doing so.

bors bot added a commit that referenced this pull request May 24, 2019
155: tests/ctor: Test that vectors can be copied r=astronouth7303 a=nbraud

Split off from #146 

Co-authored-by: Nicolas Braud-Santoni <nicolas@braud-santoni.eu>
bors bot added a commit that referenced this pull request May 24, 2019
157: Vector.__reduce__: Do not invoke __new__ directly r=astronouth7303 a=nbraud

This fixes a violation of the [protocol] for object construction:

> If `__new__` returns an instance of `cls`, then the new instance’s `__init__()` method will be invoked like `__init__(self[, ...])`, where `self` is the new instance and the remaining arguments are the same as were passed to `__new__`.

[protocol]: https://docs.python.org/3/reference/datamodel.html#object.__new__

@astronouth7303 seems [in agreement](#146 (comment))

Co-authored-by: Nicolas Braud-Santoni <nicolas@braud-santoni.eu>
bors bot added a commit that referenced this pull request May 25, 2019
156: tests/utils: Freeze all the operators sets r=astronouth7303 a=nbraud

Split off from #146.

Co-authored-by: Nicolas Braud-Santoni <nicolas@braud-santoni.eu>
@AstraLuma
Copy link
Member

Going to close this.

@AstraLuma AstraLuma closed this May 29, 2019
@nbraud nbraud deleted the tests/copy branch June 8, 2019 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants