Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #3812 and fix const of inplace assignments #4065

Merged
merged 16 commits into from
Jul 20, 2022

Conversation

Skylion007
Copy link
Collaborator

@Skylion007 Skylion007 commented Jul 14, 2022

Description

So I looked through the inplace assignments and found that many of them did not work for immutable types (int_, str, etc...). I actually couldn't find any situation where they appeared to work properly at all. Additionally, it didn't seem possible to actually properly +=, -= etc in the object_api without some virtual functionsince the = operator is overridden in the subclasses. The methods I suppose could remain const for handles since they don't modify the handle, but we need to make those const methods can't be accidentally used on the object class (perhaps through explicit deletion?).

Fix #3812.

Suggested changelog entry:

* Make augmented assignment operators non-const for the object-api
* Behavior was previously broken for augmented assignment operators.

@Skylion007 Skylion007 requested review from rwgk and henryiii July 14, 2022 17:51
@henryiii
Copy link
Collaborator

Ahh, this is the operator on pyobject, not py::self, which is what I immediately thought of (because I use it at https://github.com/scikit-hep/boost-histogram/blob/d438def268b8bfb9a51a5fed42ad88683de0e6f9/include/bh_python/register_histogram.hpp#L61 ). Seems like a reasonable fix, we could add const handle versions later if someone cares (guessing these weren't heavily used before?)

@Skylion007
Copy link
Collaborator Author

@henryiii Not sure this would actually work because we need to do the same thing on accessors as well. I was thinking about whether it would be better to do a general mixin class like we do the object_api. Thoughts?

Copy link
Collaborator

@rwgk rwgk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very minor comments, the meat of this looks good to me, but can you please give me a chance to run this through global testing? (Results late tonight or tomorrow morning.)
At the moment I'm thinking the const versions are an accident that just happened to survive amazingly long, and that we don't need to bring them back.

@@ -353,6 +353,20 @@ class object : public handle {
return *this;
}

#define PYBIND11_INPLACE_OP(inp) \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very minor technically, but I'd definitely make this iop (a lot more intuitive).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually think this is insufficient for attributes. Basically, we need to define the augmented operators for every object_api have custom operator=.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you count already how many we have?
I'm not sure about the big picture TBH (I may still be misunderstanding), but this PR seemed like a necessary good step in the right direction, just by itself. Does that make sense?

@@ -2364,26 +2378,35 @@ bool object_api<D>::rich_compare(object_api const &other, int value) const {
return result; \
}

#define PYBIND11_MATH_OPERATOR_BINARY_INPLACE(op, fn) \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd change this to iop, too (to be consistent & intuitive).

tests/test_pytypes.cpp Show resolved Hide resolved
@rwgk
Copy link
Collaborator

rwgk commented Jul 15, 2022

I still don't have global testing results unfortunately, due to some internal outage.

@rwgk
Copy link
Collaborator

rwgk commented Jul 15, 2022

Global testing passes for this PR (internal test ID OCL:461063257:BASE:461249872:1657923213067:e8a3caed).

Therefore I'm still thinking: no need to worry about const versions, at least not for the current scope of changes.

@Skylion007 Are there reasons for holding back this PR? (see my previous question)

@Skylion007
Copy link
Collaborator Author

@rwgk Main reason is I am trying to figure if there is a better way to solve this. Using the augmented assigned operators on handles is still broken with this PR and likely broken for accessors as well.

@Skylion007
Copy link
Collaborator Author

Okay, so I tested it and the augmented assignment operators for accessors are currently broken anyway due to another bug which results in a compile error (unless you are doing augmented assignments from another accessor). As such, it's not as big of a deal as I thought. I would like to actually remove the augmented assignment operators from handles if possible, but that requires further re-architecting.

@Skylion007
Copy link
Collaborator Author

Skylion007 commented Jul 17, 2022

@henryiii @rwgk Is this build error a flake or a compiler bug? Seems rather specific. Might have to revert the self-assignment optimizations if its the later.

@rwgk
Copy link
Collaborator

rwgk commented Jul 17, 2022

Looks like flakes I've seen before. I clicked the button to rerun only the failed job. Let's see what happens.

Copy link
Collaborator

@rwgk rwgk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was a flake, the rerun worked.

I would like to actually remove the augmented assignment operators from handles if possible, but that requires further re-architecting.

Sounds good (I assume you mean "in a separate PR").

include/pybind11/pytypes.h Show resolved Hide resolved
Copy link
Collaborator

@rwgk rwgk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, thanks!
I'd love to have this in the next smart_holder update (which will then be imported into the Google production environment). Do you think this can be merged soon?

@Skylion007 Skylion007 merged commit f47f1ed into pybind:master Jul 20, 2022
@Skylion007 Skylion007 deleted the skylion007/fix-inplace-operators branch July 20, 2022 15:42
@github-actions github-actions bot added the needs changelog Possibly needs a changelog entry label Jul 20, 2022
@henryiii henryiii removed the needs changelog Possibly needs a changelog entry label Oct 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG]: py::object augmented assignment doesn't work with immutable types
3 participants