Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix different behavior with unittest when warlus operator #10758

Merged
merged 13 commits into from
Mar 10, 2023

Conversation

aless10
Copy link
Contributor

@aless10 aless10 commented Feb 22, 2023

The AssertionRewriter class did not handle the assertions with the ast.NamedExpr (aka 'walrus operator'), that changed the value of a variable used in the assertion (an example can be seen in the related issue or in the tests added).

This caused different behavior from unittest/python and pytest.

This PR aims to handle this case (although very remote)

Addresses python version >= 3.8

closes #10743

@aless10
Copy link
Contributor Author

aless10 commented Feb 23, 2023

It looks like one of the checks failed but I don't know if it depends on the changes or what. What is the next action in this situation? I'd like to help in some way. Thanks

@RonnyPfannschmidt
Copy link
Member

i restarted the job, its unrelated to your change

Copy link
Member

@nicoddemus nicoddemus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for tacking this @aless10!

Left two comments, could you take a look? Thanks!

)
result = pytester.runpytest()
assert result.ret == 1
# This is not optimal as error message but it depends on how the rewrite is structured
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed this as well, can't we save the first a value in a separate temporary variable, before evaluating the walrus operator? This would provide the same (correct) error message as in test_assertion_walrus_no_variable_name_conflict.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to do so, but with my approach I did not change the value of the variable a but I saved the new value of a in another temporary variable. The problem with this is that if that variable is used later in another assert there will be an error. I think is doable, I just need to try different approach. Thanks

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with this is that if that variable is used later in another assert there will be an error

IIUC the problem, you can use self.variable() to obtain a new, unique variable to save the value to. This would prevent any conflicts in later asserts.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the tip: I refactored using the variable method but I don't think this is enough to solve the problem. For example in this test case

def test_walrus_conversion_succeed():
    a = "Hello"
    assert a != (a := a.lower())
    assert a == 'hello'

if I change the name of the first "a", then I got an error because it is referencing something that has no reference (in fact "a" is defined outside the first assert statement. I tried to change the name of the new "a" in the walrus operator, but then the second assert fails because the value of "a" is still the original one.
The only solution that came to my mind is using a map/state that lives through the asserts that enables me to retrieves the values if they have changed.
That said, this situation is very rare and the tests cases that came to my mind are far from being used in real case scenario (but this is only my opinion). I'm thinking if the juice worth the squeeze

Copy link
Member

@nicoddemus nicoddemus Mar 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That said, this situation is very rare and the tests cases that came to my mind are far from being used in real case scenario (but this is only my opinion). I'm thinking if the juice worth the squeeze

Yeah if it proves too problematic, I'm happy with having just the original fix, and we can instead add a note to the rewrite docs somewhere about this corner case. I agree it should be very rare to happen in practice.

Please feel free to revert to the previous working version.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, after #10788, all the tests pass. Right now, we have covered all of the cases that are tested in test_assertrewrite.py::TestIssue10743 where I tried to imagine most of the corner cases. Let me know if you like my solution or if you prefer to revert and add a note for this remote case. Thanks

Comment on lines 946 to 947
# Display the repr of the target name if it's a local variable or
# _should_repr_global_name() thinks it's acceptable.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was copied from visit_Name, perhaps we can update the comment to mention that this handles the walrus operator instead?

@aless10 aless10 force-pushed the main branch 2 times, most recently from 53e0aad to 0bc4bdc Compare March 1, 2023 14:04
@aless10
Copy link
Contributor Author

aless10 commented Mar 2, 2023

I don't know why all the checks are failing. Last time they passed was at commit 3e90bf5 and it looks like the failing tests are not referenced by these changes.
I tried to reproduce the error and actually the tox commands fails but if I run the tests inside a docker container, the tests pass. I'm trying to understand what's going on. Sorry for the inconvenience

@aless10
Copy link
Contributor Author

aless10 commented Mar 4, 2023

I don't know why all the checks are failing. Last time they passed was at commit 3e90bf5 and it looks like the failing tests are not referenced by these changes. I tried to reproduce the error and actually the tox commands fails but if I run the tests inside a docker container, the tests pass. I'm trying to understand what's going on. Sorry for the inconvenience

OK it looks like I had to align with this #10788 fix. Sorry

@RonnyPfannschmidt
Copy link
Member

this looks good and self-contained, but i don't have the bandwidth to get into it soon

@@ -0,0 +1 @@
Fixed different behavior from std lib unittest of asserts with expression that contains the walrus operator in it that changes the value of a variable.
Copy link
Member

@nicoddemus nicoddemus Mar 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can update this to mean that walrus operator now is properly supported by the assertion rewriting, correct?

Suggested change
Fixed different behavior from std lib unittest of asserts with expression that contains the walrus operator in it that changes the value of a variable.
The assertion rewriting mechanism now works correctly when assertion expressions that contain the walrus operator.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I pushed the changes. Thanks

@nicoddemus nicoddemus added the needs backport applied to PRs, indicates that it should be ported to the current bug-fix branch label Mar 8, 2023
@nicoddemus
Copy link
Member

Thanks! I will merge in a few days unless other maintainers want to take a closer look.

@nicoddemus nicoddemus merged commit 6e478b0 into pytest-dev:main Mar 10, 2023
@nicoddemus nicoddemus added backport 7.2.x and removed needs backport applied to PRs, indicates that it should be ported to the current bug-fix branch labels Mar 10, 2023
nicoddemus pushed a commit that referenced this pull request May 30, 2023
In #10758 we introduced the support for the use of the walrus operator in the test cases. There was a case which was not handled that caused a bug report #11028. This PR aims to fix the issue and also to improve how the walrus operator is handled in the AssertionRewriter class.

Closes #11028
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Walrus operator causes different behavior in PyTest.
3 participants