Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Incorrect introspection for MagicMock.assert_has_calls #234

Closed
progval opened this issue Mar 25, 2021 · 5 comments · Fixed by #365
Closed

Incorrect introspection for MagicMock.assert_has_calls #234

progval opened this issue Mar 25, 2021 · 5 comments · Fixed by #365

Comments

@progval
Copy link

progval commented Mar 25, 2021

Hi,

I wrote a test that calls a MagicMock() with a list of bytes, then uses assert_has_calls with a different argument (list of strings).

Here is pytest's output:

    def test_foo():
        from unittest.mock import MagicMock, call
    
        f = MagicMock()
    
        f("foo", [b"bar"])
        f("foo", [b"baz"])
    
        f.assert_has_calls(
            [
                call("foo", ["bar"]),
                call("foo", ["baz"]),
            ],
>           any_order=True
        )
E       AssertionError: (call('foo', ['bar']), call('foo', ['baz'])) not all found in call list
E       
E       pytest introspection follows:
E       
E       Args:
E       assert ('foo', [b'baz']) == ([call('foo', ['bar']), call('foo', ['baz'])],)
E         At index 0 diff: 'foo' != [call('foo', ['bar']), call('foo', ['baz'])]
E         Left contains one more item: [b'baz']
E         Full diff:
E         - ([call('foo', ['bar']), call('foo', ['baz'])],)
E         + ('foo', [b'baz'])
E       Kwargs:
E       assert {} == {'any_order': True}
E         Right contains 1 more item:
E         {'any_order': True}
E         Full diff:
E         - {'any_order': True}
E         + {}

The first line of the error(AssertionError) is correct, but the other ones aren't:

  • ('foo', [b'baz']) == ([call('foo', ['bar']), call('foo', ['baz'])],) is not exactly what was asserted; instead the assertion was that the left side is inside the right side's first element
  • assert {} == {'any_order': True} this is not part of the assertion, any_order is just an argument of the function

Instead, I would probably expect something like this:

E       AssertionError: (call('foo', ['bar']), call('foo', ['baz'])) not all found in call list
E       
E       pytest introspection follows:
E       
E       assert call('foo', [b'baz']) in [call('foo', ['bar']), call('foo', ['baz'])]

OS: Debian 10

Python version: 3.7.3

pytest version: 6.2.2

pip list 
Package                       Version                         Location
----------------------------- ------------------------------- ---------------------------------------------------------------------------
aiohttp                       3.5.1
aiohttp-utils                 3.1.1
aiosqlite                     0.16.0
alabaster                     0.7.12
amqp                          2.5.2
ansible                       2.7.7
anyjson                       0.3.3
apache-libcloud               2.4.0
apipkg                        1.5
appdirs                       1.4.3
argcomplete                   1.8.1
arrow                         0.12.1
asgiref                       3.2.10
asn1crypto                    0.24.0
aspy.yaml                     1.3.0
async-generator               1.10
async-timeout                 3.0.1
atomicwrites                  1.1.5
attrs                         19.3.0
attrs-strict                  0.1.0
azure-common                  1.1.24
azure-nspkg                   3.0.2
azure-storage                 0.36.0
Babel                         2.7.0
bcrypt                        3.1.6
beautifulsoup4                4.8.1
billiard                      3.6.3.0
binwalk                       2.1.2
black                         19.10b0
bleach                        3.1.2
blessed                       1.15.0
blinker                       1.4
borgbackup                    1.1.9
Brlapi                        0.6.7
Brotli                        1.0.7
cached-property               1.5.2
cachetools                    4.2.0
cassandra-driver              3.20.1
celery                        4.4.0
certifi                       2018.8.24
cffi                          1.14.3
cfgv                          2.0.1
chardet                       3.0.4
click                         7.1.2
colorama                      0.3.7
commonmark                    0.9.1
confluent-kafka               1.5.0
coreapi                       2.3.3
coreschema                    0.0.4
coverage                      5.0
cryptography                  2.6.1
cupshelpers                   1.0
cycler                        0.10.0
daiquiri                      2.1.1
dash                          1.12.0
dash-bootstrap-components     0.10.2
dash-core-components          1.10.0
dash-html-components          1.0.3
dash-renderer                 1.4.1
dash-table                    4.7.0
decorator                     4.3.0
defusedxml                    0.5.0
Deprecated                    1.2.6
devscripts                    2.19.5+deb10u1
diffoscope                    113
distro                        1.5.0
distro-info                   0.21
Django                        2.2.15
django-cors-headers           3.2.0
django-extensions             2.2.5
django-js-reverse             0.9.1
django-stubs                  1.7.0
django-test-migrations        1.1.0
django-webpack-loader         0.6.0
djangorestframework           3.10.3
djangorestframework-stubs     1.4.0
docker                        4.3.1
docker-compose                1.27.4
dockerpty                     0.4.1
docopt                        0.6.2
docutils                      0.15.2
dulwich                       0.19.13
ecdsa                         0.15
EditorConfig                  0.12.1
elasticsearch                 7.0.5
elementpath                   2.0.4
enjarify                      1.0.3
entrypoints                   0.3
ephem                         3.7.6.0
execnet                       1.7.1
extras                        1.0.0
fastimport                    0.9.8
feedparser                    5.2.1
filelock                      3.0.10
fixtures                      3.0.0
flake8                        3.8.3
Flask                         1.1.1
Flask-Compress                1.5.0
fossology                     0.2.0
frozendict                    1.2
future                        0.18.2
gbp                           0.9.14
geomet                        0.1.2
glob2                         0.7
gpg                           1.12.0
gunicorn                      19.9.0
html5lib                      1.0.1
htmlmin                       0.1.12
httplib2                      0.11.3
humanize                      3.2.0
hypothesis                    5.49.0
identify                      1.4.7
idna                          2.6
imagesize                     1.1.0
importlib-metadata            0.23
iniconfig                     1.0.1
iotop                         0.6
ipykernel                     4.9.0
ipython                       5.8.0
ipython-genutils              0.2.0
ipywidgets                    6.0.0
iso8601                       0.1.12
isodate                       0.6.0
isort                         5.5.3
itsdangerous                  0.24
itypes                        1.2.0
Jinja2                        2.10.3
jmespath                      0.9.4
jsbeautifier                  1.6.4
jsondiff                      1.1.1
jsonschema                    2.6.0
junit-xml                     1.8
jupyter-client                5.2.3
jupyter-core                  4.4.0
kafka-python                  2.0.0
keyring                       17.1.1
keyrings.alt                  3.1.1
kiwisolver                    1.0.1
kombu                         4.6.7
launchpadlib                  1.10.13
lazr.restfulclient            0.14.3
lazr.uri                      1.0.3
libarchive-c                  2.8
linecache2                    1.0.0
llfuse                        1.3.6
lockfile                      0.12.2
louis                         3.8.0
lxml                          4.4.1
markdown-it-py                0.5.6
MarkupSafe                    1.1.0
matplotlib                    3.0.2
mccabe                        0.6.1
mercurial                     5.7
meson                         0.49.2
mirakuru                      2.3.0
mistune                       0.8.4
more-itertools                4.2.0
msgpack                       1.0.0
multidict                     4.5.2
musicbrainzngs                0.6
mutagen                       1.40.0
mutmut                        2.0.0
mypy                          0.800
mypy-extensions               0.4.3
myst-parser                   0.12.10
nagiosplugin                  1.3.2
nbconvert                     5.4.0
nbformat                      4.4.0
ndjson                        0.3.1
netaddr                       0.7.19
nodeenv                       1.3.3
nose                          1.3.7
notebook                      5.7.8
ntlm-auth                     1.1.0
numexpr                       2.6.9
numpy                         1.16.2
oauthlib                      3.1.0
olefile                       0.46
outcome                       1.1.0
packaging                     19.2
pandas                        0.23.3+dfsg
pandocfilters                 1.4.2
paramiko                      2.4.2
parso                         0.7.0
pathspec                      0.7.0
patool                        1.12
pbr                           5.4.5
pexpect                       4.6.0
pg-activity                   2.1.0
pg8000                        1.13.2
pickleshare                   0.7.5
pifpaf                        3.0.0
pika                          1.1.0
Pillow                        5.4.1
pip                           21.0.1
pkginfo                       1.5.0.1
plotille                      3.7.2
plotly                        4.5.4
pluggy                        0.13.1
pony                          0.7.13
port-for                      0.4
powerline-status              2.7
pre-commit                    1.20.0
pristine-zip                  0.1.dev73+gefd1d06.d20200805    /home/dev/pristine_zip
progressbar                   2.5
prometheus-client             0.6.0
prompt-toolkit                1.0.15
psutil                        5.5.1
psycopg2                      2.8.6
py                            1.9.0
py4j                          0.10.8.1
pyasn1                        0.4.2
pybadges                      2.2.1
pybloomfiltermmap3            0.5.3
pycairo                       1.16.2
pycodestyle                   2.6.0
pycparser                     2.20
pycrypto                      2.6.1
pycups                        1.9.73
pycurl                        7.43.0.2
pyflakes                      2.2.0
pyfuse3                       3.1.1
Pygments                      2.3.1
PyGObject                     3.30.4
pyinotify                     0.9.6
pykerberos                    1.1.14
PyLD                          2.0.3
PyNaCl                        1.3.0
PyOpenGL                      3.1.0
pyOpenSSL                     19.0.0
pypandoc                      1.4
pyparsing                     2.2.0
PyPDF2                        1.26.0
pyqtgraph                     0.10.0
PySimpleSOAP                  1.16.2
pysmbc                        1.0.15.6
pytest                        6.2.2
pytest-cov                    2.10.0
pytest-cover                  3.0.0
pytest-coverage               0.0
pytest-django                 3.7.0
pytest-forked                 1.1.3
pytest-kafka                  0.4.0
pytest-mock                   1.13.0
pytest-parallel               0.0.10
pytest-postgresql             2.5.3
pytest-xdist                  1.31.0
python-apt                    1.8.4.3
python-cephlibs               0.94.5.post1
python-daemon                 2.2.4
python-dateutil               2.8.1
python-debian                 0.1.35
python-debianbts              2.8.2
python-dotenv                 0.15.0
python-hglib                  2.6.1
python-jose                   3.1.0
python-json-logger            0.1.11
python-keycloak               0.20.0
python-magic                  0.4.16
python-memcached              1.58
python-mimeparse              1.6.0
pytz                          2019.1
pywinrm                       0.3.0
pyxattr                       0.6.1
pyxdg                         0.25
PyYAML                        3.13
pyzmq                         17.1.2
quodlibet                     4.2.1
rados                         2.0.0
rdflib                        5.0.0
rdflib-jsonld                 0.5.0
recommonmark                  0.6.0
redis                         3.3.11
regex                         2020.2.20
reportbug                     7.5.3-deb10u1
reportlab                     3.5.13
requests                      2.25.1
requests-ftp                  0.3.1
requests-kerberos             0.11.0
requests-mock                 1.8.0
requests-ntlm                 1.1.0
retrying                      1.3.3
rsa                           4.0
scipy                         1.1.0
scour                         0.37
scramp                        1.1.0
SecretStorage                 2.3.1
Send2Trash                    1.5.0
sentry-sdk                    0.13.5
setuptools                    40.8.0
simplegeneric                 0.8.1
simplejson                    3.16.0
simpy                         3.0.13
six                           1.12.0
sniffio                       1.2.0
snowballstemmer               2.0.0
sortedcontainers              2.1.0
soupsieve                     1.9.5
Sphinx                        3.3.1
sphinx-click                  2.5.0
sphinx-rtd-theme              0.5.0
sphinx-tabs                   1.3.0
sphinxcontrib-applehelp       1.0.1
sphinxcontrib-devhelp         1.0.1
sphinxcontrib-htmlhelp        1.0.2
sphinxcontrib-httpdomain      1.7.0
sphinxcontrib-images          0.9.2
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-programoutput   0.16
sphinxcontrib-qthelp          1.0.2
sphinxcontrib-serializinghtml 1.1.3
SQLAlchemy                    1.3.10
sqlalchemy-stubs              0.2
sqlitedict                    1.6.0
sqlparse                      0.3.1
subvertpy                     0.10.1
swh.clearlydefined            0.1.dev5+g430c67f               /home/dev/swh-environment/swh-clearlydefined
swh.core                      0.12.0                          /home/dev/.local/lib/python3.7/site-packages
swh.deposit                   0.6.1.dev24+g22172d4e           /home/dev/swh-environment/swh-deposit
swh.docs                      0.0.1.dev287+g54fe755.d20210325 /home/dev/swh-environment/swh-docs
swh.fuse                      0.9.2                           /home/dev/.local/lib/python3.7/site-packages
swh.graph                     0.2.10.dev45+g5a987aa           /home/dev/swh-environment/swh-graph
swh.icinga-plugins            0.2.2.dev1+g441faf2             /home/dev/swh-environment/swh-icinga-plugins
swh.indexer                   0.6.4.dev4+g3baf8bb             /home/dev/swh-environment/swh-indexer
swh.journal                   0.7.2.dev3+g458a405.d20210322   /home/dev/swh-environment/swh-journal
swh.lister                    0.4.1.dev9+gc782275             /home/dev/swh-environment/swh-lister
swh.loader.core               0.20.1.dev4+gcf8fc9e            /home/dev/swh-environment/swh-loader-core
swh.loader.git                0.3.6                           /home/dev/swh-environment/swh-loader-git
swh.loader.mercurial          0.4.0                           /home/dev/swh-environment/swh-loader-mercurial
swh.loader.svn                0.0.54.post2                    /home/dev/swh-environment/swh-loader-svn
swh.mirror.forge              0.0.5.post20                    /home/dev/swh-environment/swh-mirror-forge
swh.model                     2.3.0                           /home/dev/.local/lib/python3.7/site-packages/swh.model-2.3.0-py3.7.egg
swh.objstorage                0.2.2                           /home/dev/.local/lib/python3.7/site-packages/swh.objstorage-0.2.2-py3.7.egg
swh.objstorage.replayer       0.2.1                           /home/dev/swh-environment/swh-objstorage-replayer
swh.scanner                   0.3.1.dev0+g33a9cd4.d20210301   /home/dev/swh-environment/swh-scanner
swh.scheduler                 0.10.0                          /home/dev/.local/lib/python3.7/site-packages
swh.search                    0.3.3.dev1+g4390bea.d20201211   /home/dev/swh-environment/swh-search
swh.storage                   0.26.0                          /home/dev/swh-environment/swh-storage
swh.vault                     0.0.35.dev3+gd64687d.d20200904  /home/dev/swh-environment/swh-vault
swh.web                       0.0.284.dev6+g0b974c89          /home/dev/swh-environment/swh-web
swh.web.client                0.2.2                           /home/dev/.local/lib/python3.7/site-packages
systemd-python                234
tables                        3.4.4
tblib                         1.6.0
tenacity                      6.0.0
terminado                     0.8.1
testinfra                     4.0.0
testing.common.database       2.0.3
testing.postgresql            1.3.0
testpath                      0.4.2
testresources                 2.0.1
testtools                     2.4.0
texttable                     1.6.3
tlsh                          0.2.0
toml                          0.10.2
tornado                       5.1.1
tox                           3.7.0
traceback2                    1.4.0
traitlets                     4.3.2
trio                          0.17.0
typed-ast                     1.4.0
typing-extensions             3.7.4.1
unattended-upgrades           0.1
unidiff                       0.5.4
unittest2                     1.1.0
uritemplate                   3.0.1
urllib3                       1.24.1
vcversioner                   2.16.0.0
vine                          1.3.0
virtualenv                    16.7.7
wadllib                       1.3.4
wcwidth                       0.1.7
webencodings                  0.5.1
websocket-client              0.57.0
Werkzeug                      0.16.0
wheel                         0.32.3
widgetsnbextension            2.0.0
wrapt                         1.11.2
xattr                         0.9.7
xmlschema                     1.3.1
xmltodict                     0.11.0
yarl                          1.3.0
zipp                          0.6.0

Thanks!
@asottile
Copy link
Member

'pytest introspection follows" doesn't appear in our codebase, this must be something from a faulty plugin -- appears to be pytest-mock

@RonnyPfannschmidt RonnyPfannschmidt transferred this issue from pytest-dev/pytest Mar 30, 2021
@nicoddemus
Copy link
Member

Hi @progval,

Indeed this is a bug in the introspection done by pytest-mock, thanks for reporting.

The problem seems to be how the internal assert_wrapper works, it can be followed from here:

pytest-mock/src/pytest_mock/plugin.py

Lines 442 to 444 in be33132

def wrap_assert_has_calls(*args: Any, **kwargs: Any) -> None:
__tracebackhide__ = True
assert_wrapper(_mock_module_originals["assert_has_calls"], *args, **kwargs)

I probably won't get to this any time soon, but others feel free to give it a shot. 👍

cc @asfaltboy

@TommyDew42
Copy link

Ran into the same error today too! The pytest-mock version is 3.6.1

@idow09
Copy link

idow09 commented Feb 27, 2023

Me too! with pytest-mock version 3.10.0

@Astral1020 Astral1020 mentioned this issue Apr 24, 2023
Closed
@Astral1020
Copy link
Contributor

I created this PR #359

@nicoddemus nicoddemus mentioned this issue Jun 15, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add assert_has_calls_wrapper & unit tests acovaci/pytest-mock
7 participants
@RonnyPfannschmidt @progval @nicoddemus @asottile @idow09 @Astral1020 @TommyDew42