Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed mtultiple chained errors thrown when using pytest Tavern #367

Merged
merged 1 commit into from
Jun 10, 2024

Conversation

virdok
Copy link
Contributor

@virdok virdok commented Jun 10, 2024

Description

A framework exists above Pytest which is called Tavern. It is using YAML files to execute API tests. During the execution of the tests in this framework and due to the difference it has in the structure of the test suites in respect to BDD for example, some changes were needed as some exceptions were thrown and the integration of the pytest execution with Reportportal was not possible / blocked.

Points of failure

Original name attribute

The first point of failure is due to the fact that the Tavern framework do not use the originalname in the test item. This change fixes the issue with the originalname checking if its existing in the item as an attribute.

If the originalname attribute does not exist then instead of using the item.name defined in this if / else statement, it was throwing the following error:

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/_pytest/main.py", line 270, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/_pytest/main.py", line 324, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_callers.py", line 182, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/_pytest/main.py", line 349, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_callers.py", line 182, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_callers.py", line 88, in _multicall
INTERNALERROR>     next(wrapper_gen)  # first yield
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pytest_reportportal/plugin.py", line 263, in pytest_runtest_protocol
INTERNALERROR>     service.start_pytest_item(item)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pytest_reportportal/service.py", line 129, in wrap
INTERNALERROR>     func(*args, **kwargs)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pytest_reportportal/service.py", line 710, in start_pytest_item
INTERNALERROR>     self._process_metadata_item_start(current_leaf)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pytest_reportportal/service.py", line 650, in _process_metadata_item_start
INTERNALERROR>     leaf['code_ref'] = self._get_code_ref(item)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pytest_reportportal/service.py", line 460, in _get_code_ref
INTERNALERROR>     item.originalname is not None \
INTERNALERROR> AttributeError: 'YamlItem' object has no attribute 'originalname'

Parent attribute

In the line 467 a check for the parent attribute has been added. The reason of this is the item structure of the Tavern framework in comparison with the rest of the Pytest Item structure. After getting the first time the parent object in line 461, then the Tavern framework does not have at all parents set. (Although it depends somehow to the test implementation, but always after the root object, parent is NoneType).

Falling in this case the framework throws the following exception:

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/_pytest/main.py", line 270, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/_pytest/main.py", line 324, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_callers.py", line 182, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/_pytest/main.py", line 349, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_callers.py", line 182, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_callers.py", line 88, in _multicall
INTERNALERROR>     next(wrapper_gen)  # first yield
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pytest_reportportal/plugin.py", line 263, in pytest_runtest_protocol
INTERNALERROR>     service.start_pytest_item(item)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pytest_reportportal/service.py", line 129, in wrap
INTERNALERROR>     func(*args, **kwargs)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pytest_reportportal/service.py", line 709, in start_pytest_item
INTERNALERROR>     self._process_metadata_item_start(current_leaf)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pytest_reportportal/service.py", line 649, in _process_metadata_item_start
INTERNALERROR>     leaf['code_ref'] = self._get_code_ref(item)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pytest_reportportal/service.py", line 466, in _get_code_ref
INTERNALERROR>     classes.append(parent.name)
INTERNALERROR> AttributeError: 'NoneType' object has no attribute 'name'

So in the line 467 we check if parent attribute exists so as to step into it otherwise breaks the loop. If the loop is not broken at this point it ends up having an infinite loop.

Parent name

In the line 464 we check if the attribute name exists in the parent. The reason of doing so is the fact that at a point the parent object is not related to the test (It may be Session object for example) and this was throwing a name does not exist error as follows:

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/_pytest/main.py", line 270, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/_pytest/main.py", line 324, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_callers.py", line 182, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/_pytest/main.py", line 349, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_callers.py", line 182, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pluggy/_callers.py", line 88, in _multicall
INTERNALERROR>     next(wrapper_gen)  # first yield
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pytest_reportportal/plugin.py", line 263, in pytest_runtest_protocol
INTERNALERROR>     service.start_pytest_item(item)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pytest_reportportal/service.py", line 129, in wrap
INTERNALERROR>     func(*args, **kwargs)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pytest_reportportal/service.py", line 708, in start_pytest_item
INTERNALERROR>     self._process_metadata_item_start(current_leaf)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pytest_reportportal/service.py", line 648, in _process_metadata_item_start
INTERNALERROR>     leaf['code_ref'] = self._get_code_ref(item)
INTERNALERROR>   File "/Users/dkirchan/Library/Caches/pypoetry/virtualenvs/qa-earnings-app-qps9_NMR-py3.9/lib/python3.9/site-packages/pytest_reportportal/service.py", line 467, in _get_code_ref
INTERNALERROR>     parent = parent.parent
INTERNALERROR> AttributeError: 'NoneType' object has no attribute 'parent'

@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 70.66%. Comparing base (fbaa07e) to head (be082f8).
Report is 5 commits behind head on develop.

Files Patch % Lines
pytest_reportportal/service.py 80.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #367      +/-   ##
===========================================
+ Coverage    70.24%   70.66%   +0.42%     
===========================================
  Files            6        6              
  Lines          904      900       -4     
===========================================
+ Hits           635      636       +1     
+ Misses         269      264       -5     
Flag Coverage Δ
unittests 70.66% <80.00%> (+0.42%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@HardNorth
Copy link
Member

@virdok Thanks!

@HardNorth HardNorth merged commit 1c33b33 into reportportal:develop Jun 10, 2024
5 checks passed
@virdok
Copy link
Contributor Author

virdok commented Jun 11, 2024

Is there any expected date to release this fix in master in a new version? @HardNorth

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants