Skip to content

Commit

Permalink
test/verify: sort tests by nondestructiveness
Browse files Browse the repository at this point in the history
When running test/verify tests via pytest, sort them by their
nondestructiveness.  This has two advantages:

 - the tests that can use the global machine are placed beside each
   other, preventing the machine from being killed in between

 - we sort nondestructive tests to the start under the assumption that
   they're usually smaller.  This allows us to run a lot of tests at the
   start to quickly catch issues.
  • Loading branch information
allisonkarlitskaya authored and martinpitt committed Jun 17, 2023
1 parent 3400f09 commit 4ebb9b3
Showing 1 changed file with 16 additions and 1 deletion.
17 changes: 16 additions & 1 deletion test/verify/conftest.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
import importlib.machinery
import importlib.util
from pathlib import Path
from typing import Optional
from typing import List, Optional

import pytest
import testlib


@pytest.hookimpl
Expand All @@ -24,3 +25,17 @@ def pytest_collect_file(file_path: Path, parent: pytest.Collector) -> Optional[p
return collector

return None


@pytest.hookimpl
def pytest_collection_modifyitems(session: pytest.Session, items: List[pytest.Item]) -> None:
"""Sorts the tests to place all non-destructive tests together"""
assert isinstance(items, list)

def is_nondestructive(item: pytest.Item) -> bool:
assert isinstance(item, pytest.Function)
assert isinstance(item.parent, pytest.Class | pytest.Module)
return testlib.get_decorator(item.obj, item.parent.obj, "nondestructive", False)

# put the destructive tests last under the assumption that they're slower
items.sort(key=is_nondestructive, reverse=True)

0 comments on commit 4ebb9b3

Please sign in to comment.