Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up parameterized tests which use Git #761

Open
wants to merge 25 commits into
base: master
Choose a base branch
from
Open

Conversation

akaihola
Copy link
Owner

@akaihola akaihola commented Oct 20, 2024

This is done by using the new module-scoped Git repository fixture from Darkgraylib.

To get aggregate timings for parameterized tests:

pip install pytest-reportlog
pytest --report-log=pytest.log.json
python analyze_pytest_log.py
analyze_pytest_log.py
import json
from collections import defaultdict


def aggregate_test_durations(log_contents):
    """Aggregate durations of parameterized tests from pytest-reportlog output.

    Args:
        log_contents (str): The contents of the pytest-reportlog file

    Returns:
        dict: A dictionary mapping test names to their total duration

    """
    # Dictionary to store total durations for each base test name
    test_durations = defaultdict(float)

    # Process each line
    for line in log_contents.strip().split("\n"):
        try:
            report = json.loads(line)

            # Extract the base test name by removing the parameter portion
            full_nodeid = report["nodeid"]
            base_name = full_nodeid.split("[")[0]

            # Add the duration to the total for this test
            test_durations[base_name] += report["duration"]

        except (json.JSONDecodeError, KeyError, IndexError) as e:
            print(f"Error processing line: {e} in {line}")
            continue

    return dict(test_durations)

def main():
    # Sample usage
    log_contents = open("pytest.log.json").read()

    results = aggregate_test_durations(log_contents)

    # Print results in a formatted way
    print("\nAggregated test durations:")
    print("-" * 50)
    by_duration = sorted(results.items(), key=lambda x: x[1], reverse=True)
    for test_name, duration in by_duration:
        print(f"{test_name:<40} {duration:.6f} seconds")

if __name__ == "__main__":
    main()

@akaihola akaihola added enhancement New feature or request CI labels Oct 20, 2024
@akaihola akaihola added this to the Darker 3.1.0 milestone Oct 20, 2024
@akaihola akaihola self-assigned this Oct 20, 2024
@akaihola akaihola force-pushed the git-test-cache branch 2 times, most recently from 85d9279 to f49c0dc Compare November 4, 2024 20:15
This is done by using the new module-scoped Git repository fixture from
Darkgraylib.
@akaihola akaihola force-pushed the git-test-cache branch 2 times, most recently from 50388df to 2211f88 Compare November 9, 2024 21:00
@akaihola akaihola force-pushed the git-test-cache branch 3 times, most recently from e41fa1d to 094a466 Compare November 16, 2024 18:47
@akaihola akaihola force-pushed the git-test-cache branch 2 times, most recently from 8e2d766 to e6e4ade Compare November 16, 2024 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI enhancement New feature or request
Projects
Development

Successfully merging this pull request may close these issues.

1 participant