Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(benchmark): allow overriding lockfile #196

Merged
merged 6 commits into from
Nov 14, 2023

Conversation

KSXGitHub
Copy link
Contributor

Make it easier to benchmark different lockfiles without having to modify the fixture content.

Copy link

github-actions bot commented Nov 13, 2023

Micro-Benchmark Results

Linux

group                          main                                   pr
-----                          ----                                   --
tarball/download_dependency    1.00      8.1±0.11ms   533.7 KB/sec    1.02      8.3±0.63ms   525.5 KB/sec

Copy link

github-actions bot commented Nov 13, 2023

Integrated-Benchmark Report (Linux)

Scenario: Frozen Lockfile

Command Mean [ms] Min [ms] Max [ms] Relative
pacquet@HEAD 142.9 ± 10.1 126.6 164.4 1.04 ± 0.10
pacquet@main 137.8 ± 8.4 125.7 158.9 1.00
BENCHMARK_REPORT.json
{
  "results": [
    {
      "command": "pacquet@HEAD",
      "mean": 0.1428908758685714,
      "stddev": 0.010082594067539467,
      "median": 0.14413127744,
      "user": 0.06734682285714286,
      "system": 0.20059271714285712,
      "min": 0.12661095794000002,
      "max": 0.16437400394,
      "times": [
        0.15518144894000002,
        0.14637185494000002,
        0.13644853994,
        0.14869871194,
        0.14517859794000001,
        0.13212219294000002,
        0.12661095794000002,
        0.14667448394000002,
        0.16437400394,
        0.13724930294,
        0.14933162494000002,
        0.13673736394000002,
        0.14308395694,
        0.13240922094000002
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ]
    },
    {
      "command": "pacquet@main",
      "mean": 0.13781609267333336,
      "stddev": 0.008377426861409534,
      "median": 0.13791584494,
      "user": 0.05628367999999999,
      "system": 0.21385735999999997,
      "min": 0.12565650594000002,
      "max": 0.15892093394,
      "times": [
        0.13098355894000002,
        0.13724597494000002,
        0.12919311894000002,
        0.14439213894000003,
        0.13307198594000003,
        0.13869599894,
        0.12775115894000003,
        0.14416606494,
        0.13843282794,
        0.13503398494000002,
        0.13791584494,
        0.14313661794000002,
        0.12565650594000002,
        0.14264467394000002,
        0.15892093394
      ],
      "exit_codes": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ]
    }
  ]
}

Copy link

codecov bot commented Nov 13, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (6b00165) 87.11% compared to head (3d7392f) 87.11%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #196   +/-   ##
=======================================
  Coverage   87.11%   87.11%           
=======================================
  Files          56       56           
  Lines        2824     2825    +1     
=======================================
+ Hits         2460     2461    +1     
  Misses        364      364           
Files Coverage Δ
crates/testing-utils/src/fs.rs 100.00% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@zkochan
Copy link
Member

zkochan commented Nov 14, 2023

I don't understand what this change is about and why it is needed.

@KSXGitHub
Copy link
Contributor Author

I don't understand what this change is about and why it is needed.

Before this PR, if I want to benchmark a different lockfile, I would have to copy the lockfile and paste it to the lockfile fixture. After this PR, I'll simply specify the directory containing the lockfile.

@KSXGitHub
Copy link
Contributor Author

Also, please ignore the snapshot changes. They are from #197 as I need the test to pass.

@zkochan zkochan merged commit 078359e into main Nov 14, 2023
13 checks passed
@zkochan zkochan deleted the integrated-benchmark-fixture-dir branch November 14, 2023 23:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants