Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: additional test cases #249

Merged
merged 1 commit into from
Jul 6, 2023

Conversation

baszalmstra
Copy link
Collaborator

Adds additional environments to the tests and benches.

@baszalmstra baszalmstra self-assigned this Jul 6, 2023
@wolfv
Copy link
Contributor

wolfv commented Jul 6, 2023

More numbers (with reduced sample size):

test result: ok. 0 passed; 0 failed; 11 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running benches/bench.rs (target/release/deps/bench-aa13894860bf5bb8)
Gnuplot not found, using plotters backend
solve python=3.9/libsolv-sys
                        time:   [6.9505 ms 6.9767 ms 6.9993 ms]
                        change: [-4.2947% -1.5263% +0.1617%] (p = 0.29 > 0.05)
                        No change in performance detected.
Benchmarking solve python=3.9/libsolv_rs: AnalyzingCriterion.rs ERROR: error: Failed to access file "/Users/wolfv/Programs/rattler/target/criterion/solve python=3.9/libsolv_rs/change/estimates.json": Permission denied (os error 13)
solve python=3.9/libsolv_rs
                        time:   [3.9177 ms 3.9324 ms 3.9473 ms]
                        change: [-5.2255% -2.5835% -0.5930%] (p = 0.04 < 0.05)
                        Change within noise threshold.

solve xtensor, xsimd/libsolv-sys
                        time:   [5.5154 ms 5.5297 ms 5.5407 ms]
                        change: [-6.7763% -2.6881% -0.0525%] (p = 0.20 > 0.05)
                        No change in performance detected.
solve xtensor, xsimd/libsolv_rs
                        time:   [2.1800 ms 2.1919 ms 2.2091 ms]
                        change: [-2.3806% -1.2543% -0.1335%] (p = 0.05 > 0.05)
                        No change in performance detected.

Benchmarking solve tensorflow/libsolv-sys: Warming up for 1.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 6.3s.
solve tensorflow/libsolv-sys
                        time:   [627.38 ms 633.77 ms 641.76 ms]
                        change: [-1.5146% -0.4558% +1.0479%] (p = 0.51 > 0.05)
                        No change in performance detected.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
Benchmarking solve tensorflow/libsolv_rs: Warming up for 1.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 18.2s.
solve tensorflow/libsolv_rs
                        time:   [1.8568 s 1.8837 s 1.9105 s]
                        change: [-0.3374% +1.3184% +2.9652%] (p = 0.10 > 0.05)
                        No change in performance detected.

Benchmarking solve quetz/libsolv-sys: Warming up for 1.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 12.7s.
solve quetz/libsolv-sys time:   [1.2514 s 1.2589 s 1.2659 s]
Benchmarking solve quetz/libsolv_rs: Warming up for 1.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 30.3s.
solve quetz/libsolv_rs  time:   [2.8320 s 2.8913 s 2.9478 s]

solve tensorboard=2.1.1, grpc-cpp=1.39.1/libsolv-sys
                        time:   [473.33 ms 478.27 ms 483.15 ms]
                        change: [+4.2432% +5.3482% +6.3898%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking solve tensorboard=2.1.1, grpc-cpp=1.39.1/libsolv_rs: Warming up for 1.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 6.6s.
Benchmarking solve tensorboard=2.1.1, grpc-cpp=1.39.1/libsolv_rs: AnalyzingCriterion.rs ERROR: error: Failed to access file "/Users/wolfv/Programs/rattler/target/criterion/solve tensorboard=2.1.1, grpc-cpp=1.39.1/libsolv_rs/change/estimates.json": Permission denied (os error 13)
solve tensorboard=2.1.1, grpc-cpp=1.39.1/libsolv_rs
                        time:   [663.65 ms 669.52 ms 678.60 ms]
                        change: [+4.9273% +5.9323% +7.3542%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe

     Running unittests src/lib.rs (target/release/deps/rattler_virtual_packages-8b0906455c8194c3)

@wolfv wolfv merged commit 505700e into conda:main Jul 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants