Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(vitest-runner): add first implementation #26

Closed
wants to merge 10 commits into from

Conversation

adriencaccia
Copy link
Member

No description provided.

@codspeed-hq
Copy link

codspeed-hq bot commented Oct 2, 2023

CodSpeed Performance Report

Merging #26 will create unknown performance changes

Comparing feat/vitest-runner (161e9e3) with main (7f00e39)

Summary

🆕 2 new benchmarks
⁉️ 64 dropped benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark main feat/vitest-runner Change
🆕 short body N/A 502.7 µs N/A
🆕 long body N/A 1.4 ms N/A
⁉️ RegExp#test 20.5 µs N/A N/A
⁉️ String#indexOf 14.3 µs N/A N/A
⁉️ test_recursive_fibo_20 18.5 ms N/A N/A
⁉️ test_recursive_cached_fibo_30 152.6 µs N/A N/A
⁉️ switch 2 17.8 µs N/A N/A
⁉️ test_recursive_fibo_10 170.6 µs N/A N/A
⁉️ test_recursive_cached_fibo_20 106.4 µs N/A N/A
⁉️ switch 2 17.7 µs N/A N/A
⁉️ switch 1 316.5 µs N/A N/A
⁉️ test async baz 10 1.7 ms N/A N/A
⁉️ test_iterative_fibo_100 74.6 µs N/A N/A
⁉️ test_recursive_cached_fibo_10 68.5 µs N/A N/A
⁉️ test sync baz 10 40.9 µs N/A N/A
⁉️ test_iterative_fibo_10 43.2 µs N/A N/A
⁉️ test_iterative_fibo_10 43.2 µs N/A N/A
⁉️ test_recursive_fibo_10 467.7 µs N/A N/A
⁉️ test_recursive_fibo_20 18.5 ms N/A N/A
⁉️ test async baz 100 1.7 ms N/A N/A
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

@codspeed-adrien-dev
Copy link

codspeed-adrien-dev bot commented Oct 24, 2023

CodSpeed Performance Report

Merging #26 will improve performances by ×7

⚠️ No base runs were found

Falling back to comparing feat/vitest-runner (a9eaac7) with main (67a4f23)

🎉 Hooray! codspeed-node just leveled up to 2.2.0!

A heads-up, this is a breaking change and it might affect your current performance baseline a bit. But here's the exciting part - it's packed with new, cool features and promises improved result stability 🥳!
Curious about what's new? Visit our releases page to delve into all the awesome details about this new version.

Summary

⚡ 4 improvements

🆕 80 new benchmarks

Benchmarks breakdown

Benchmark main feat/vitest-runner Change
String#indexOf 135.5 µs 14.2 µs ×9.5
switch 2 172.4 µs 17.6 µs ×9.8
switch 1 155.2 µs 17 µs ×9.1
🆕 long body N/A 1.4 ms N/A
🆕 short body 2 N/A 481.8 µs N/A
🆕 short body N/A 502.7 µs N/A
🆕 long body N/A 1.6 ms N/A
🆕 short body N/A 262.3 µs N/A
RegExp#test 142.5 µs 20.5 µs ×7
🆕 short body 4 N/A 497.7 µs N/A
🆕 long body N/A 1.4 ms N/A
🆕 short body N/A 234.7 µs N/A
🆕 short body 5 N/A 222.8 µs N/A
🆕 short body N/A 502.9 µs N/A
🆕 short body N/A 500.5 µs N/A
🆕 test_recursive_fibo_10 N/A 170.7 µs N/A
🆕 fibo 10 N/A 161.8 µs N/A
🆕 long body N/A 1.4 ms N/A
🆕 short body N/A 388.5 µs N/A
🆕 switch 1 N/A 17 µs N/A
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

};
}

export async function withCodSpeed(
Copy link

@sheremet-va sheremet-va Nov 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend implementing this as a plugin since this is how config is usually modified in the ecosystem:

// vitest.config.ts
import { CodeSpeedPlugin } from '@codespeed/vitest-runner'
import { defineConfig } from 'vitest/config'

export default defineConfig((mode) => ({
  plugins: [mode === 'benchmark' ? CodeSpeedPlugin() : null]
}))
// plugin.ts
export default () => {
  return {
    name: 'codespeed:vitest'
    config() {
      // you don't need to merge it yourself, just return modified version
      return {
        test: {
          poolOptions: {
          forks: {
              // eslint-disable-next-line @typescript-eslint/ban-ts-comment
              // @ts-ignore - remove when https://github.com/vitest-dev/vitest/pull/4383 is merged and released
              execArgv: getV8Flags(),
            },
          },
          runner: `${__dirname}/runner.es5.js`,
          globalSetup: [
            `${__dirname}/globalSetup.es5.js`,
          ],
        }
      }
    }
  }
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion, I implemented it in the new PR #28

@adriencaccia adriencaccia deleted the feat/vitest-runner branch November 13, 2023 10:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants