Support parallel_test options: pattern, exclude-pattern & group-by

[inkstak]

This PR add suport for some parallel_test options:

    -p, --pattern [PATTERN]          run tests matching this regex pattern
        --exclude-pattern [PATTERN]  exclude tests matching this regex pattern
        --group-by [TYPE]            group tests by:
                                     found - order of finding files
                                     steps - number of cucumber/spinach steps
                                     scenarios - individual cucumber scenarios
                                     filesize - by size of the file
                                     runtime - info from runtime log
                                     default - runtime when runtime log is filled otherwise filesize

[inkstak]

There could be one issue around --group-by default:
it relies on private methods from parallel_tests : https://github.com/serpapi/turbo_tests/pull/41/files#diff-36313b614b64b302e0ceec80e7a9a2dbaea194523dc46a44b7fffda7741674a5R109

[inkstak]

I would also suggest another PR to handle properly Fuubar #23 depending on --pattern and --exclude-pattern.
It'll also required one of those private methods.

[ilyazub]

Thanks for your contribution, @inkstak!

Let me think if we want turbo_tests to depend more on parallel_tests.

[kalashnikovisme]

Hey, @ilyazub! Do you have a decision about the level of dependency on parallel_tests? 🙂

NOTE: as I know, these options belong to rspec.

[steobrien]

+1, this doesn't increase the dependency on parallel_tests (although this gem is a drop-in replacement for that, so should support anyway), because --pattern and --exclude-pattern are RSpec flags

Any chance you could merge, @ilyazub?

[inkstak]

FYI: --pattern & --exclude-pattern are both supported by RSpec & parallel_tests but they don't use the same implementation and doesn't support the same arguments.

For example, we use a CI script to launch our specs :

if parallel
  command = "bundle exec parallel_rspec"
  command = "bundle exec turbo_tests" if parallel == "turbo_tests"

  command += " --exclude-pattern spec/system" if arg == "unit"
  command += " --pattern spec/system"         if arg == "system"
else
  command = "bundle exec rspec"

  command += " --exclude-pattern 'system/**/*_spec.rb'" if arg == "unit"
  command += " --pattern 'system/**/*_spec.rb'"         if arg == "system"
end

In this PR, I forward those arguments to parallel_tests so it uses its implementation and not the one from RSpec.

[ilyazub]

@inkstak @kalashnikovisme @steobrien Thanks for your work and sorry for not reviewing this PR sooner.

Let's record runtime log only when --group-by runtime is passed.

Because we want to support most of the ParallelTests options, may we call ParallelTests::CLI.send(:parse_options!, @argv) in TurboTests::CLI#run?

turbo_tests/lib/turbo_tests/cli.rb

Line 20 in b4b8856

parallel_options = {}

[inkstak]

Because we want to support most of the ParallelTests options, may we call ParallelTests::CLI.send(:parse_options!, @argv) in TurboTests::CLI#run?

PR updated ! You can pass any options supported by parallel_tests after -- :

bundle exec turbo_tests -n 4 -- --only-group 1 --pattern spec/system

Because OptionParser raises an exception on unknown options, I found it easier to split options like this.
Parallel_tests suggests the same syntax to pass test options :

parallel_rspec -- -t acceptance -f progress -- spec/foo_spec.rb spec/acceptance

[inkstak]

Let's record runtime log only when --group-by runtime is passed.

Not sure if this line is still relevant...

[darokel]

Thanks for all your work on this. @ilyazub Is this close to being merged? @ilyazub looks like the changes have been addressed - are we good to merge this?