Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start adding a new rerun order #2067

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 42 additions & 5 deletions features/order.feature
Original file line number Diff line number Diff line change
@@ -1,23 +1,23 @@
Feature: Set the execution order

Background:
Background:
Given a file named "features/a.feature" with:
"""
Feature: some feature
@a
Scenario: first scenario
Given a step

@b
Scenario Outline: second scenario - <ID>
Given a step

@c
Examples:
| ID |
| X |
| Y |

@d
Examples:
| ID |
Expand All @@ -26,7 +26,7 @@ Feature: Set the execution order
And a file named "features/step_definitions/cucumber_steps.js" with:
"""
const {Given} = require('@cucumber/cucumber')

Given(/^a step$/, function() {})
"""

Expand All @@ -47,3 +47,40 @@ Feature: Set the execution order
| second scenario - X |
| second scenario - Y |
| first scenario |

Rule: Scenarios can be run in the order of a rerun file

Scenario: run in rerun order
Given a file named "@rerun.txt" with:
"""
features/a.feature:19
features/a.feature:3
features/a.feature:14
"""
When I run cucumber-js with `--order rerun @rerun.txt`
Then it runs the scenarios:
| NAME |
| second scenario - Z |
| first scenario |
| second scenario - Y |

Scenario: run in rerun order with one incorrect line number
Given a file named "@rerun.txt" with:
"""
features/a.feature:19
features/a.feature:2
features/a.feature:14
"""
When I run cucumber-js with `--order rerun @rerun.txt`
Then it runs the scenarios:
| NAME |
| second scenario - Z |
| second scenario - Y |

Scenario: run in rerun order without a rerun file
When I run cucumber-js with `--order rerun`
Then it fails
And the error output contains the text:
"""
Cannot use rerun order because features/a.feature:13 was not in the rerun order. Did you forget to specify @rerun.txt?
"""
24 changes: 8 additions & 16 deletions src/api/gherkin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,25 +2,12 @@ import {
GherkinStreams,
IGherkinStreamOptions,
} from '@cucumber/gherkin-streams'
import {
Envelope,
GherkinDocument,
IdGenerator,
Location,
ParseError,
Pickle,
} from '@cucumber/messages'
import { Envelope, IdGenerator, ParseError } from '@cucumber/messages'
import { Query as GherkinQuery } from '@cucumber/gherkin-utils'
import PickleFilter from '../pickle_filter'
import { orderPickles } from '../cli/helpers'
import { orderPickles, PickleWithDocument } from '../cli/helpers'
import { ISourcesCoordinates } from './types'

interface PickleWithDocument {
gherkinDocument: GherkinDocument
location: Location
pickle: Pickle
}

export async function getFilteredPicklesAndErrors({
newId,
cwd,
Expand Down Expand Up @@ -85,7 +72,12 @@ export async function getFilteredPicklesAndErrors({
pickle,
}
})
orderPickles(filteredPickles, coordinates.order, logger)
orderPickles(
filteredPickles,
coordinates.order,
unexpandedFeaturePaths,
logger
)
return {
filteredPickles,
parseErrors,
Expand Down
49 changes: 47 additions & 2 deletions src/cli/helpers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,12 @@ import { OptionSplitter } from '../configuration'
import { Readable } from 'stream'
import os from 'os'
import * as messages from '@cucumber/messages'
import { IdGenerator } from '@cucumber/messages'
import {
GherkinDocument,
IdGenerator,
Location,
Pickle,
} from '@cucumber/messages'
import detectCiEnvironment from '@cucumber/ci-environment'
import { ISupportCodeLibrary } from '../support_code_library_builder/types'
import TestCaseHookDefinition from '../models/test_case_hook_definition'
Expand All @@ -16,12 +21,19 @@ import { PickleOrder } from '../models/pickle_order'
import { builtinParameterTypes } from '../support_code_library_builder'
import { version } from '../version'

export interface PickleWithDocument {
gherkinDocument: GherkinDocument
location: Location
pickle: Pickle
}

interface IParseGherkinMessageStreamRequest {
cwd?: string
eventBroadcaster: EventEmitter
eventDataCollector: EventDataCollector
gherkinMessageStream: Readable
order: string
unexpandedFeaturePaths?: string[]
pickleFilter: PickleFilter
}

Expand All @@ -34,13 +46,15 @@ interface IParseGherkinMessageStreamRequest {
* @param eventDataCollector
* @param gherkinMessageStream
* @param order
* @param unexpandedFeaturePaths
* @param pickleFilter
*/
export async function parseGherkinMessageStream({
Copy link
Contributor

@davidjgoss davidjgoss Jun 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is deprecated and will be removed in a future major release, however in the meantime it'd be better to avoid making a breaking change to it. Could we make the new field optional?

(It's fine to change orderPickles, that's an internal one.)

eventBroadcaster,
eventDataCollector,
gherkinMessageStream,
order,
unexpandedFeaturePaths,
pickleFilter,
}: IParseGherkinMessageStreamRequest): Promise<string[]> {
return await new Promise<string[]>((resolve, reject) => {
Expand All @@ -59,7 +73,7 @@ export async function parseGherkinMessageStream({
}
})
gherkinMessageStream.on('end', () => {
orderPickles(result, order, console)
orderPickles(result, order, unexpandedFeaturePaths, console)
resolve(result)
})
gherkinMessageStream.on('error', reject)
Expand All @@ -70,6 +84,7 @@ export async function parseGherkinMessageStream({
export function orderPickles<T = string>(
pickleIds: T[],
order: PickleOrder,
unexpandedFeaturePaths: string[] | undefined,
logger: Console
): void {
const [type, seed] = OptionSplitter.split(order)
Expand All @@ -85,6 +100,36 @@ export function orderPickles<T = string>(
shuffle(pickleIds, seed)
}
break
case 'rerun':
{
if (unexpandedFeaturePaths === undefined) {
throw new Error(
'Cannot order by rerun because no unexpandedFeaturePaths were provided'
)
}

const picklesWithDocument =
pickleIds as unknown[] as PickleWithDocument[]

picklesWithDocument.sort((a, b) => {
const pathA = `${a.pickle.uri}:${a.location.line}`
const pathB = `${b.pickle.uri}:${b.location.line}`
const indexA = unexpandedFeaturePaths.indexOf(pathA)
const indexB = unexpandedFeaturePaths.indexOf(pathB)
if (indexA === -1) {
throw new Error(
`Cannot use rerun order because ${pathA} was not in the rerun order. Did you forget to specify @rerun.txt?`
)
}
if (indexB === -1) {
throw new Error(
`Cannot use rerun order because ${pathB} was not in the rerun order. Did you forget to specify @rerun.txt?`
)
}
return indexA - indexB
Copy link
Member

@charlierudolph charlierudolph Aug 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note I think this could be something like

const ordered = new Array(unexpandedFeaturePaths.length);
picklesWithDocument.forEach(item => {
  // get index in unexpandedFeaturePaths, throw if not found
  ordered[index] = item
})

// remove empty indexes (only occurs if have duplicates in list and we just use the first) 

That should be O(n) instead O(nlogn)

})
}
break
default:
throw new Error(
'Unrecgonized order type. Should be `defined` or `random`'
Expand Down
2 changes: 1 addition & 1 deletion src/configuration/argv_parser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ const ArgvParser = {

.option(
'--order <TYPE[:SEED]>',
'run scenarios in the specified order. Type should be `defined` or `random`'
"run scenarios in the specified order. Type should be 'defined', 'random' or 'rerun'"
)
.option(
'-p, --profile <NAME>',
Expand Down
2 changes: 1 addition & 1 deletion src/models/pickle_order.ts
Original file line number Diff line number Diff line change
@@ -1 +1 @@
export type PickleOrder = 'defined' | 'random' | string
export type PickleOrder = 'defined' | 'random' | 'rerun' | string