Allure prevents cypress-cucumber-preprocessor to produce proper JSON reports

[mysticdevx]

Hi, I am using allure in Cypress framework. I use allure for reporting and cypress-cucumber-preprocessor reporter to have json reports in order to import results to xRay. Whenever I activate allure (either in cypress.json file or through cli) the cypress report JSONs don't include the failed scenarios. but when I deactivate allure (allure=false), the cypress cucumber report behaves as expected. Can you suggest a workaround or solution to this issue?

Expected behaviour
Even if the allure is activates, cypress-cucumber-preprocessor should prepare proper results, including failed scenarios

Environment (please complete the following information):

• Cypress version: 5.6.0 (tried with 6.x too)
• cypress-cucumber-preprocessor. Latest
• Allure dependencies Latest
• OS: Mac - Big Sur (Tried in previous version too)

In package json, I have:

 "cypress-cucumber-preprocessor": {
    "nonGlobalStepDefinitions": true,
    "cucumberJson": {
      "generate": true,
      "outputFolder": "cypress/cucumber-json",
      "filePrefix": "",
      "fileSuffix": ".cucumber"
    }
  }

and in cypress.json I have:

  "env": {
    "issuePrefix": "xxxxxxx",
    "allure": true,
    "allureResultsPath": "allure-results"
  }

I also tried allure: false and run with --env allure=true in cli

Here is sample project:
https://github.com/mysticdevx/webPackExample

[markroche32]

This is a valid issue when allure : true in cypress.json,

cypress-allure-plugin and cypress-cucumber-preprocessor are interfering with each other.

cypress-allure-plugin is preventing the generation of cucumber.json for failed scenarios.

All this has been mentioned above.

However, The Question is where does the fix need to be implemented ? From the side of (1) cypress-allure-plugin (2) cypress-cucumber-preprocessor

My inclination without doing any real technical troubleshooting is that the fix might require at least some changes from the side of (2) cypress-cucumber-preprocessor, to take into consideration integrating with cypress-allure-plugin and case where allure:true , that this specific case should not impact the standard flow in generation of cucumber.json results for passed, failed, passed and failed scenarios.

[Shelex]

@mysticdevx Thank you for providing example!
Have checked myself, also reproduces, however I am not sure about source of issue.
I have found that in case Cypress has listener on 'fail' event it leads to empty cucumber report:

Cypress.on('fail', (err) => {
            console.warn(`[cypress] fail ${err}`)
            throw err;
        });

cypress-allure-plugin is listening "fail" event to finish all previous commands happening in allure reporter.

So here are action items I have for myself:

• investigate what is going on under the hood of Cypress when this event fired
• investigate why this happening inside cucumberDataCollector or cucumber/lib/formatter dependency
• investigate if allure fail handler could be moved elsewhere or removed completely

[Shelex]

@mysticdevx @markroche32
As I see there are several issues which involve different parties :)

1. It seems that Cypress can have only one listener for global Cypress.on("fail") event which:

exists because it’s extremely useful for debugging purposes.

Maybe it makes sense to open issue for Cypress to check if it is possible to have multiple listeners on this event.

2. cypress-allure-plugin uses Cypress.on('fail') event while it could be handled in mocha.on('fail') to avoid such issues with Cypress or other plugins. Fixed in v2.3.7.

3. cypress-cucumber-preprocessor uses Cypress.on('fail') which may lead to issues when users have own listeners. Maybe makes sense to open issue in their repo to switch this fail listener to mocha.on('fail').

Anyway, please check if cypress-allure-plugin v2.3.7 works for you now.

[mysticdevx]

Hey @Shelex
Thank you for the update. I have installed v2.3.7 and now I can get the failed tests in cucumber-json files. Many thanks