--format-options '{"colorsEnabled": false}' does not apply on error messages

[hdorgeval]

Hello Cucumber Team,

Yet another issue related to displaying error messages with the internal HTML formatter:

Given I disable colors on the CLI with --format-options '{"colorsEnabled": false}'
When a step fails
Then all logs are uncolored
But the Error message still uses Chalk red/green colors
Example in the console:

Example in the html report

And the corresponding envelope is:

{
  "testStepFinished": {
    "testStepResult": {
      "status": "FAILED",
      "message": "Error: \u001b[2mexpect(\u001b[22m\u001b[31mreceived\u001b[39m\u001b[2m).\u001b[22mtoBe\u001b[2m(\u001b[22m\u001b[32mexpected\u001b[39m\u001b[2m) // Object.is equality\u001b[22m\n\nExpected: \u001b[32m13\u001b[39m\nReceived: \u001b[31m12\u001b[39m\n    at CustomWorld.<anonymous> (/Users/HDO/VSCodeProjects/cucumber7-ts-starter/step-definitions/maths/simple-maths-steps.ts:19:34)",
      "duration": {
        "seconds": "0",
        "nanos": 12000000
      }
    },
    "timestamp": {
      "seconds": "1611571916",
      "nanos": 62000000
    },
    "testStepId": "71ddfe75-49af-4db7-b5ba-d264dfd2fd64",
    "testCaseStartedId": "75fd16c7-5a2a-45b2-8d7e-907f9dc03988"
  }

In v6 the work around was to set --no-color ( see chalk support color) on the CLI.

Since it not possible any more in v7, the only way to fix this issue in v7 is to set the environment variable:

process.env['FORCE_COLOR'] = '0';

It would be great if you could either give back support for the --no-color option on the CLI or that you set the FORCE_COLOR environment variable.

In resume:
Given I disable colors on the CLI with --format-options '{"colorsEnabled": false}'
And I set the environment variable FORCE_COLOR to 0
When a step fails
Then all logs are uncolored
And the Error message is also uncolored
Example:

And the corresponding envelope is:

{
  "testStepFinished": {
    "testStepResult": {
      "status": "FAILED",
      "message": "Error: expect(received).toBe(expected) // Object.is equality\n\nExpected: 13\nReceived: 12\n    at CustomWorld.<anonymous> (/Users/HDO/VSCodeProjects/cucumber7-ts-starter/step-definitions/maths/simple-maths-steps.ts:19:34)",
      "duration": {
        "seconds": "0",
        "nanos": 15000000
      }
    },
    "timestamp": {
      "seconds": "1611573757",
      "nanos": 480000000
    },
    "testStepId": "a1324e42-c3fb-4e76-a860-3973fe6a7b34",
    "testCaseStartedId": "c6a53836-32af-4528-8811-2400f22a4ce5"
  }
}

Thanks

[davidjgoss]

@hdorgeval I’m struggling to see how this relates to the html formatter - this is a console output issue?

[hdorgeval]

Hi @davidjgoss, the problem i described has almost no impact when the formatter outputs to the console.

But it has a great impact when the preferred formatter is the html formatter as, in this case, error messages are unreadable.

My point is that error messages are, at some point, handled by Chalk and there is no way to configure Chalk behavior from the cucumber-js command-line options, as it was the case before in v6.

So my point is that in v7, when colors are disabled on the cucumber-js command-line, you should setup also environment variables for the Chalk module, maybe like this:

get_color_fns.ts

export default function getColorFns(enabled: boolean): IColorFns {
  if (enabled) {
    return {
      forStatus(status: messages.TestStepFinished.TestStepResult.Status) {
        return {
          [Status.AMBIGUOUS]: colors.red.bind(colors),
          [Status.FAILED]: colors.red.bind(colors),
          [Status.PASSED]: colors.green.bind(colors),
          [Status.PENDING]: colors.yellow.bind(colors),
          [Status.SKIPPED]: colors.cyan.bind(colors),
          [Status.UNDEFINED]: colors.yellow.bind(colors),
          [Status.UNKNOWN]: colors.yellow.bind(colors),
        }[status]
      },
      location: colors.gray.bind(colors),
      tag: colors.cyan.bind(colors),
    }
  } else {
    // CONFIGURE CHALK BEHAVIOR FOR ALL MODULES THAT USES CHALK
    process.env['FORCE_COLOR'] = '0';
    return {
      forStatus(status: messages.TestStepFinished.TestStepResult.Status) {
        return _.identity
      },
      location: _.identity,
      tag: _.identity,
    }
  }
}

[hdorgeval]

@davidjgoss , another way to solve this issue could be to say that the HTML formatter should support Chalk colors

[davidjgoss]

@hdorgeval okay I'm with you now.

So as I see it we have a couple of issues:

• --format-options '{"colorsEnabled": false}' not being respected for error messages - I'm pretty sure this was not intentionally removed, more likely a regression
• terminal color characters being emitted in the messages (and thus picked up by HTML formatter)

I don't think we should be passing anything through chalk (or equiv) to emit messages, so we should stop that from happening as part of addressing the first issue. I think to display assertion errors nicely in the HTML formatter we should look to emit a more structured error object with expected vs actual values (which assertion libraries already provide for us).

I'll dig into this more a little later.

[hdorgeval]

@davidjgoss , for the first issue, this was a hidden issue in v6 because it was possible to set --no-color option on the command-line to solve it.

I definitely agree with you that emitted messages should not be generated with chalk and that error messages should be a structured object.

Thnks for your time 👍

For people reading this issue, the minimal workaround to correctly display error messages when using the HTML formatter is to set the environment variable FORCE_COLOR to 0. There is no need to set --format-options '{"colorsEnabled": false}'.

[davidjgoss]

@hdorgeval what assertion library are you using here?

[davidjgoss]

Just to fill some of the blanks here, the assertion library in use is Jest's expect, which uses chalk directly and therefore can't be influenced by our format option. Setting FORCE_COLOR ourselves would be invasive - I'd rather take the suggested line of advising users to set FORCE_COLOR.

[davidjgoss]

Closing after changes in 8.2.0, see https://github.com/cucumber/cucumber-js/blob/main/docs/formatters.md#colored-output