send reason for failed session to browserstack

[codekie]

Closes #2356

This PR will send the name of the failed tests to Browserstack.

[AshishKapoor]

LGTM

[MicahLC]

I can't merge this, but it lgtm!

[beatfactor]

I'm not sure this would be very useful in the case of a big number of failed testcases, also we cannot be sure that the title of the test cases is safe to be sent to browserstack, as in it might contain sensitive information, such as emails and other privacy related data.

The only thing we could do here is to provide a mechanism so that users can define their own "reason" exporting function.

[codekie]

With a big number of failed testcases (per test-suite that is, as they are collected on suite-level), this is not optimal that is true (the title gets truncated after 255 characters). However, right now it only shows "failed" which isn't very helpful (even less if there are a lot of failed tests) and it always involves quite some effort to backtrace the failed test (for every single failed test, without even knowing how many of them failed).

Regarding sensitive information in the title of test cases:
Browserstack saves screenshots and even screen recordings of the test-scenarios, which may contain way more sensitive information than just in the test-title. Everyone who uses Browserstack is aware of that. So, I think, having the test-titles there should not be much of an issue unless someone puts critical information like passwords or environment-variables there.

If the reason-exporting function has access to the failed tests (and thus can extract the titles automatically, if desired) and if it can be declared globally that's fine as well.

As it is right now, for a browserstack-test-setup with a multitude of OSes, browsers and browser versions (which is the purpose of browserstack) and more than one test per test-suite, the overhead of identifying the failed tests is too high to make use of nightwatch with the browserstack-reporter.

[beatfactor]

Which browserstack-reporter are you referring to?

[codekie]

Sorry, I meant "transport", not "reporter"

[stale]

This issue has been automatically marked as stale because it has not had any recent activity.
If possible, please retry using the latest Nightwatch version and update the issue with any relevant details. If no further activity occurs, it will be closed. Thank you for your contribution.

[MicahLC]

I still think this would be a nice-to-have option.