wp cgr run command exits wrong

[Shelob9]

We need thewp cgr run command to exit in a way that is interpreted by Travis as a a failure of a specific test

[Shelob9]

BACKGROUND:
We are using Ghost Inspector to run acceptance tests. Basically they give you point and click way to record Selenium tests. It's cool.

Currently we have a Pantheon site that we manually switch to the branch that we're testing. That's dumb.

This plugin imports all of the forms we need to test and puts them on pages. Since the names of the pages are predictable, we can use Ghost Inspector's API to point at a dynamic URL/known slug for each test. We can related the forms to Ghost Inspector IDs, because the form import files we're using are in spreadsheet that has the ID of the test. We're using Google docs and a AWS Lamda function to serve it as JSON via HTTPS as the back-end for organizing the tests, that's all solid.

That plugin has two WP CLI commands. One to import the forms and puts them on pages, and one to make the API calls to Ghost Inspector to ask it to run the tests on the current site. This approach I went with since what we needed for the API call is the test ID -- the import script puts that in post meta -- and the URL of the page.

Then I had Mike redo our Travis setup so on some builds it installs that plugin, imports the forms using wp cli, serves the site publicly using ngrok, then wp cli is used to tell Ghost Inspector, via its API to run the tests against the current site. That all works. In fact I reverted the part that makes it work with Ghost Inspector because I wanted the better way of working with phpunit that Mike set up. Also, the actual reason I need help.

REQUIRMENTS:
We require changes to wp cgr run command and/or the Travis file in Caldera Forms so that:

1. If any tests fail, let the build finish then report failure to Github so the PR is blocked.
2. Every test that failed needs to "become known to us".

By "known to us" I mean either:

1. It's printed in the output of the Travis log.
2. Which tests failed is reported in the issue as a comment. https://developer.github.com/v3/issues/comments/#create-a-comment
   https://github.com/KnpLabs/php-github-api/blob/master/doc/issue/comments.md#update-a-comment-on-an-issue

2 is preferable, but I need CalderaWP/Caldera-Forms#2254 resolved and to start making more tests yesterday.