Test against latest stable browser versions (open new PR to do so)

[jennifer-shehane]

Current behavior:

We recently implemented this update to our internal testing process to always test against the latest browser version: #6115

This has become problematic, as evidenced by Chrome 80 update that occurred Feb 4. We are trying to release 4.0 and in the midst of this release the tests are not passing due to some change in Chrome 80. This makes it difficult to isolate which test failures are coming from the current branch changes and which are coming from the new Chrome version changes.

This was effectively removed in #6329

Furthermore this issue will just compound as we add more browser support.

Desired behavior:

We should do something similar to how our renovatebot works today for our repo.

Upon release of a new stable browser version:

• Create a new docker image with new Chrome version?
• Initiate a PR against cypress that runs all tests against the latest browser version.

Then we can take the time to isolate which tests failed due to the new stable release and track them down.

Questions

• Do we need to do this on EVERY stable release? How often do these happen? Will this be work overload? We could potentially auto-merge them once that pass all tests automatically so that the devs don't have to merge in a thousand stable releases and request review for any PRs that fail.

Versions

Cypress 3.8.3

[jennifer-shehane]

Hotfix for develop: #6329

[clarmso]

At least we've tried. :'(

We could create a new docker image for the new Chrome release, but the docker image creation needs to be triggered.

I subscribe to this calendar to be notified of the Chrome release:
https://calendar.google.com/calendar/embed?src=google.com_c1f795f7cg83sv2qu15c4v1kfk%40group.calendar.google.com&ctz=America%2FToronto

[jennifer-shehane]

Closing and tracking internally.