Visual regression tests to protect against accidental CSS side effects

[cjcenizal]

Visual regression testing makes more sense in the UI Framework than inside of a consumer of EUI, e.g. Kibana, because regressions will be on a more granular scale and easier to address. Ideally we'd implement this using a tool which also supports cross-browser testing (here's a list of services we can consider). See https://blog.hichroma.com/why-design-systems-are-a-single-point-of-failure-ec9d30c107c2 for more background.

[cuff-links]

I will run with this.

[snide]

@silne30 yeah! ❤️

[cuff-links]

Visual Regression Tests

• Add WebDriver.io and drivers as dependencies.
  • WebDriver.io
  • wdio-visual-regression-service
  • GeckoDriver 1.10.0
  • Chromedriver 2.36
  • IE Driver (for IE 11)
• Configuration
  • Test Runner
    • Mocha/Chai (for now)
      - [ ] Switch to Jest once [support has been added](Jest Support webdriverio/webdriverio#2052).
  • Cross Browser Service
    • Sauce Labs

The webdriver.io team is not adding Jest support anymore. Honestly, it's not needed.

[cuff-links]

Had a mini-demo and here are some of the take away items for what's left.

- [ ] Issue with Chromedriver running locally. See issue

• Using Selenium 3.4 until above lissted issue is fixed.
• Add .png files and src-docs to be excluded from npm
• Map each user in data_store.js to a different country.
  - [ ] Abstract the png baseline files to a submodule (or possibly even the entire test framework).
• Create README.md to show how to write and run tests.

[christian-bromann]

Just leaving this here: webdriverio/webdriverio#2052 (comment) 😉

[cuff-links]

Creating a JS script to run the tests (creating baselines and then rerunning). For CI, there will be a --destroy flag or --create-baseline flag to remove cached images.