Add tests for Image Heatmap question creation and response

[jxjj]

This adds tests for image heatmap questions via cypress image snapshots. 🖼️ ✅ 👏

Notable... err... notes:

• New dev dependency: cypress-image-snapshot, which is based on americanexpress/jest-image-snapshot.
• Change .env.example so that, by default, DEBUG_BAR=false. Debug bar will cause all sorts of false positives with image diffing, and cur CI workflow .env file is a copy of .env.example. So, this makes sure DEBUG_BAR won't be in any CI screenshots.
• cypress/<testname>/__diff_output__ will contain any generated diffs from failing snapshot tests 😎. Add this to .gitignore to prevent accidental committing.
• cypress/plugins/index.js is renamed to index.ts. We're using type: module in this project now, but cypress doesn't support ESM or.cjs file extensions yet. A workaround is naming it with a .ts extension so that cypress parses the file as if it were typescript.
• Testing image snapshots was a little flakey due to timing issues. Using cy.intercept to wait for the PUT request helped. I also changed the diff threshold to 0.5% to reduce false positives.

Thanks for the review!

[cmcfadden]

this is super cool. I'll be interested to live with it for a bit and get a sense of whether it ends up flapping much

[jxjj]

Let the flapping begin...

I'll have to make a few changes tomorrow before merging. The local images and the ones generated in CI are different sizes. Probably need to generate the shots using the cypress docker image like CI uses.

[jxjj]

TLDR

So, this now passes consistently in headless mode (cypress run) both locally and in CI.

When cypress is in interactive mode (cypress open), the test will likely fail due to environment differences: OS, browser, default fonts, screen size, dpi, etc all impact the screenshot.

More details

Screenshot size was the biggest problem for me when not in headless. Even within the same environment, I found that running tests on my iPad screen gave different results from my 13" macbook pro (even when specifically told to use a window specific browser size like of 1920x1080. Presumably due to pixels per inch differences between the screens (264 in iPad Pro vs 227 in Macbook Pro)?

This seems to be a common issue with Cypress and visual regression testing. More Discussion:

• Taking screenshots during cypress open versus cypress run result in different screenshot sizes cypress-io/cypress#3324
• Test failing because of images sizes are not same jaredpalmer/cypress-image-snapshot#219
• snapshots recorded during cypress run mismatch cypress open jaredpalmer/cypress-image-snapshot#95

Cypress' line is that one should ensure consistent environment when testing for visual regressions... which makes sense I guess.

I wonder if 3rd party services like Percy or Applitools solve this problem?

Interestingly, it looks like jest-image-snapshot – which the cypress package is base one – is working on a structural similarity (SSIM) approach rather than a direct pixel-by-pixel comparison. Also, they have implemented the ability to ignore size mismatches. However, neither option worked for me in the cypress snapshot package.

I'll merge this test for now. Like you mentioned, we can explore an alternate approach if it flakes.

[jxjj]

Oh! I should also mention I explored just moving to all local cypress tests to their docker image. I kept running into roadblocks trying to run this on the M1. Maybe in the future?