Solving for flaky tests both in CI and locally

[mikedidomizio]

Is your feature request related to a problem? Please describe.

There seems to be a number of flaky tests in this project as you're aware. I think stability is important here, both for the author but also as a contributor.

Describe the solution you'd like

• GitHub 30 seconds - Link - 5 days ago
• GitHub 30 seconds - Link - last week
• GitHub 30 seconds - Link - 2 weeks ago
• GitHub 30 seconds - Link - 2 weeks ago
• This one it timed out I guess not due to the speed of loading, but maybe an issue on what GitHub loaded - Link - 3 weeks ago

Navigates the "_top" frame of a detached browser

• Link - 5 days ago
• Link - yesterday

Returns the ISO week number and other date tests.

Date tests fail locally for me.

The reason behind this is because I'm in Eastern Canada with a Timezone of GMT-2.
I think you're in Sweden, which is GMT+2.
GitHub action runners are unknown... I can't seem to get a clear answer. I think they're UTC+0.

If you add:

+    beforeAll(() => {
+	    process.env.TZ = 'America/Toronto'
+    })

describe('HTMLInputElementDateUtility', () => {

You'll probably witness that these tests fail now.

If I go with

+    beforeAll(() => {
+	    process.env.TZ = 'Etc/GMT-2'
+    })

describe('HTMLInputElementDateUtility', () => {

The tests pass. When it comes to dates in JavaScript I think we should set a timezone so that we have consistency across all machines.

Describe alternatives you've considered

There's more test failures with testing locally. I don't know how many of these are captured in GitHub. This all stems from my workplace choosing happy-dom for a project, which we might have to go back and weigh the pros and cons and make a choice again between jsdom and happy-dom.

Additional context

Brainstorming potential solutions for test timed out

For the timeout issues, this would be great because as a contributor I can't re-run the workflow against your repo. When I create a PR and everything passes locally and it gets to CI and fails, I'm stuck.

1. Remove tests that touch third-party entities.

As your Browser class mentions much of the interface has been taken from Puppeteer/Playwright, I wanted to say that Playwright even mentions avoid testing-third party dependencies. This might be necessary for this project but perhaps some smaller custom static website you build and host might get the job done. In the above test failures, it seems to be GitHub being more of the problem. Is it necessary to test GitHub and NPM? The tests look similar.

2. MSW to intercept HTTP requests.

Instead of making real requests to GitHub/NPM, use MSW to intercept the requests and return HTML that we can make our checks against. This should give stability as you're never actually hitting those websites. This will still achieve what you're trying to test, which is verifying the Browser class works as intended.

That's off the top of my head this early in the morning. If I had to lean towards a solution, maybe number 2, although removing or skipping the GitHub test if it gets you the same coverage would probably be a quick solution, one that can be remedied later.
I see there's a custom test runner framework in "TestFunctions.js" so not sure how difficult it would be to use MSW.