set Playwright actionTimeout to 15 seconds

[ryanrosello-og]

Playwright global config change to prevent exceedingly long timeouts when certain actions fail
e.g. button click

[qa-wolf]

QA Wolf here! As you write new code it's important that your test coverage is keeping up.
Click here to request test coverage for this PR!

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
modeling-app	✅ Ready (Inspect)	Visit Preview	Aug 7, 2024 10:35am

[Irev-Dev]

Oh cool!

I think the failing macos playwright tests are related seeing some errors like
TimeoutError: page.waitForEvent: Timeout 15000ms exceeded while waiting for event "console"

15000ms makes me think it's this. Should we increase the timeout? or increase it just for these tests?

[Irev-Dev]

Or maybe we need to figure out why these tests need so long?

[ryanrosello-og]

I'm afraid this change won't address those particular issues yet. This will only tweak the test execution so that tests fail faster - (so I dont have to wait too long when rerunning builds).

Currently, actionTimeout is not specified, when a test fails on a particular action
e.g. clicking a button that is not ready to be clicked.

It will fallback to the test timeout (2 minutes)

I've also pushed some changes to:

• enable the html report, this will give insight into which tests are slow.
• enable screenshot on failure - for some reason the canvas does not render the lines in tracefiles. Hoping the screenshot will capture the state of the canvas on failure

[Irev-Dev]

Okay, well I'm not sure exactly what's happening, but these 4 are consistently failing on this branch

So they must be related right, I've tried re-running the tests to make sure it's not a fluke.

[ryanrosello-og]

some progress, I'm getting a different failure now after my most recent commit. The most recent failure is on my list to fix.

  1) [webkit] › flow-tests.spec.ts:4165:7 › Sketch tests › Can delete most of a sketch and the line tool will still work 

    Error: Timed out 10000ms waiting for expect(locator).toBeVisible()

    Locator: locator('[data-message-type="execution-done"]').last()
    Expected: visible
    Received: <element(s) not found>
    Call log:
      - expect.toBeVisible with timeout 10000ms
      - waiting for locator('[data-message-type="execution-done"]').last()


       at test-utils.ts:49

      47 |
      48 | async function expectCmdLog(page: Page, locatorStr: string, timeout = 5000) {
    > 49 |   await expect(page.locator(locatorStr).last()).toBeVisible({ timeout })
         |                                                 ^
      50 | }
      51 |
      52 | async function waitForDefaultPlanesToBeVisible(page: Page) {

        at expectCmdLog (/Users/runner/work/modeling-app/modeling-app/e2e/playwright/test-utils.ts:49:49)
        at Object.expectCmdLog (/Users/runner/work/modeling-app/modeling-app/e2e/playwright/test-utils.ts:254:7)
        at /Users/runner/work/modeling-app/modeling-app/e2e/playwright/flow-tests.spec.ts:4200:13

I'll work on that first

[ryanrosello-og]

Oh cool!

I think the failing macos playwright tests are related seeing some errors like TimeoutError: page.waitForEvent: Timeout 15000ms exceeded while waiting for event "console"

15000ms makes me think it's this. Should we increase the timeout? or increase it just for these tests?

I've extended the timeout for this particular call:

53fb965

[ryanrosello-og]

#3115 (comment)

^ these failures are longer occuring