[Meta] Fix intermittent issues in the integration tests #18396
Comments
|
Note to self, the following is useful for reproducing scripting-related intermittents locally: diff --git a/web/pdf_scripting_manager.js b/web/pdf_scripting_manager.js
index 63d08c604..68d1b8a21 100644
--- a/web/pdf_scripting_manager.js
+++ b/web/pdf_scripting_manager.js
@@ -177,6 +177,7 @@ class PDFScriptingManager {
return; // The document was closed while the properties resolved.
}
+ await new Promise(r => setTimeout(r, 2000));
await this.#scripting.createSandbox({
objects,
calculationOrder,diff --git a/src/pdf.sandbox.external.js b/src/pdf.sandbox.external.js
index 365d94a60..af646c29f 100644
--- a/src/pdf.sandbox.external.js
+++ b/src/pdf.sandbox.external.js
@@ -67,7 +67,7 @@ export class SandboxSupportBase {
}
try {
args = this.exportValueToSandbox(args);
- this.commFun(name, args);
+ this.win.setTimeout(this.commFun.bind(this, name, args), 300);
} catch (e) {
this.win.console.error(e);
|
|
For the |
…tests The `waitForClick` helper function is functionality-wise mostly a reduced copy of the more generic `waitForEvent` helper function that we use for other integration tests, so we can safely replace it to reduce the amount of code. Moreover, the `waitForClick` code is prone to intermittent failures given recent assertion failures we have seen on the bots (one of them is linked in mozilla#18396) while `waitForEvent` has recently been fixed to avoid such intermittent failures, so this should also remove the flakiness for these integration tests.
…tests The `waitForClick` helper function is functionality-wise mostly a reduced copy of the more generic `waitForEvent` helper function that we use for other integration tests, so we can safely replace it to reduce the amount of code. Moreover, the `waitForClick` code is prone to intermittent failures given recent assertion failures we have seen on the bots (one of them is linked in mozilla#18396) while `waitForEvent` has recently been fixed to avoid intermittent failures, so usiong it should also get rid of the flakiness for these integration tests.
|
I have debugged the
My suspicion is that the fact that this throws during serialization breaks the annotation storage's map and causes the next However, I don't know enough about the stamp editor implementation to pinpoint exactly what's wrong. It looks like a race condition where the stamp editor may not expect its @calixteman Does this ring a bell for you, since you're much more familiar with the stamp editor than I am? |
|
Could you check if waiting for the canvas being visible is enough ? await page.waitForSelector(`${editorSelector} canvas`, { visible: true }); |
The `must check input for US zip format` integration test fails pretty consistently in Puppeteer 23.4.0+ with `Expected '12341' to equal '12345'`. This is reproducible with the `pdf.sandbox.external.js` hack from mozilla#18396 (comment). Investigation uncovered two issues at play here: 1. We do two `clearInput` calls, but don't await processing of the two sandbox events that are triggered by that action. The three tests that use `issue14307.pdf` are in different `describe` blocks and therefore reload the PDF file, so we can simply remove those calls because the inputs are already empty by default. 2. We don't await processing of the sandbox events that occur after switching to another text field. This causes the expectation failure because the typing actions will happen too soon and interfere with the sandbox event processing. We solve the issue by explicitly awaiting the sandbox roundtrip. Moreover, similar to PR mozilla#19001 and mozilla#18399 we remove any remaining room for intermittent issues by directly checking for the expected value, which also results in shorter code.
Despite having fixed the most occurring intermittent failures in the integration tests (see the closed issues with the
intermittentlabel) we unfortunately still have a few of them left that trigger from time to time. We should list and fix them. This issue is a meta issue for this effort.The following intermittents have been found on the bots as of July 4th, 2024, which is after landing all previously identified improvements:
The text was updated successfully, but these errors were encountered: