Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flakiness on custom time range saved searches #165454

Merged
merged 16 commits into from
Sep 25, 2023
Merged

Fix flakiness on custom time range saved searches #165454

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
e61629a
Wait for the custom time range toggle animation
gsoldevila Sep 1, 2023
f74edaa
Use existing common.sleep() when possible
gsoldevila Sep 1, 2023
fb9d771
[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…
kibanamachine Sep 1, 2023
49027fa
Make sure we toggle to the right state
gsoldevila Sep 5, 2023
b960129
Merge branch 'main' into fix-flakiness-on-custom-time-range-saved-sea…
gsoldevila Sep 5, 2023
a1c6678
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Sep 5, 2023
46a7566
Address PR feedback
gsoldevila Sep 6, 2023
5869890
Add retry mechanism for toggles' missed clicks
gsoldevila Sep 6, 2023
cc323cc
Merge branch 'main' into fix-flakiness-on-custom-time-range-saved-sea…
gsoldevila Sep 6, 2023
5b47af8
Fix custom time range state detection
gsoldevila Sep 7, 2023
83d14d2
Fix logic using isSelected incorrectly
gsoldevila Sep 14, 2023
76ab208
Merge branch 'main' into fix-flakiness-on-custom-time-range-saved-sea…
gsoldevila Sep 14, 2023
5b033d2
Misc enhancements
gsoldevila Sep 14, 2023
3fc0ccc
Use a retry block to make logic resilient to lost clicks
gsoldevila Sep 21, 2023
5bf114e
Merge branch 'main' into fix-flakiness-on-custom-time-range-saved-sea…
gsoldevila Sep 21, 2023
28132c7
Rely on #166239 for handling the opening of the super date picker
gsoldevila Sep 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 48 additions & 7 deletions test/functional/services/dashboard/panel_settings.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
import { FtrProviderContext } from '../../ftr_provider_context';
import { CommonlyUsed } from '../../page_objects/time_picker';

export function DashboardCustomizePanelProvider({ getService }: FtrProviderContext) {
export function DashboardCustomizePanelProvider({ getService, getPageObject }: FtrProviderContext) {
const log = getService('log');
const retry = getService('retry');
const toasts = getService('toasts');
Expand Down Expand Up @@ -55,10 +55,30 @@ export function DashboardCustomizePanelProvider({ getService }: FtrProviderConte
return await this.findFlyoutTestSubject('superDatePickerToggleQuickMenuButton');
}

public async clickToggleQuickMenuButton() {
log.debug('clickToggleQuickMenuButton');
public async openDatePickerQuickMenu() {
log.debug('openDatePickerQuickMenu');
const button = await this.findToggleQuickMenuButton();
await button.click();
await retry.try(async () => {
if (!(await button.isSelected())) {
await button.click();
}
if (!(await button.isSelected())) {
throw new Error('openDatePickerQuickMenu click missed');
}
});
}

public async closeDatePickerQuickMenu() {
log.debug('closeDatePickerQuickMenu');
const button = await this.findToggleQuickMenuButton();
await retry.try(async () => {
if (await button.isSelected()) {
await button.click();
}
if (await button.isSelected()) {
throw new Error('closeDatePickerQuickMenu click missed');
}
});
}

public async clickCommonlyUsedTimeRange(time: CommonlyUsed) {
Expand Down Expand Up @@ -112,9 +132,30 @@ export function DashboardCustomizePanelProvider({ getService }: FtrProviderConte
});
}

public async clickToggleShowCustomTimeRange() {
log.debug('clickToggleShowCustomTimeRange');
await testSubjects.click(this.TOGGLE_TIME_RANGE_TEST_SUBJ);
public async enableCustomTimeRange() {
Copy link
Contributor

@nreese nreese Sep 6, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great. One more thing, since it sounds like missing clicks is one cause of flakyness, how about adding a retry

const toggle = await testSubjects.find(this.TOGGLE_TIME_RANGE_TEST_SUBJ);
await retry.try(async () => {
  if (!(await toggle.isSelected())) {
    await toggle.click();
    if (!(await toggle.isSelected())) {
      throw new Error('Switch click missed');
    }
  }
});

log.debug('enableCustomTimeRange');
const toggle = await testSubjects.find(this.TOGGLE_TIME_RANGE_TEST_SUBJ);
await retry.try(async () => {
if (!(await toggle.isSelected())) {
await toggle.click();
if (!(await toggle.isSelected())) {
throw new Error('enableCustomTimeRange click missed');
}
}
});
}

public async disableCustomTimeRange() {
log.debug('disableCustomTimeRange');
const toggle = await testSubjects.find(this.TOGGLE_TIME_RANGE_TEST_SUBJ);
await retry.try(async () => {
if (await toggle.isSelected()) {
await toggle.click();
if (await toggle.isSelected()) {
throw new Error('disableCustomTimeRange click missed');
}
}
});
}
})();
}
12 changes: 6 additions & 6 deletions x-pack/test/functional/apps/dashboard/group2/panel_time_range.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
it('can add a custom time range to a panel', async () => {
await PageObjects.lens.createAndAddLensFromDashboard({});
await dashboardPanelActions.customizePanel();
await dashboardCustomizePanel.clickToggleShowCustomTimeRange();
await dashboardCustomizePanel.clickToggleQuickMenuButton();
await dashboardCustomizePanel.enableCustomTimeRange();
await dashboardCustomizePanel.openDatePickerQuickMenu();
await dashboardCustomizePanel.clickCommonlyUsedTimeRange('Last_30 days');
await dashboardCustomizePanel.clickSaveButton();
await PageObjects.dashboard.waitForRenderComplete();
Expand All @@ -56,7 +56,7 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {

it('can remove a custom time range from a panel', async () => {
await dashboardBadgeActions.clickTimeRangeBadgeAction();
await dashboardCustomizePanel.clickToggleShowCustomTimeRange();
await dashboardCustomizePanel.disableCustomTimeRange();
await dashboardCustomizePanel.clickSaveButton();
await PageObjects.dashboard.waitForRenderComplete();
await dashboardBadgeActions.expectMissingTimeRangeBadgeAction();
Expand All @@ -68,8 +68,8 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
it('can add a custom time range to panel', async () => {
await dashboardPanelActions.saveToLibrary('My by reference visualization');
await dashboardPanelActions.customizePanel();
await dashboardCustomizePanel.clickToggleShowCustomTimeRange();
await dashboardCustomizePanel.clickToggleQuickMenuButton();
await dashboardCustomizePanel.enableCustomTimeRange();
await dashboardCustomizePanel.openDatePickerQuickMenu();
await dashboardCustomizePanel.clickCommonlyUsedTimeRange('Last_30 days');
await dashboardCustomizePanel.clickSaveButton();
await PageObjects.dashboard.waitForRenderComplete();
Expand All @@ -80,7 +80,7 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {

it('can remove a custom time range from a panel', async () => {
await dashboardBadgeActions.clickTimeRangeBadgeAction();
await dashboardCustomizePanel.clickToggleShowCustomTimeRange();
await dashboardCustomizePanel.disableCustomTimeRange();
await dashboardCustomizePanel.clickSaveButton();
await PageObjects.dashboard.waitForRenderComplete();
await dashboardBadgeActions.expectMissingTimeRangeBadgeAction();
Expand Down
6 changes: 3 additions & 3 deletions x-pack/test/functional/apps/dashboard/group3/drilldowns/explore_data_panel_action.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
await dashboard.gotoDashboardEditMode(drilldowns.DASHBOARD_WITH_PIE_CHART_NAME);

await panelActions.customizePanel();
await dashboardCustomizePanel.clickToggleShowCustomTimeRange();
await dashboardCustomizePanel.disableCustomTimeRange();
await dashboardCustomizePanel.clickSaveButton();
await dashboard.saveDashboard('Dashboard with Pie Chart');
});
Expand Down Expand Up @@ -80,8 +80,8 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
await dashboard.gotoDashboardEditMode(drilldowns.DASHBOARD_WITH_PIE_CHART_NAME);

await panelActions.customizePanel();
await dashboardCustomizePanel.clickToggleShowCustomTimeRange();
await dashboardCustomizePanel.clickToggleQuickMenuButton();
await dashboardCustomizePanel.enableCustomTimeRange();
await dashboardCustomizePanel.openDatePickerQuickMenu();
await dashboardCustomizePanel.clickCommonlyUsedTimeRange('Last_90 days');
await dashboardCustomizePanel.clickSaveButton();

Expand Down
7 changes: 3 additions & 4 deletions x-pack/test/functional/apps/discover/saved_searches.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,7 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
await PageObjects.common.unsetTime();
});

// FLAKY: https://github.com/elastic/kibana/issues/104578
describe.skip('Customize time range', () => {
describe('Customize time range', () => {
it('should be possible to customize time range for saved searches on dashboards', async () => {
await PageObjects.common.navigateToApp('dashboard');
await PageObjects.dashboard.clickNewDashboard();
Expand All @@ -55,8 +54,8 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
expect(await dataGrid.getDocCount()).to.be(500);

await panelActions.customizePanel();
await dashboardCustomizePanel.clickToggleShowCustomTimeRange();
await dashboardCustomizePanel.clickToggleQuickMenuButton();
await dashboardCustomizePanel.enableCustomTimeRange();
await dashboardCustomizePanel.openDatePickerQuickMenu();
await dashboardCustomizePanel.clickCommonlyUsedTimeRange('Last_90 days');
await dashboardCustomizePanel.clickSaveButton();

Expand Down
8 changes: 4 additions & 4 deletions x-pack/test/functional/apps/lens/open_in_lens/tsvb/dashboard.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,8 @@ export default function ({ getPageObjects, getService }: FtrProviderContext) {
await dashboard.waitForRenderComplete();
const originalEmbeddableCount = await canvas.getEmbeddableCount();
await dashboardPanelActions.customizePanel();
await dashboardCustomizePanel.clickToggleShowCustomTimeRange();
await dashboardCustomizePanel.clickToggleQuickMenuButton();
await dashboardCustomizePanel.enableCustomTimeRange();
await dashboardCustomizePanel.openDatePickerQuickMenu();
await dashboardCustomizePanel.clickCommonlyUsedTimeRange('Last_30 days');
await dashboardCustomizePanel.clickSaveButton();
await dashboard.waitForRenderComplete();
Expand Down Expand Up @@ -80,8 +80,8 @@ export default function ({ getPageObjects, getService }: FtrProviderContext) {
await dashboard.waitForRenderComplete();
const originalEmbeddableCount = await canvas.getEmbeddableCount();
await dashboardPanelActions.customizePanel();
await dashboardCustomizePanel.clickToggleShowCustomTimeRange();
await dashboardCustomizePanel.clickToggleQuickMenuButton();
await dashboardCustomizePanel.enableCustomTimeRange();
await dashboardCustomizePanel.openDatePickerQuickMenu();
await dashboardCustomizePanel.clickCommonlyUsedTimeRange('Last_30 days');
await dashboardCustomizePanel.clickSaveButton();
await dashboard.waitForRenderComplete();
Expand Down
4 changes: 2 additions & 2 deletions x-pack/test/functional/apps/uptime/overview.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import { FtrProviderContext } from '../../ftr_provider_context';
export const UPTIME_HEARTBEAT_DATA = 'x-pack/test/functional/es_archives/uptime/full_heartbeat';

export default ({ getPageObjects, getService }: FtrProviderContext) => {
const { uptime } = getPageObjects(['uptime']);
const { uptime, common } = getPageObjects(['uptime', 'common']);
const retry = getService('retry');
const esArchiver = getService('esArchiver');

Expand Down Expand Up @@ -104,7 +104,7 @@ export default ({ getPageObjects, getService }: FtrProviderContext) => {
});
await uptime.setMonitorListPageSize(50);
// the pagination parameter should be cleared after a size change
await new Promise((resolve) => setTimeout(resolve, 1000));
await common.sleep(1000);
await retry.try(async () => {
await uptime.pageUrlContains('pagination', false);
});
Expand Down
7 changes: 3 additions & 4 deletions x-pack/test/functional/page_objects/navigational_search.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,10 @@ interface SearchResult {
label: string;
}

const delay = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

export class NavigationalSearchPageObject extends FtrService {
private readonly find = this.ctx.getService('find');
private readonly testSubjects = this.ctx.getService('testSubjects');
private readonly common = this.ctx.getPageObject('common');

async focus() {
const field = await this.testSubjects.find('nav-search-input');
Expand Down Expand Up @@ -69,7 +68,7 @@ export class NavigationalSearchPageObject extends FtrService {
// without heavy flakiness in this situation.
// there is NO ui indication of any kind to detect when all the emissions are done,
// so we are forced to fallback to awaiting a given amount of time once the first options are displayed.
await delay(waitUntil);
await this.common.sleep(waitUntil);
}

async getDisplayedResults() {
Expand All @@ -79,7 +78,7 @@ export class NavigationalSearchPageObject extends FtrService {

async isNoResultsPlaceholderDisplayed(checkAfter: number = 3000) {
// see comment in `waitForResultsLoaded`
await delay(checkAfter);
await this.common.sleep(checkAfter);
return this.testSubjects.exists('nav-search-no-results');
}

Expand Down
2 changes: 1 addition & 1 deletion x-pack/test/functional/services/cases/single_case_view.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ export function CasesSingleViewServiceProvider({ getService, getPageObject }: Ft
'[data-test-subj="euiMarkdownEditorToolbarButton"][aria-label="Visualization"]'
);
await addVisualizationButton.moveMouseTo();
await new Promise((resolve) => setTimeout(resolve, 500)); // give tooltip time to open
await common.sleep(500); // give tooltip time to open
},

async assertCaseTitle(expectedTitle: string) {
Expand Down
7 changes: 3 additions & 4 deletions x-pack/test/functional/services/data_stream.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,14 @@
import type { MappingProperty } from '@elastic/elasticsearch/lib/api/types';
import type { FtrProviderContext } from '../ftr_provider_context';

const waitFor = (time: number = 1000) => new Promise((r) => setTimeout(r, time));

/**
* High level interface to operate with Elasticsearch data stream and TSDS.
*/
export function DataStreamProvider({ getService }: FtrProviderContext) {
export function DataStreamProvider({ getService, getPageObject }: FtrProviderContext) {
const es = getService('es');
const log = getService('log');
const retry = getService('retry');
const common = getPageObject('common');

const downsampleDefaultOptions = {
isStream: true,
Expand Down Expand Up @@ -65,7 +64,7 @@ export function DataStreamProvider({ getService }: FtrProviderContext) {
waitTime / 1000
}s before running the downsampling to avoid a null_pointer_exception`
);
await waitFor(waitTime);
await common.sleep(waitTime);

try {
log.info(`downsampling "${sourceIndex}" index...`);
Expand Down
10 changes: 7 additions & 3 deletions x-pack/test/functional/services/infra_source_configuration_form.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,14 @@
import { FtrProviderContext } from '../ftr_provider_context';
import { WebElementWrapper } from '../../../../test/functional/services/lib/web_element_wrapper';

export function InfraSourceConfigurationFormProvider({ getService }: FtrProviderContext) {
export function InfraSourceConfigurationFormProvider({
getService,
getPageObject,
}: FtrProviderContext) {
const retry = getService('retry');
const testSubjects = getService('testSubjects');
const browser = getService('browser');
const common = getPageObject('common');

return {
/**
Expand Down Expand Up @@ -94,15 +98,15 @@ export function InfraSourceConfigurationFormProvider({ getService }: FtrProvider
const movementDifference = destinationIndex - sourceIndex;
await moveLogColumnHandle.pressKeys(browser.keys.SPACE);
for (let i = 0; i < Math.abs(movementDifference); i++) {
await new Promise((res) => setTimeout(res, KEY_PRESS_DELAY_MS));
await common.sleep(KEY_PRESS_DELAY_MS);
if (movementDifference > 0) {
await moveLogColumnHandle.pressKeys(browser.keys.ARROW_DOWN);
} else {
await moveLogColumnHandle.pressKeys(browser.keys.ARROW_UP);
}
}
await moveLogColumnHandle.pressKeys(browser.keys.SPACE);
await new Promise((res) => setTimeout(res, KEY_PRESS_DELAY_MS));
await common.sleep(KEY_PRESS_DELAY_MS);
},

/**
Expand Down
2 changes: 1 addition & 1 deletion x-pack/test/functional/services/uptime/navigation.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ export function UptimeNavigationProvider({ getService, getPageObjects }: FtrProv
return {
async refreshApp() {
await browser.refresh();
await new Promise((resolve) => setTimeout(resolve, 1000));
await PageObjects.common.sleep(1000);
await PageObjects.header.waitUntilLoadingHasFinished();
},

Expand Down