Intelij: 2024.2 Update Test Automation in Windows and Linux

[vaisakhkannan]

Update UI test automation on Windows to support IntelliJ 2024.2.

• SingleModJakartaLSTestCommon - [Done]
• SingleModLibertyLSTestCommon - [Done]
• SingleModMPLSTestCommon -[Done]
• SingleModMPProjectTestCommon -[Done]
• SingleModNLTRestProjectTestCommon - [Done]

[vaisakhkannan]

SingleModLibertyLSTestCommon

• prepareEnv & cleanup
• testServerXMLFeatureHover
• testServerXMLNonFeatureHover
• testInsertFeatureIntoServerXML
• testInsertLibertyConfigElementIntoServerXML
• testInsertLibertyConfigIntoServerEnv
• testInsertLibertyConfigIntoBootstrapProps
• testServerEnvCfgHover
• testBootstrapPropsCfgHover
• testDiagnosticInServerXML
• testQuickFixInServerXML
• testDiagnosticInServerEnv
• testDiagnosticInBootstrapProperties

[vaisakhkannan]

SingleModNLTRestProjectTestCommon

• testManualProjectAddRemoveActionUsingSearch
• testsRefreshProjectWithServerXmlOnly
• testsRefreshProjectWithLTBuildCfgOnlyWithBldScriptBlock
• testsRefreshProjectWithLTBuildCfgOnly

[vaisakhkannan]

SingleModJakartaLSTestCommon

• testInsertJakartaCodeSnippetIntoJavaPart
• testJakartaDiagnosticsInJavaPart
• testJakartaQuickFixInJavaPart

[vaisakhkannan]

SingleModMPLSTestCommon

• testInsertCodeSnippetIntoJavaPart
• testMPDiagnosticsInJavaPart
• testMPQuickFixInJavaFile
• testInsertMicroProfileProperty
• testMicroProfileConfigHover
• testDiagnosticInMicroProfileConfigProperties
• testQuickFixInMicroProfileConfigProperties

[vaisakhkannan]

SingleModMPProjectTestCommon

• testOpenBuildFileActionUsingPopUpMenu
• testStartWithParamsActionUsingDropDownMenu
• testStartWithParamsActionUsingPlayToolbarButton
• testStartWithParamsActionUsingPopUpMenu
• testStartWithParamsActionUsingSearch
• testRunTestsActionUsingDropDownMenu
• testRunTestsActionUsingPlayToolbarButton
• testRunTestsActionUsingPopUpMenu
• testRunTestsActionUsingSearch
• testStartWithConfigInDebugModeUsingToolbar
• testStartWithConfigInDebugModeUsingMenu
• testStartWithConfigInRunModeUsingToolbar
• testStartWithConfigInRunModeUsingMenu
• testMultipleConfigEditHistory
• testStartInContainerActionUsingDropDownMenu - @EnabledOnOs({OS.LINUX})
• testStartInContainerActionUsingPlayToolbarButton - @EnabledOnOs({OS.LINUX})
• testStartInContainerActionUsingPopUpMenu - @EnabledOnOs({OS.LINUX})
• testStartInContainerActionUsingSearch - @EnabledOnOs({OS.LINUX})
• testCustomStartParametersClearedOnConfigRemoval

[vaisakhkannan]

GHA Build Run : https://github.com/vaisakhkannan/liberty-tools-intellij/actions/runs/11723063340/job/32653929671

Found issue when try to access Run Menu

upload-run-issue.mov

[vaisakhkannan]

Found an issue in GHA in testOpenBuildFileActionUsingPopUpMenu test, Unable to close editor tab of build File

closetab-issue.mov

Solution :Instead of trying to click the close button (X) on the file tab, just right-click on the file tab name and choose the 'Close' menu item. This will close the file tab.

[vaisakhkannan]

@TrevCraw , I found a behavioral difference between Linux and Windows. In Linux, First robot clicks on the Main Menu(four lines), then the robot only needs to hover over the Action Menu after navigate to the corresponding Action Menu, while in Windows, I have written the code to click on the Action menu. This caused a failure in Linux, as it doesn't require a click on the action menu. To make it compatible with both systems, I modified the code to use moveMouse() instead of click(), which will work on both Linux and Windows.

[vaisakhkannan]

I found 2 common failures in GHA

First one is

MavenSingleModMPSIDProjectTest > initializationError FAILED
    java.lang.RuntimeException: Unable to expand the Liberty tool window project tree.
        at io.openliberty.tools.intellij.it.UIBotTestUtils.expandLibertyToolWindowProjectTree(UIBotTestUtils.java:653)
        at io.openliberty.tools.intellij.it.SingleModMPProjectTestCommon.prepareEnv(SingleModMPProjectTestCommon.java:922)
        at io.openliberty.tools.intellij.it.MavenSingleModMPSIDProjectTest.setup(MavenSingleModMPSIDProjectTest.java:102)

        Caused by:
        com.intellij.remoterobot.utils.WaitForConditionTimeoutException: Exceeded timeout (PT20S) for condition function (Failed to find 'ComponentFixture' by '//div[@class='LibertyExplorer']//div[@accessiblename='Expand All' and @class='ActionButton']' in 20s) 
            at com.intellij.remoterobot.utils.RepeatUtilsKt.waitFor(RepeatUtils.kt:51)
            at com.intellij.remoterobot.utils.RepeatUtilsKt.waitFor$default(RepeatUtils.kt:37)
            at com.intellij.remoterobot.SearchContext$find$1.invoke(SearchContext.kt:53)
            at com.intellij.remoterobot.SearchContext$find$1.invoke(SearchContext.kt:51)
            at com.intellij.remoterobot.stepsProcessing.StepWorkerKt.step(StepWorker.kt:23)
            at com.intellij.remoterobot.SearchContext$DefaultImpls.find(SearchContext.kt:51)
            at com.intellij.remoterobot.fixtures.ContainerFixture.find(ContainerFixture.kt:12)
            at io.openliberty.tools.intellij.it.fixtures.ProjectFrameFixture.getActionButton(ProjectFrameFixture.java:62)
            at io.openliberty.tools.intellij.it.UIBotTestUtils.expandLibertyToolWindowProjectTree(UIBotTestUtils.java:634)
            ... 2 more

GHA Run link : https://github.com/vaisakhkannan/liberty-tools-intellij/actions/runs/11790595086/job/32841292475

The MavenSingleModMPSIDProjectTest class inherits from SingleModMPProjectTestCommon, which has a total of four child classes. Only MavenSingleModMPSIDProjectTest is failing, so I am trying to add some delays to the MavenSingleModMPSIDProjectTest class code to make it work correctly.

Second one is ,

GradleSingleModMPProjectTest > initializationError FAILED
    com.intellij.remoterobot.utils.WaitForConditionTimeoutException: Exceeded timeout (PT10S) for condition function (Failed to find 'ComponentFixture' by '//div[@class='ActionMenu' and @text='Window']//div[@class='ActionMenu' and @text='Editor Tabs']' in 10s) 
        at com.intellij.remoterobot.utils.RepeatUtilsKt.waitFor(RepeatUtils.kt:51)
        at com.intellij.remoterobot.utils.RepeatUtilsKt.waitFor$default(RepeatUtils.kt:37)
        at com.intellij.remoterobot.SearchContext$find$1.invoke(SearchContext.kt:53)
        at com.intellij.remoterobot.SearchContext$find$1.invoke(SearchContext.kt:51)
        at com.intellij.remoterobot.stepsProcessing.StepWorkerKt.step(StepWorker.kt:23)
        at com.intellij.remoterobot.SearchContext$DefaultImpls.find(SearchContext.kt:51)
        at com.intellij.remoterobot.fixtures.ContainerFixture.find(ContainerFixture.kt:12)
        at io.openliberty.tools.intellij.it.fixtures.ProjectFrameFixture.getChildActionMenu(ProjectFrameFixture.java:106)
        at io.openliberty.tools.intellij.it.UIBotTestUtils.closeAllEditorTabs(UIBotTestUtils.java:807)
        at io.openliberty.tools.intellij.it.SingleModMPProjectTestCommon.prepareEnv(SingleModMPProjectTestCommon.java:931)
        at io.openliberty.tools.intellij.it.GradleSingleModMPProjectTest.setup(GradleSingleModMPProjectTest.java:83)

GHA Run link : https://github.com/vaisakhkannan/liberty-tools-intellij/actions/runs/11790595086/job/32841292151

[vaisakhkannan]

I resolved the above second issue successfully. For the first issue, I attempted various approaches, including adding delays and modifying the UI component class to access specific element. However, the issue persisted with the error:
MavenSingleModMPSIDProjectTest > initializationError FAILED
The MavenSingleModMPSIDProjectTest class inherits from SingleModMPProjectTestCommon, which has four child classes in total. Only MavenSingleModMPSIDProjectTest is failing. From the GHA results, I observed that MavenSingleModMPSIDProjectTest is executed after MavenSingleModMPProjectTest (another child class of the same parent class). To address this, I renamed the MavenSingleModMPProjectTest class to MavenSingleModMPTest. This change ensured that MavenSingleModMPSIDProjectTest executes first, followed by MavenSingleModMPTest. Fortunately, this resolved the initialization error for MavenSingleModMPSIDProjectTest.

You can review the linux GHA results below:
linux build 1
linux build 2
For Linux, the results show 120 tests completed, 16 failed, and 16 skipped. The 16 failures are unit test failures, while all other tests pass

You can review the Windows GHA results below:
windows build 1
windows build 2
the results were 120 tests completed, 17 failed, and 16 skipped. Of the 17 failures, 16 are unit test failures, and 1 is a known issue (testDiagnosticInServerEnv()).
Renaming MavenSingleModMPProjectTest to MavenSingleModMPTest consistently resolves the initialization error.

[vaisakhkannan]

I am attaching the video of above mentioned failure

setup_recording_2024_18_11_11_05_06.mov

As a workaround, I am thinking of refreshing the Liberty tool window before performing the Expand All action.

Refreshing the Liberty tool window did not resolve the problem. Upon investigating further, I noticed in the video that during the initialization phase, the project frame opens with the build file displayed. Additionally, when importing the project, I made a change to the code to maximize the IDE window, which caused a modification in the open build file in the editor. To address this, I updated the cleanup() method in the parent class SingleModMPProjectTestCommon to close all tabs, and this resolved the issue.

[vaisakhkannan]

Videos of some tests running in IntelliJ IDEA version 2024.2.

Linux test example :

setup_recording_2024_18_11_16_08_51.mov

Mac test example :

testRunTestsActionUsingPopUpMenu_recording_2024_19_11_05_51_17.mov

Windows test example :

setup_recording_2024_18_11_16_23_14.mov

[vaisakhkannan]

Windows test example based on the code in the LTI version 24.0.9:

testDiagnosticInServerEnv_recording_2024_18_11_10_08_45.mov

As part of comparing all three operating systems, I noticed that the Windows UI appears as if it is already in zoomed-in mode. I checked the current main UI test on Windows, and it also seems to be in a similar zoomed-in state. But, the IntelliJ 2024.2 UI is slightly wider in both the Project View panel and the Liberty Dashboard panel. This change appears to come automatically with the adoption of the 2024.2 UI. It looks like a problem with the Windows machines as well.

[vaisakhkannan]

As a resolution, we can decrease the zoom level using the View > Appearance option in Windows OS (I think windows is necessary only for reducing zoom based on the screen records). However, when I tried this, I found that only the font size is decreased. @turkeylurkey , is this what you suggested? We can't adjust to side ways to minimise the view (decreasing the width of the dashboards) for both the Project View and Liberty Dashboard views, but we can reduce the font size using the View > Appearance option. So can I try the same in windows os?

[turkeylurkey]

Does it really have to be a video?

[turkeylurkey]

I was talking about compact mode: https://www.jetbrains.com/help/idea/new-ui.html#compact-mode

[vaisakhkannan]

@turkeylurkey Can you try checking the video on browser chrome?

I was talking about compact mode: https://www.jetbrains.com/help/idea/new-ui.html#compact-mode

Ok,So you suggesting to do compact mode on , right?

[vaisakhkannan]

Compact Mode off Image:

Compact Mode on Image:

It looks like enabling Compact Mode reduced the size of the UI elements, so I made changes to accommodate that.

[vaisakhkannan]

I ran the GHA build today, with the updated UI changes along with Anusree's recent PR changes. Now, I can see that both unit and UI tests are passing on Linux, while on Windows, all unit tests are passing, but 2 UI tests failed.

Test summary in windows:

148 tests completed: 2 failed, 16 skipped

Failed UI tests:

GradleSingleModLSTest > testDiagnosticInServerEnv() - Failed (known issue)
GradleSingleModMPSIDProjectTest > testMultipleConfigEditHistory() - Failed (intermittent issue)

https://github.com/vaisakhkannan/liberty-tools-intellij/actions/runs/11951604250/attempts/1