Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-actions: Extend action to run Frontend Unit tests #1998

Conversation

orfeas-k
Copy link
Contributor

@orfeas-k orfeas-k commented Nov 3, 2022

This is a follow up PR to this #1977. Here, we extend the GH action Frontend Test to include a job that runs KWA frontend unit tests as well. This also is needed before this PR #1991 (comment) is merged to ensure that nothing breaks.

@orfeas-k orfeas-k changed the title gh-actions: Extend action to run Frontend Unit tests gh-actions: Extend action to run Frontend Unit tests (WIP) Nov 3, 2022
@orfeas-k orfeas-k changed the title gh-actions: Extend action to run Frontend Unit tests (WIP) WIP - gh-actions: Extend action to run Frontend Unit tests Nov 3, 2022
Copy link
Member

@tenzen-y tenzen-y left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@orfeas-k Thanks for creating this!
I left a few comments.

/assign @kimwnasptd

.github/workflows/test-node.yaml Outdated Show resolved Hide resolved
.github/workflows/test-node.yaml Outdated Show resolved Hide resolved
@orfeas-k orfeas-k force-pushed the feature-orfeas-k-frontend-unit-tests-gh-action branch 2 times, most recently from 698d600 to 0e90f57 Compare November 3, 2022 13:17
@coveralls
Copy link

coveralls commented Nov 3, 2022

Coverage Status

Coverage decreased (-0.1%) to 73.419% when pulling d358963 on arrikto:feature-orfeas-k-frontend-unit-tests-gh-action into b1ed847 on kubeflow:master.

@orfeas-k orfeas-k force-pushed the feature-orfeas-k-frontend-unit-tests-gh-action branch from 0e90f57 to 240e641 Compare November 4, 2022 08:18
@orfeas-k
Copy link
Contributor Author

orfeas-k commented Nov 7, 2022

While working on this PR, I noticed that all jobs were queued and ran and thus, had to wait for a long time in order to verify if something breaks. After some discussion with @kimwnasptd , we thought it would be preferable if we could eliminate and run the jobs that are specific to each PR's changes. For that, we could utilise https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#example-excluding-paths and exclude the frontend path in jobs that are not related to it.

WDYT? Which workflows do you think we should run when there's a change in the frontend? cc @johnugeorge @andreyvelich @tenzen-y

@johnugeorge
Copy link
Member

I think, for UI related changes, we need to run only https://github.com/kubeflow/katib/blob/master/.github/workflows/katib-ui-e2e-test.yaml and can skip all e2e tests.

@tenzen-y Any comments?

@kimwnasptd
Copy link
Member

@johnugeorge @tenzen-y in this PR we also changed the .github/workflows/test-node.yaml for running the unit tests, and would extend it in the future to also run the integration tests.

Should we include this action as well for running unit/integration tests or should we migrate everything for the UI into the action you mentioned above?
https://github.com/kubeflow/katib/blob/master/.github/workflows/katib-ui-e2e-test.yaml

Extend Frontend Test action to run also KWA frontend unit tests.

Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
@johnugeorge
Copy link
Member

@kimwnasptd If the integration tests requires images to be built, it is best to be added in https://github.com/kubeflow/katib/blob/master/.github/workflows/katib-ui-e2e-test.yaml as the workflow already does it for you.
If not, you can keep all tests in github/workflows/test-node.yaml itself.

@orfeas-k orfeas-k force-pushed the feature-orfeas-k-frontend-unit-tests-gh-action branch from 240e641 to e1170f9 Compare November 7, 2022 15:50
@tenzen-y
Copy link
Member

tenzen-y commented Nov 7, 2022

I think, for UI related changes, we need to run only https://github.com/kubeflow/katib/blob/master/.github/workflows/katib-ui-e2e-test.yaml and can skip all e2e tests.

@tenzen-y Any comments?

@johnugeorge
Thanks for reminding me.

@kimwnasptd If the integration tests requires images to be built, it is best to be added in https://github.com/kubeflow/katib/blob/master/.github/workflows/katib-ui-e2e-test.yaml as the workflow already does it for you.
If not, you can keep all tests in github/workflows/test-node.yaml itself.

I agree with you.

@kimwnasptd
I also think that we can skip any other e2e tests except katib-ui-e2e-test for only UI code changes.

@orfeas-k orfeas-k force-pushed the feature-orfeas-k-frontend-unit-tests-gh-action branch from e1170f9 to 61e3598 Compare November 8, 2022 10:25
@google-oss-prow google-oss-prow bot added size/L and removed size/M labels Nov 8, 2022
@orfeas-k orfeas-k force-pushed the feature-orfeas-k-frontend-unit-tests-gh-action branch from 61e3598 to 52528c6 Compare November 8, 2022 10:31
@orfeas-k
Copy link
Contributor Author

orfeas-k commented Nov 8, 2022

@kimwnasptd @johnugeorge @tenzen-y I pushed another commit in order to exclude workflows from running when there are changes only in the frontend, just as we discussed here. I'm also removing the WIP - also from this PR.

@orfeas-k orfeas-k changed the title WIP - gh-actions: Extend action to run Frontend Unit tests gh-actions: Extend action to run Frontend Unit tests Nov 8, 2022
.github/workflows/lint.yaml Outdated Show resolved Hide resolved
.github/workflows/publish-core-images.yaml Show resolved Hide resolved
Prevent the following workflows when a PR contains changes that affect
only the frontend:
 - Charmed Katib
 - E2E Test with darts-cnn-cifar10
 - E2E Test with enas-cnn-cifar10
 - E2E Test with mxnet-mnist
 - E2E Test with pytorch-mnist
 - E2E Test with simple-pbt
 - E2E Test with tf-mnist-with-summaries
 - Go Test
 - Publish AutoML Algorithm Images
 - Publish Katib Core Images
 - Publish Trial Images
 - Python Test
 - Shellcheck

Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
@orfeas-k orfeas-k force-pushed the feature-orfeas-k-frontend-unit-tests-gh-action branch from 52528c6 to 0ee57ce Compare November 10, 2022 09:10
@kimwnasptd
Copy link
Member

Thanks @orfeas-k. LGTM from my side. I think we'll only need to re-run one action.

/lgtm

@google-oss-prow google-oss-prow bot added the lgtm label Nov 10, 2022
@johnugeorge
Copy link
Member

/assign @tenzen-y

Signed-off-by: Orfeas Kourkakis <orfeas@arrikto.com>
@orfeas-k orfeas-k force-pushed the feature-orfeas-k-frontend-unit-tests-gh-action branch from 0ee57ce to d358963 Compare November 11, 2022 07:50
@google-oss-prow google-oss-prow bot removed the lgtm label Nov 11, 2022
Copy link
Member

@tenzen-y tenzen-y left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@orfeas-k Thanks for updating!
/lgtm

/assign @kubeflow/wg-automl-leads

@google-oss-prow google-oss-prow bot added the lgtm label Nov 11, 2022
@johnugeorge
Copy link
Member

/approve

@google-oss-prow
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: johnugeorge, orfeas-k

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-prow google-oss-prow bot merged commit 0ee6062 into kubeflow:master Nov 11, 2022
@kimwnasptd kimwnasptd deleted the feature-orfeas-k-frontend-unit-tests-gh-action branch November 11, 2022 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants