Switch to new workflows #909

uncleDecart · 2023-10-20T10:29:08Z

Following work done to split eden test workflows in #863 and integration
of this workflows in EVE repository, this commit switches to use new
workflows in eden itself

Also, new workflows are running on Buildjet runners, however, they
are not available in personal forks and it is useful to run test.yml
in fork, that is why this commit introduces determine-runner job,
which checks if we are in fork and changes runner to ubuntu-2204 which
is available everywhere

uncleDecart · 2023-10-20T10:32:32Z

I did not test it yet in my fork and would be great if @yash-zededa can check workflow files as well

milan-zededa · 2023-10-20T10:39:53Z

.github/workflows/test.yml

+    steps:
+      - id: fork-check
+        run: |
+          if ["${{ github.event.repository.full_name}}" == "lf-edge/eve" || "${{ github.event.repository.full_name}}" == "lf-edge/eden"]; then


Would it be possible to extend this check down to action/setup-environment and enable eve.accel if workflow is running on buildjet?

I thought we could actually change setup-environment and use some utilities to determine if hw acceleration is possible, that way we don't have coupling with runner, but rather with capability

Something like this: https://manpages.ubuntu.com/manpages/impish/man1/kvm-ok.1.html ?

Yes, I also found option with lscpu :), sending it rn

I am not sure that current logic works. I can see SandyBridge here, but it should be host in case of acceleration enabled.

giggsoff · 2023-10-20T11:41:25Z

.github/workflows/eden.yml

-              ${{ github.workspace }}/adam.log
+  test_suite_pr:
+    if: github.event.review.state == 'approved'
+    uses: ./eden/.github/workflows/test.yml@master


Should we split the PR into two if you would like to modify the file you point onto here, but it doesn't exists in master branch?

this file exists in master, only change here is runners for forks will be ubuntu :D

Actually I cannot understand, why we point onto branch here. I want to test my changes in eden with the workflow. But what we will check in that case? Main branch or my changes?

syntax is GHA is tricky, my intention was to use local action, fixed to use just that. GHA seems to work, I'll wait for it to finish.

It looks like another problem. It is really tricky

That's just my poor bash-ninja skills :D

Probably

[[ "${{ github.event.repository.full_name }}" == "lf-edge/eve" ]] || [[ "${{ github.event.repository.full_name }}" == "lf-edge/eden" ]]

With spaces around braces ;)

Seems to be working

I think it'll make sense to include similar thing to eden_setup to speed up process in eden repository

.github/workflows/eden.yml

uncleDecart · 2023-10-20T12:04:04Z

Something weird is happening with GHA I see error which I shouldn't see here will retry in some time

.github/workflows/test.yml

uncleDecart · 2023-10-20T14:25:22Z

onborading seems to fail. Checked locally with master, works

giggsoff · 2023-10-20T14:40:14Z

I can see level=fatal msg="Setup eden failed: cannot setup EVE: tag not present". So it looks like a problem with the check here. But it looks solid...

Update: Can we check something like:
if: ${{ inputs.eve_image != '' && contains(inputs.eve_image, ":" ) }}?

uncleDecart · 2023-10-23T12:16:54Z

I think you're right @giggsoff, let me check your changes

uncleDecart · 2023-10-23T12:47:56Z

Trying to make it work https://media.giphy.com/media/OVwfSuQXVUUI8/giphy.gif

uncleDecart · 2023-10-23T13:14:12Z

Now it's working, but we need to check if it works with EVE

uncleDecart · 2023-10-23T13:19:15Z

I think that we can merge it, try to run manually with eve_image on master branch and if it doesn't work, we do more fixes before, releasing eden and switching on EVE. Any objections @giggsoff @yash-zededa ?

uncleDecart · 2023-11-07T10:01:21Z

So it gets curiouser and curiouser. I found this

actions/checkout#1418

Let's try to checkout to 3.5.3

Following on lf-edge#910 we saw that EVE checks out Eden master code and not code from tag from yml file and if master fails tests are failing on EVE, which is undesirable https://github.com/lf-edge/eve/actions/runs/6613253197/job/17960730583?pr=3516#step:2:474 Investigating it more turns out in version 3.5.3 of github checkout action behaviour is what we need. Check lf-edge#909 for more info Signed-off-by: Pavel Abramov <uncle.decart@gmail.com>

.github/workflows/test.yml

Following on lf-edge#910 we saw that EVE checks out Eden master code and not code from tag from yml file and if master fails tests are failing on EVE, which is undesirable https://github.com/lf-edge/eve/actions/runs/6613253197/job/17960730583?pr=3516#step:2:474 Investigating it more turns out in version 3.5.3 of github checkout action behaviour is what we need. Check lf-edge#909 for more info Signed-off-by: Pavel Abramov <uncle.decart@gmail.com>

giggsoff · 2023-11-07T11:28:27Z

.github/actions/setup-environment/action.yml

@@ -34,7 +34,7 @@ runs:
    - name: Install Packages
      run: |
        sudo add-apt-repository ppa:stefanberger/swtpm-jammy
-        sudo apt install -y qemu-utils qemu-system-x86 jq swtpm
+        sudo apt install -y qemu-utils qemu-system-x86 jq swtpm lscpu


Seems lscpu comes from util-linux package.

Following on lf-edge#910 we saw that EVE checks out Eden master code and not code from tag from yml file and if master fails tests are failing on EVE, which is undesirable https://github.com/lf-edge/eve/actions/runs/6613253197/job/17960730583?pr=3516#step:2:474 Investigating it more turns out in version 3.5.3 of github checkout action behaviour is what we need. Check lf-edge#909 for more info Signed-off-by: Pavel Abramov <uncle.decart@gmail.com>

Following on #910 we saw that EVE checks out Eden master code and not code from tag from yml file and if master fails tests are failing on EVE, which is undesirable https://github.com/lf-edge/eve/actions/runs/6613253197/job/17960730583?pr=3516#step:2:474 Investigating it more turns out in version 3.5.3 of github checkout action behaviour is what we need. Check #909 for more info Signed-off-by: Pavel Abramov <uncle.decart@gmail.com>

giggsoff · 2023-11-08T13:05:18Z

Please rebase on top of master with #924 merged

giggsoff · 2023-11-08T14:36:56Z

Well, I can see host cpu in EVE-OS start line. So we use acceleration. Let's wait for the rest of the test. But seems we should workaround cloud-init test (revert changes or stabilize and adjust EVE-OS hash).

Following work done to split eden test workflows in lf-edge#863 and integration of this workflows in EVE repository, this commit switches to use new workflows in eden itself Signed-off-by: Pavel Abramov <uncle.decart@gmail.com>

New workflows are running on Buildjet runners, however, they are not available in personal forks and it is useful to run test.yml in fork, that is why this commit introduces determine-runner job, which checks if we are in fork and changes runner to ubuntu-2204 which is available everywhere Signed-off-by: Pavel Abramov <uncle.decart@gmail.com>

Signed-off-by: Pavel Abramov <uncle.decart@gmail.com>

This closes lf-edge#902 Signed-off-by: Pavel Abramov <uncle.decart@gmail.com>

giggsoff · 2023-11-09T19:15:09Z

New day, new problems: I can see The self-hosted runner lost communication with the server again and again https://github.com/lf-edge/eden/actions/runs/6811998752?pr=909

uncleDecart · 2023-11-10T08:19:29Z

I'll ask BuildJet about it

uncleDecart · 2023-11-22T08:14:39Z

@giggsoff buildjet did not respond, but I see that workflows are running, any objections to merge it?

Following on lf-edge#910 we saw that EVE checks out Eden master code and not code from tag from yml file and if master fails tests are failing on EVE, which is undesirable https://github.com/lf-edge/eve/actions/runs/6613253197/job/17960730583?pr=3516#step:2:474 Investigating it more turns out in version 3.5.3 of github checkout action behaviour is what we need. Check lf-edge#909 for more info Signed-off-by: Pavel Abramov <uncle.decart@gmail.com>

Following on #910 we saw that EVE checks out Eden master code and not code from tag from yml file and if master fails tests are failing on EVE, which is undesirable https://github.com/lf-edge/eve/actions/runs/6613253197/job/17960730583?pr=3516#step:2:474 Investigating it more turns out in version 3.5.3 of github checkout action behaviour is what we need. Check #909 for more info Signed-off-by: Pavel Abramov <uncle.decart@gmail.com>

uncleDecart requested a review from giggsoff October 20, 2023 10:29

uncleDecart force-pushed the switch-to-new-workflows branch from 5d22b03 to 394f61c Compare October 20, 2023 10:31

milan-zededa reviewed Oct 20, 2023

View reviewed changes

uncleDecart force-pushed the switch-to-new-workflows branch 2 times, most recently from 39f3909 to 75bd362 Compare October 20, 2023 11:22

giggsoff reviewed Oct 20, 2023

View reviewed changes

.github/workflows/eden.yml Outdated Show resolved Hide resolved

uncleDecart force-pushed the switch-to-new-workflows branch from 75bd362 to 80c3c31 Compare October 20, 2023 12:01

uncleDecart force-pushed the switch-to-new-workflows branch 4 times, most recently from d488b5a to 07c3f93 Compare October 20, 2023 12:36

giggsoff reviewed Oct 20, 2023

View reviewed changes

.github/workflows/test.yml Outdated Show resolved Hide resolved

uncleDecart force-pushed the switch-to-new-workflows branch 7 times, most recently from 25f3184 to 258ca47 Compare October 20, 2023 14:08

uncleDecart force-pushed the switch-to-new-workflows branch from 0653840 to 7795d2b Compare October 23, 2023 12:46

giggsoff mentioned this pull request Oct 23, 2023

Remove zombie code #911

Merged

uncleDecart force-pushed the switch-to-new-workflows branch from a87dee5 to f708402 Compare November 7, 2023 10:01

giggsoff reviewed Nov 7, 2023

View reviewed changes

.github/workflows/test.yml Outdated Show resolved Hide resolved

uncleDecart force-pushed the switch-to-new-workflows branch from f708402 to 50eac3a Compare November 7, 2023 10:18

giggsoff reviewed Nov 7, 2023

View reviewed changes

uncleDecart force-pushed the switch-to-new-workflows branch from 50eac3a to ef2c49f Compare November 8, 2023 08:32

uncleDecart force-pushed the switch-to-new-workflows branch from ef2c49f to d51627c Compare November 8, 2023 08:38

uncleDecart mentioned this pull request Nov 8, 2023

Use checkout@3.5.3 version to pull correct code tag, not master #924

Merged

uncleDecart force-pushed the switch-to-new-workflows branch from d51627c to 8a3abbd Compare November 8, 2023 13:20

uncleDecart added 5 commits November 9, 2023 13:47

Use new workflows in eden.yml

7b58a62

Following work done to split eden test workflows in lf-edge#863 and integration of this workflows in EVE repository, this commit switches to use new workflows in eden itself Signed-off-by: Pavel Abramov <uncle.decart@gmail.com>

GitHub Actions: fix set up eve version step to work with no eve_img

3e823fa

Signed-off-by: Pavel Abramov <uncle.decart@gmail.com>

GitHub Actions handle accelaration based on runner capabilities

1682f5e

Signed-off-by: Pavel Abramov <uncle.decart@gmail.com>

Run VNC test without hardware accelaration

f4bbf15

This closes lf-edge#902 Signed-off-by: Pavel Abramov <uncle.decart@gmail.com>

uncleDecart force-pushed the switch-to-new-workflows branch from 8a3abbd to f4bbf15 Compare November 9, 2023 12:48

giggsoff approved these changes Nov 22, 2023

View reviewed changes

milan-zededa approved these changes Nov 24, 2023

View reviewed changes

uncleDecart merged commit 87f7b8a into lf-edge:master Nov 24, 2023
16 of 20 checks passed

europaul mentioned this pull request Feb 7, 2024

Use checkout@3.5.3 version to pull correct code tag, not master #957

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch to new workflows #909

Switch to new workflows #909

uncleDecart commented Oct 20, 2023 •

edited

Loading

uncleDecart commented Oct 20, 2023

milan-zededa Oct 20, 2023

uncleDecart Oct 20, 2023

milan-zededa Oct 23, 2023

uncleDecart Oct 23, 2023

giggsoff Oct 25, 2023

giggsoff Oct 20, 2023

uncleDecart Oct 20, 2023

giggsoff Oct 20, 2023

uncleDecart Oct 20, 2023

giggsoff Oct 20, 2023

uncleDecart Oct 20, 2023

giggsoff Oct 20, 2023 •

edited

Loading

giggsoff Oct 20, 2023

uncleDecart Oct 20, 2023

uncleDecart Oct 20, 2023

uncleDecart commented Oct 20, 2023

uncleDecart commented Oct 20, 2023

giggsoff commented Oct 20, 2023 •

edited

Loading

uncleDecart commented Oct 23, 2023

uncleDecart commented Oct 23, 2023

uncleDecart commented Oct 23, 2023 •

edited

Loading

uncleDecart commented Oct 23, 2023

uncleDecart commented Nov 7, 2023

giggsoff Nov 7, 2023

giggsoff commented Nov 8, 2023

giggsoff commented Nov 8, 2023

giggsoff commented Nov 9, 2023

uncleDecart commented Nov 10, 2023

uncleDecart commented Nov 22, 2023

Switch to new workflows #909

Switch to new workflows #909

Conversation

uncleDecart commented Oct 20, 2023 • edited Loading

uncleDecart commented Oct 20, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

giggsoff Oct 20, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

uncleDecart commented Oct 20, 2023

uncleDecart commented Oct 20, 2023

giggsoff commented Oct 20, 2023 • edited Loading

uncleDecart commented Oct 23, 2023

uncleDecart commented Oct 23, 2023

uncleDecart commented Oct 23, 2023 • edited Loading

uncleDecart commented Oct 23, 2023

uncleDecart commented Nov 7, 2023

Choose a reason for hiding this comment

giggsoff commented Nov 8, 2023

giggsoff commented Nov 8, 2023

giggsoff commented Nov 9, 2023

uncleDecart commented Nov 10, 2023

uncleDecart commented Nov 22, 2023

uncleDecart commented Oct 20, 2023 •

edited

Loading

giggsoff Oct 20, 2023 •

edited

Loading

giggsoff commented Oct 20, 2023 •

edited

Loading

uncleDecart commented Oct 23, 2023 •

edited

Loading