test-vm-e2e doesn't work in CI pipeline

[wenyingd]

Describe the bug

vm-agent doesn't work in the CI e2e jobs. After checking on the Linux VM, we found that the service antrea-agent is not active, and these errors were got,

Starting antrea-agent service
● antrea-agent.service - "antrea-agent as a systemd service"
   Loaded: loaded (/etc/systemd/system/antrea-agent.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Wed 2023-05-24 16:19:33 UTC; 3ms ago
  Process: 26884 ExecStart=/usr/sbin/antrea-agent --config=/etc/antrea/antrea-agent.conf --logtostderr=false --log_file=/var/log/antrea/antrea-agent.log (code=exited, status=1/FAILURE)
  Process: 26866 ExecStartPre=/etc/antrea/install-vm.sh --validate-ovs --ovs-bridge br-int (code=exited, status=0/SUCCESS)
 Main PID: 26884 (code=exited, status=1/FAILURE)

May 24 16:19:33 vmbmtest0-1 systemd[1]: antrea-agent.service: Main process exited, code=exited, status=1/FAILURE
May 24 16:19:33 vmbmtest0-1 systemd[1]: antrea-agent.service: Failed with result 'exit-code'.
May 24 16:19:34 vmbmtest0-1 systemd[1]: antrea-agent.service: Service hold-off time over, scheduling restart.
May 24 16:19:34 vmbmtest0-1 systemd[1]: antrea-agent.service: Scheduled restart job, restart counter is at 1.
May 24 16:19:34 vmbmtest0-1 systemd[1]: Stopped "antrea-agent as a systemd service".
May 24 16:19:34 vmbmtest0-1 systemd[1]: Starting "antrea-agent as a systemd service"...
May 24 16:19:34 vmbmtest0-1 systemd[1]: Started "antrea-agent as a systemd service".
May 24 16:19:34 vmbmtest0-1 antrea-agent[26911]: /usr/sbin/antrea-agent: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /usr/sbin/antrea-agent)
May 24 16:19:34 vmbmtest0-1 antrea-agent[26911]: /usr/sbin/antrea-agent: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /usr/sbin/antrea-agent)
May 24 16:19:34 vmbmtest0-1 systemd[1]: antrea-agent.service: Main process exited, code=exited, status=1/FAILURE

After checking the setup, we found that the Linux VM to run antrea-agent is using Ubuntu 18.04

~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.6 LTS
Release:        18.04
Codename:       bionic

But antrea-agent bits is compiled based on Ubuntu docker image 22.04 (https://github.com/antrea-io/antrea/blob/main/ci/jenkins/test-vm.sh#L312), this caused that the antrea-agent's dependent library was lost on the actual VM in the runtime.

We need to resolve this issue by providing static link when compiling antrea-agent bits inside a container env.

To Reproduce

1. run make docker-bin in directory antrea on any Linux VM,
2. copy antrea/bin/antrea-agent to Ubuntu 18.04
3. configure the necessary configurations needed by antrea-agent
4. run antrea-agent bits

Expected

antrea-agent is supposed to run successfully.

Actual behavior

antrea-agent is crashed.

Versions:

Antrea main branch, it may exist for several releases.

Additional context

[prakrit55]

hey @wenyingd, did you tried apt update and apt upgrade after that ?? I think that info after apt upgrade wd be beneficial, to know further about the issue : )

[tnqn]

@prakrit55 The issue has explained the problem:

But antrea-agent bits is compiled based on Ubuntu docker image 22.04 (https://github.com/antrea-io/antrea/blob/main/ci/jenkins/test-vm.sh#L312), this caused that the antrea-agent's dependent library was lost on the actual VM in the runtime.

We need to resolve this issue by providing static link when compiling antrea-agent bits inside a container env.

It should be fixed by #5988, no further investigation is required.

[antoninbas]

Fixed by #5988 and #6008