Investigate disabling unnecessary services on hosted runner

[varunsh-coder]

Description

step-security/harden-runner is a GitHub Action that correlates and blocks outbound traffic from GitHub workflows. In some workflows, traffic to cdn.fwupd.org was observed which was not from the workflow itself. As an example, in this workflow run, traffic to cdn.fwupd.org was detected and blocked.

This call is made by fwupdmgr. This is a system daemon to allow session software to update firmware. I believe this is not really used, and just increases attack surface.

Similarly, in some workflows, traffic to motd.ubuntu.com is observed. This is to fetch message of the day.

Please investigate disabling/ removing unnecessary services on hosted runners, to reduce attack surface and risk of software supply chain security issues.

Virtual environments affected

• Ubuntu 18.04
• Ubuntu 20.04
• macOS 10.15
• macOS 11
• Windows Server 2016
• Windows Server 2019
• Windows Server 2022

Image version and build link

https://github.com/nvm-sh/nvm/actions/runs/1614399726

https://github.com/harden-runner-canary/kyverno/actions/runs/1685857420

Is it regression?

No response

Expected behavior

There should be no unexpected outbound calls from unnecessary services on hosted runner.

Actual behavior

In some workflows, outbound calls are made to cdn.fwupd.org and motd.ubuntu.com that are not expected.

Repro steps

You can use https://github.com/step-security/harden-runner to correlate and block traffic on ubuntu hosted runner. The unexpected outbound calls from unnecessary services are not made on every run, so cannot be reproed every time, but can be seen off and on.

[al-cheb]

Hey, @varunsh-coder
We will take a look at it.

[al-cheb]

@varunsh-coder , I have created a #4873 PR to disable motd network activity. It's definitely not a security issue, because those services pre-installed and enabled by default.

[varunsh-coder]

Thanks @al-cheb for the PR!

I see that motd is being disabled in the PR, but I do not see changes to disable fwupd. As I explained in the issue, this service makes outbound calls to cdn.fwupd.org. There were two sources of outbound calls - 1) motd and 2) fwupd.

Is fwupd needed on the hosted runner? If not, then it would be great to disable it as well. Here are some links about it:
https://askubuntu.com/questions/1227508/consequences-of-disabling-fwupd
https://michael.kjorling.se/blog/2021/turning-off-fwupdmgr-and-lvfs-automatic-updates-on-debian-11-bullseye/

I agree it is not an immediate security issue, but it is a best practice to disable services that are not needed, to reduce attack surface.

CC: @maxim-lobanov since you had opened an issue earlier to harden the hosted runner, you might want to weigh in on this issue.

[al-cheb]

@varunsh-coder, I have added a command to mask the fwupd service - systemctl mask fwupd-refresh.timer

[maxim-lobanov]

These proposals make sense to me!
@varunsh-coder , thank you for proposing.
@al-cheb thank you for implementing.

[varunsh-coder]

Hi @al-cheb please let me know if you want me to create a separate issue for this. The GitHub-hosted Ubuntu VM also makes calls to api.snapcraft.io (example workflow run). I believe this call is made by snapd. Can this be disabled too? Thanks!

[al-cheb]

api.snapcraft.io

@varunsh-coder, Thank you. No need to create a separate issue. I will check.

[miketimofeev]

@varunsh-coder please ignore this spammy user @freddy123098

[al-cheb]

@varunsh-coder , The new images with disabled motd updates have been deployed. We also have disabled snap auto refresher in scope of this #4768 PR. Looks like it's not enough to disable all calls to api.snapcraft.io , we can't disable snapd service totally, because it will affect a lot of customers.

[varunsh-coder]

Thanks a lot @al-cheb!

I am curious why disabling snapd will affect lot of customers? Are these VMs used for something other than hosted-runners? Trying to understand why customers would be depending on snapd on ephemeral hosted-runners.

[veikkoeeva]

It appears there are still/now active outbounds calls to cdn.fwupd.org. F.ex. at https://github.com/Lumoin/Verifiable/actions/runs/12468001138 is an example.