xenu: XNU Emulating Non-Unfree

The goal of this project is to create a QEMU-like CLI interface for the Apple Virtualization framework. The original motivation was to have a nice way to run x86_64-linux binaries on Apple Silicon for binary exploitation challenges.

Run it

You must have Xcode 15+ setup (run nix shell nixpkgs#darwin.xcode_15_1 for instructions on how to install Xcode; I would also recommend creating a gc root for Xcode to prevent it from being gc’d). When newer versions of the MacOS SDK get added to nixpkgs you shouldn’t need to install Xcode, but for now we will accept this necessary limitation.

nix build .#xenu
nix build .#linux -o linux
mkfs.ext4 -L nixos nixos.raw; qemu-img create -f raw nixos.raw 5120M
./result/bin/xenu ./linux/system/kernel ./linux/system/initrd ./nixos.raw

Then on the guest:

let pkgs = import <nixpkgs> {}; in
pkgs.pkgsCross.gnu64.mkShell {
  packages = with pkgs; [ ];
  X86_GLIBC = pkgs.pkgsCross.gnu64.glibc; # dynamic linker for use with patchelf
  X86_GLIBC_BIN = pkgs.pkgsCross.gnu64.glibc.bin; # ldd
}

I’ll put up a asciinema when it’s more fleshed out

FAQ

Who is this for?

• You want to run an aarch64-linux OS and run x86_64-linux binaries
• You want a hackable QEMU-like interface to Apple Virtualisation
• You are a masochist Nix Darwin enthusiast

Do I need to use Nix to take advantage of this?

Not necessarily; although this is developed with Nix as the primary usecase, it’s meant to provide the same no-frills but feature rich interface that QEMU provides, but with Apple Virtualisation. If you don’t need Rosetta Linux, I highly recommend using QEMU with accel=hvf.

Why not QEMU with emulation?

It’s slow. Ideally we could put some benchmarks here…

I’m on x86_64-darwin.

Rosetta only works one-way; to my knowledge, there is no way to run aarch64 binaries on x86 Macs. You can virtualize an x86 VM with QEMU very easily using stock nixpkgs.

I need a GUI.

Use UTM. Although adding rudimentary GUI support is fairly straightforward, ironing out the kinks is not. UTM and Tart are significantly more mature and have larger communities to answer questions.

I want to use this to do pwn!

Unless you are stupid like me, you are better off (for the time being) just using a Docker container or running Debian using UTM (UTM instructions for using Rosetta Linux). Eventually, you should be able to just type nix run xenu and get NixOS with Rosetta, but that’s a while off.

How do I fix <mysterious kernel panic>?

¯\_(ツ)_/¯. You can try running a bleeding edge kernel or adding fresh off the press patches like so:

customNixosSystem = xenu.nixosConfigurations.linuxVM.extendModules {
  modules = [
    ({ config, ... }: {
      # see https://nixos.wiki/wiki/Linux_kernel for more exotic arrangements
      boot.kernelPackages = pkgs.linuxPackages_latest;
	  boot.kernelPatches = [
		(pkgs.fetchpatch {
		  url = "...";
		  hash = "sha256-...";
		})
	  ];
    })
  ];
};

Future Work

Use Nix to build xenu

MacOS guests

Network support

Ctrl-C should be sent to guest, not host process

Separate process group?

Make it easier to specify Debian/Ubuntu

An experienced Nixer can hack this together pretty quickly, but the goal is to make this a very easy tool to use, without compromising on extensibility.

Package binaries on Github

You currently need to have Xcode installed to build this… yuck! The developer of alt-tab-macos sums it up pretty well.

Automate patchelf workflow/improve guest workflow

Acknowledgements

• Apple Virtualization Docs
• nixos-on-darwin — Doesn’t appear active, but seems similar in spirit, if not scope
• @Surg-Dev for praying for my downfall

Thoughts from my rewrite

Xenu takes heavy inspiration from UTM, which is an absolutely fantastic project (really! you should go check them out!)

UTM has even started to polish the CLI experience with utmctl, which has great commands for e.g. exec-ing stuff in VMs. (mine is meant to be a little more in-line with QEMU, and offer better debugger support out of the box).

The main reasons for building Xenu are thus:

1. Make a super hackable, extensible, and thin wrapper over the virtualization capabilities Apple already provides,
2. Xenu should always be super easy to build, and must (once SDK 13 lands in nixpkgs) build in the Nix sandbox. I was even reluctant to include Swift ArgumentParser because it meant brining in SwiftPM, but it’s probably the easiest “third” library library to install.
3. Although Xenu is meant to be qemu like, it isn’t qemu, so we can drop a bunch of stuff to support qemu.
4. Seriously, the only dependency you need is Apple’s own ArgumentParser library. That’s it!
5. CLI first. I probably won’t add a GUI.

   Another similar project https://github.com/crc-org/vfkit OrbStack (unfree/costs money) Colima (almost perfect, but lacks some customizations and is larger in scope, being meant for containers)

Xcode

So this is not ideal, but I managed to get a pure version of Xenu building by using the unwrapped Swift compiler from nixpkgs and writing my own xcrun shim to make Swift aware of all the Xcode frameworks and headers and so on. Hopefully it’s a very simple transition from this to SDK 13 vendored from nixpkgs (which I only now realize is the same damn thing that Xcode has! “S”oftware “D”evelopment “K”it, not just distributable libraries!)

add unixbench between NixOS x86_64-linux emulated with QEMU and aarch64 virtualized with avf