Aarch64 SD images missing pi-zero-2.dtb file require to boot. #216886

deftdawg · 2023-02-17T23:42:27Z

[Apologies in advance if this is not the correct place to report this issue; there is basically no current documentation that I could find on how to boot and run nix on Raspberry Pi Zero 2]

Attempting to boot the Aarch64 SD card images built by Hydra on a Raspberry Pi Zero 2 fails because uboot cannot find gd19f48w8fwcbvb3g0pmfrlyj74zcnb0-linux-5.15.94-dtbs/broadcom/bcm2837-rpi-zero-2.dtb

Based on this line in master:

nixpkgs/pkgs/os-specific/linux/kernel/linux-rpi.nix

Line 78 in 270de46

copyDTB bcm2710-rpi-zero-2.dtb bcm2837-rpi-zero-2.dtb

It appears the file and a few others should be included, but for whatever reason (maybe this linux-rpi is not used or Hydra building off an different branch?) the file is not present in the image.

Here are the files actually present on the flashed SD card:

ls -1 /run/media/${USER}/NIXOS_SD/boot/nixos/gd19f48w8fwcbvb3g0pmfrlyj74zcnb0-linux-5.15.94-dtbs/broadcom
bcm2711-rpi-4-b.dtb
bcm2711-rpi-400.dtb
bcm2837-rpi-3-a-plus.dtb
bcm2837-rpi-3-b-plus.dtb
bcm2837-rpi-3-b.dtb
bcm2837-rpi-cm3-io3.dtb
bcm4908
northstar2
stingray

Steps To Reproduce

Steps to reproduce the behavior:

Download an AArch64 image from Hydra
Use etcher or dd to flash a micro sd card
Insert micro sd and boot a Pi Zero 2

Expected behavior

Expect it to boot to a nixOS shell

Workaround

It is possible to workaround the issue by mounting the micro SD on a operable Linux system by copying the RPi-3-b dtb file to the expected Pi Zero 2 file name:

cd /run/media/${USER}/NIXOS_SD/boot/nixos/gd19f48w8fwcbvb3g0pmfrlyj74zcnb0-linux-5.15.94-dtbs/broadcom/
cp bcm2837-rpi-3-b.dtb bcm2837-rpi-zero-2.dtb

Then unmount the SD and boot with your RPi Zero 2

Notify maintainers

Notify the 2 folks listed as having touched the RPi Zero 2 dtb line in linux-rpi.nix
@leo60228
@zhaofengli

The text was updated successfully, but these errors were encountered:

zhaofengli · 2023-02-18T00:05:09Z

Could you try the new_kernel variant of the SD image? It should have the DTB which was added in 5.18. The Hydra images use the mainline kernel, and linux-rpi.nix carries the RPi Foundation kernel. Alternatively, you can try this expression if you want to use the Foundation kernel.

deftdawg · 2023-02-18T15:30:00Z

Thanks @zhaofengli!

Tried the new_kernel variant... still panics because of a typo.

The file expected by uboot is

/boot/nixos/dm53ipsnxinwkx2vmgmaxfh5rcl05hv9-linux-6.1.12-dtbs/broadcom/bcm2837-rpi-zero-2.dtb

The file provided by the image has -w in it's name:

/boot/nixos/dm53ipsnxinwkx2vmgmaxfh5rcl05hv9-linux-6.1.12-dtbs/broadcom/bcm2837-rpi-zero-2-w.dtb

Copying the file:

cd /run/media/${USER}/NIXOS_SD/boot/nixos/dm53ipsnxinwkx2vmgmaxfh5rcl05hv9-linux-6.1.12-dtbs/broadcom
sudo cp bcm2837-rpi-zero-2-w.dtb bcm2837-rpi-zero-2.dtb

Allows it to boot 👍 and wlan0 shows up (unlike with the RPi3 workaround above)... sweet!

deftdawg · 2023-02-18T20:07:16Z

Looks like I can get things to run, though the Pi Zero 2 needs help to keep from dying when running nixos-rebuild switch as it runs out of memory.

I've made a couple of tweaks:

Disable RPi Camera support and reduce VRAM to 16MB from the default of 128MB.

edit /run/media/${USER}/FIRMWARE/config.txt:

[pi02]
kernel=u-boot-rpi3.bin
# Give up VRAM for more Free System Memory
# - Disable camera which automatically reserves 128MB VRAM
start_x=0 
# - Reduce allocation of VRAM to 16MB minimum for non-rotated (32MB for rotated)
gpu_mem=16

Create a 2 GB swap file in root

export SWAP_FILE=/swap
# create a 2GB swap file
if [ ! -f ${SWAP_FILE} ]; then
  sudo dd if=/dev/zero of=${SWAP_FILE} bs=100M count=20
  # format file as swap fs
  sudo mkswap -f ${SWAP_FILE}
fi
# activate swap file
sudo swapon ${SWAP_FILE}
echo

Still need to figure out permanent configuration, but at least with these I can rebuild without killing the Pi.

The expected dtb file name is `bcm2837-rpi-zero-2-w.dtb` on Raspberry Pi Zero 2 W; the current code is missing the `-w` from the file name which results in u-boot failing to load. see NixOS#216886

nixos-discourse · 2023-03-15T22:24:43Z

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/raspberry-pi-compute-module-3-compatibility/16210/11

deftdawg · 2023-03-31T06:50:41Z

Looks like the problem is stale references in the uboot bins

[deftdawg@nixos:~/sda1]$ grep  bcm2837-rpi-zero-2.dtb *                                   
grep: u-boot-rpi3.bin: binary file matches
grep: u-boot-rpi4.bin: binary file matches

[deftdawg@nixos:~/sda1]$ strings u-boot-rpi3.bin | grep broadcom
... 
broadcom/bcm2837-rpi-zero-2.dtb
...

This is based on the last new kernel sd image on hydra from Mar 20, 2023.

deftdawg · 2023-03-31T07:09:07Z

Looks like the problem was fixed in uboot 17 days ago... So we'd need a uboot update in nixos and a working hydra build

u-boot/u-boot@0e8c940

plmercereau · 2023-08-04T09:40:18Z

The PR with a newer version of uboot has been merged to nixpkgs, but it unfortunately does not solve the problem for me.

I created a small module that copies the right uboot file, creates a swap partition and allows to add options to the firmware configuration. Here's the gist.

Eventually my config looks like this:

{ config, lib, pkgs, ... }:
{
  imports = [
    <nixpkgs/nixos/modules/installer/sd-card/sd-image-aarch64-installer.nix>
    ./sd-image.nix
  ];
  system.stateVersion = "23.11";

  # Pi Zero 2 struggles to work without swap
  sdImage.swap.enable = true;
  sdImage.extraFirmwareConfig = {
    # Give up VRAM for more Free System Memory
    # - Disable camera which automatically reserves 128MB VRAM
    start_x = 0;
    # - Reduce allocation of VRAM to 16MB minimum for non-rotated (32MB for rotated)
    gpu_mem = 16;
  };
  # bzip2 compression takes loads of time with emulation, skip it. Enable this if you're low on space.
  sdImage.compressImage = false;

  networking = {
    interfaces."wlan0".useDHCP = true;
    wireless = {
      enable = true;
      interfaces = [ "wlan0" ];
      networks = {
        "<ssid>" = {
          psk = "<ssid-key>";
        };
      };
    };
  };

  # Enable OpenSSH out of the box.
  services.sshd.enable = true;

  # NTP time sync.
  services.timesyncd.enable = true;

}

alex-nt · 2023-08-05T10:53:05Z

@plmercereau many thanks, I used your config and added it to my fork of nixpkgs here: https://github.com/NixOS/nixpkgs/compare/release-23.05...alex-nt:nixpkgs:feature-23.05-rpi?expand=1

Using sd-image-aarch64-new-kernel-no-zfs-installer.nix everything works! But whenever I try the rpi kernel via

boot = {
    kernelPackages = pkgs.linuxPackages_rpi3;
}

I loose wifi (wlan0 fails to init). I've been using a fork of nixpkgs since 2022-12 till now on RPI02w, latest kernel always provided usable results compared with the rpi provided one. I wonder what I keep messing up.

deftdawg · 2023-08-05T15:55:34Z

For wifi I observed everytime I got a message in journalctl -b -u dhcpcd relating to expiring ipv6 my wifi would go down... not sure if what you've got is related, but this workaround did the trick for me:

#87802 (comment)

plmercereau · 2023-08-05T17:12:21Z

my wifi works fine with the above configuration (connecting to a router with DHCP enabled), but sometimes IPv6 somehow takes precedence over IPv4, for instance:

[nixos@nixos:~]$ ping www.github.com
PING www.github.com (140.82.121.4) 56(84) bytes of data.
64 bytes from lb-140-82-121-4-fra.github.com (140.82.121.4): icmp_seq=1 ttl=49 time=38.9 ms
64 bytes from lb-140-82-121-4-fra.github.com (140.82.121.4): icmp_seq=2 ttl=49 time=31.6 ms
64 bytes from lb-140-82-121-4-fra.github.com (140.82.121.4): icmp_seq=3 ttl=49 time=28.8 ms
^C
--- www.github.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 28.768/33.080/38.882/4.261 ms


[nixos@nixos:~]$ ping www.google.com
PING www.google.com(ams16s32-in-x04.1e100.net (2a00:1450:400e:80c::2004)) 56 data bytes
^C
--- www.google.com ping statistics ---
9 packets transmitted, 0 received, 100% packet loss, time 8170ms


[nixos@nixos:~]$ ping -4 www.google.com
PING  (142.250.179.164) 56(84) bytes of data.
64 bytes from ams15s41-in-f4.1e100.net (142.250.179.164): icmp_seq=1 ttl=56 time=28.3 ms
64 bytes from ams15s41-in-f4.1e100.net (142.250.179.164): icmp_seq=2 ttl=56 time=27.6 ms
64 bytes from ams15s41-in-f4.1e100.net (142.250.179.164): icmp_seq=3 ttl=56 time=23.2 ms
^C
---  ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 23.231/26.375/28.291/2.240 ms

I don't know if it is related to your problem though. I will try your suggestion @deftdawg

alex-nt · 2023-08-05T17:17:53Z

@plmercereau @deftdawg

With the following?

boot = {
    kernelPackages = pkgs.linuxPackages_rpi3;
}

Guys, you are giving me hope here, I might burn a few hours tomorrow and check to see what happens. All in all I am still very happy with your config @plmercereau. I have everything working (even i2c, but I didn't get to test spi yet, last time I failed to get spi working). Thanks yet again!

rjpcasalino · 2023-10-21T13:21:06Z

I am to understand that the core issue pi-zero-2.dtb file missing isn't fixed with latest u-boot? Many thanks to @deftdawg for helping me boot my Zero 2 with that hint. I think we also need to persevere @plmercereau's config someplace to help others that might not be looking at GitHub issues and considering the u-boot update didn't fix this yet. Perhaps updating the NixOS ARM wiki? In any case, thanks all!

rjpcasalino · 2023-11-05T18:35:07Z

@deftdawg cp bcm2837-rpi-3-b.dtb bcm2837-rpi-zero-2.dtb is still the work around I use on a fresh pi zero 2 but @plmercereau would be nice to get your module to work on a flake system like it's done here for a Pi 3 https://myme.no/posts/2022-12-01-nixos-on-raspberrypi.html. The price of zero 2 has dropped and it would be nice to get a more stable system

plmercereau · 2023-11-05T22:23:35Z

Hello folks,
I'll try to write a flake example in the upcoming days, and adapt the gist accordingly.
As regards pushing this upstream, I don't think it is a good idea because:

on second thought, creating a swap partition may not be a good idea, as it would tend to dramatically reduce the lifetime of SD cards (too many write operations). Maybe we should try an option in using zram swap, or to suggest building the system remotely instead of on the pi zero
I feel the cp bcm2837-rpi-3-b.dtb bcm2837-rpi-zero-2.dtb line is too hacky and should be solved somewhere else - but where? :/

rjpcasalino · 2023-11-06T00:00:03Z

awesome! looking forward to it. I tried a bit but couldn't manage to get anything working since I've moved to fully using flakes. Many thanks. I'm not sure where cp bcm2837-rpi-3-b.dtb bcm2837-rpi-zero-2.dtb should go either... @zhaofengli sorry to ping but thought you might be able to help figure that one out? The latest .img doesn't work without this hack but it would be nice if we could "support" the zero 2 a bit better but I'm clueless here...

plmercereau · 2023-11-06T15:04:51Z

Hello,
I wrote a small example that works on my end with flakes: https://github.com/plmercereau/nixos-pi-zero-2
What it does (and doesn't):

when using the unstable channel, there is no need to use the cp bcm2837 ... thing. It works as is, as long as the add these lines for enabling the wifi:

  hardware.enableRedistributableFirmware = lib.mkForce false;
  hardware.firmware = [pkgs.raspberrypiWirelessFirmware];

for the swap issue, I preferred the option to build the system in another machine. I used deploy-rs, but I am sure it would work with nixos-rebuild --target-host too
Still couldn't find a way to use boot.kernelPackages = pkgs.linuxKernel.packages.linux_rpi3...

rjpcasalino · 2023-11-06T15:20:40Z

Nice! The flake works like a dream!

nixos-discourse · 2024-01-09T11:24:28Z

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixos-on-raspberry-pi-zero-w/38018/1

deftdawg · 2024-01-12T05:29:48Z

Been a minute since I had a chance to play with this, but the hydra images @ https://hydra.nixos.org/job/nixos/release-23.11/nixos.sd_image_new_kernel_no_zfs.aarch64-linux now boot on my Pi Zero 2W without needing to copy dtb files, though I'm missing or forgetting a step as plugging an OTG adapter isn't working with my keyboard.

One thing to note the no_zfs work, however the other build https://hydra.nixos.org/job/nixos/release-23.11/nixos.sd_image.aarch64-linux, which I presume use zfs, don't work on my pi.

Going to close this since it seems the original defect is now solved. 🍻

NixOS/nixpkgs#216886 (comment)

deftdawg added the 0.kind: bug Something is broken label Feb 17, 2023

deftdawg mentioned this issue Feb 20, 2023

Raspberry Pi Zero 2 W provided dtb file name does not match expected #217355

Closed

diegobfernandez mentioned this issue Apr 26, 2023

uboot: 2023.01 -> 2023.04 #228353

Closed

12 tasks

deftdawg closed this as completed Jan 12, 2024

go3ranh added a commit to go3ranh/flake that referenced this issue Mar 18, 2024

fmt + pwnzero config

5afb4e0

NixOS/nixpkgs#216886 (comment)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Aarch64 SD images missing pi-zero-2.dtb file require to boot. #216886

Aarch64 SD images missing pi-zero-2.dtb file require to boot. #216886

deftdawg commented Feb 17, 2023

zhaofengli commented Feb 18, 2023

deftdawg commented Feb 18, 2023

deftdawg commented Feb 18, 2023

nixos-discourse commented Mar 15, 2023

deftdawg commented Mar 31, 2023 •

edited

Loading

deftdawg commented Mar 31, 2023

plmercereau commented Aug 4, 2023

alex-nt commented Aug 5, 2023

deftdawg commented Aug 5, 2023

plmercereau commented Aug 5, 2023 •

edited

Loading

alex-nt commented Aug 5, 2023

rjpcasalino commented Oct 21, 2023 •

edited

Loading

rjpcasalino commented Nov 5, 2023

plmercereau commented Nov 5, 2023

rjpcasalino commented Nov 6, 2023 •

edited

Loading

plmercereau commented Nov 6, 2023

rjpcasalino commented Nov 6, 2023 •

edited

Loading

nixos-discourse commented Jan 9, 2024

deftdawg commented Jan 12, 2024

Aarch64 SD images missing pi-zero-2.dtb file require to boot. #216886

Aarch64 SD images missing pi-zero-2.dtb file require to boot. #216886

Comments

deftdawg commented Feb 17, 2023

Steps To Reproduce

Expected behavior

Workaround

Notify maintainers

zhaofengli commented Feb 18, 2023

deftdawg commented Feb 18, 2023

deftdawg commented Feb 18, 2023

nixos-discourse commented Mar 15, 2023

deftdawg commented Mar 31, 2023 • edited Loading

deftdawg commented Mar 31, 2023

plmercereau commented Aug 4, 2023

alex-nt commented Aug 5, 2023

deftdawg commented Aug 5, 2023

plmercereau commented Aug 5, 2023 • edited Loading

alex-nt commented Aug 5, 2023

rjpcasalino commented Oct 21, 2023 • edited Loading

rjpcasalino commented Nov 5, 2023

plmercereau commented Nov 5, 2023

rjpcasalino commented Nov 6, 2023 • edited Loading

plmercereau commented Nov 6, 2023

rjpcasalino commented Nov 6, 2023 • edited Loading

nixos-discourse commented Jan 9, 2024

deftdawg commented Jan 12, 2024

deftdawg commented Mar 31, 2023 •

edited

Loading

plmercereau commented Aug 5, 2023 •

edited

Loading

rjpcasalino commented Oct 21, 2023 •

edited

Loading

rjpcasalino commented Nov 6, 2023 •

edited

Loading

rjpcasalino commented Nov 6, 2023 •

edited

Loading