USB device boot not working on RPi4

[ptesarik]

USB device mode boot does not seem to work for me. I connected my laptop to my RPi4 with a straight Type-C cable and I can see the USB device:

[292272.958931] usb 1-2: new high-speed USB device number 90 using xhci_hcd
[292273.107496] usb 1-2: config index 0 descriptor too short (expected 55, got 32)
[292273.107972] usb 1-2: New USB device found, idVendor=0a5c, idProduct=2711
[292273.107977] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[292273.107980] usb 1-2: Product: BCM2711 Boot
[292273.107983] usb 1-2: Manufacturer: Broadcom
[292273.107986] usb 1-2: SerialNumber: af04b2da

The warning seems to be a red herring; I am not sure where this expected size of 55 comes from, as you can see that wTotalLength is actually 32:

ezekiel:~ # lsusb -v -s 1:90

Bus 001 Device 090: ID 0a5c:2711 Broadcom Corp. 
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0 
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  idVendor           0x0a5c Broadcom Corp.
  idProduct          0x2711 
  bcdDevice            0.00
  iManufacturer           1 Broadcom
  iProduct                2 BCM2711 Boot
  iSerial                 3 af04b2da
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           32
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xc0
      Self Powered
    MaxPower              100mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x01  EP 1 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
Device Status:     0x0000
  (Bus Powered)

At this point, I modified usbboot to recognize the new product id (0x2711), but whatever I try I am unable to move forward. I have extracted the original firmware blob from my EEPROM (using flashrom), removed the 8-byte header and truncated it to just 44852 bytes, so this should be now exactly the same code that would be loaded over SPI, but still no luck.

[ghollingworth]

In what way does it 'not work' what's the response from rpiboot?

[ptesarik]

Let me copy the relevant output here:

Waiting for BCM2835/6/7
Found device 12 idVendor=0x0a5c idProduct=0x2711
Bus: 1, Device: 89 Path: 1-2
Found candidate Compute Module...Device located successfully
Initialised device correctly
Found serial number 3
Sending bootcode.bin
libusb_bulk_transfer sent 24 bytes; returned 0
Writing 44852 bytes
libusb_bulk_transfer sent 44852 bytes; returned 0
Successful read 4 bytes 

So, all looks good, I received a zero status code, but the board appears to be hanging - no output whatsoever. Here's the patch I applied to the usbboot utility:

diff --git a/main.c b/main.c
index bab7a68..cbcc175 100644
--- a/main.c
+++ b/main.c
@@ -102,7 +102,8 @@ libusb_device_handle * LIBUSB_CALL open_device_with_vid(
 		
 		if (desc.idVendor == vendor_id) {
 			if(desc.idProduct == 0x2763 ||
-			   desc.idProduct == 0x2764)
+			   desc.idProduct == 0x2764 ||
+			   desc.idProduct == 0x2711)
 			{
 				if(verbose == 2)
 				      printf("Found candidate Compute Module...");
@@ -644,7 +645,7 @@ int main(int argc, char *argv[])
 		while (ret);
 
 		last_serial = desc.iSerialNumber;
-		if(desc.iSerialNumber == 0)
+		if(desc.iSerialNumber == 0 || desc.idProduct == 0x2711)
 		{
 			printf("Sending bootcode.bin\n");
 			second_stage_boot(usb_device);

[ptesarik]

As an aside, I have discovered where the 55 comes from. When the Linux kernel first asks for a configuration descriptor, the BCM2711 boot ROM program responds with wTotalLength = 55. But this first request allocates only the minimum 9-byte buffer to find out how big the configuration descriptor actually is. The kernel immediately allocates a buffer for 55 bytes of data and sends a new request, but this time the boot ROM sets the wTotalLength field to 32 and also sends only 32 bytes.

Looking at the difference, 55 - 32 = 23, i.e. just enough for one more interface (9 bytes) with 2 endpoints (2 × 7 = 14 bytes). There is a comment in usbboot code which suggests that a mass storage interface is also available on BCM2837. It looks like this interface was removed in 2711, but the wTotalLength field in the USB configuration descriptor was not adjusted. OTOH it does not explain why that field's value is correct on all subsequent queries…

Anyway, this is unrelated to the question of what exactly should be sent to the USB boot endpoint and/or how to debug failures. It must be a silly mistake I'm doing here, but without any feedback from the board, it's extremely difficult to debug…

Oh, that reminds me: Would it be possible to debug the VPU using JTAG?

[pelwell]

Oh, that reminds me: Would it be possible to debug the VPU using JTAG?

Not without a licence from Broadcom (at least), so no.

[ghollingworth]

Yeah, USB mass storage (as a device) was enableable but now removed (it wasn't actually very useful in the end).

We're not going to be looking at USB boot for a while, we're first working on the network and MSD boot modes. But it does work (I've tested it when 2711 first came back).

Have you tried sending the recovery.bin file, rather than hand hacking the EEPROM?

[ptesarik]

Yes, that was the first thing to try. I copied recover.bin to boot/bootcode.bin and then ran usbboot like this:

tesarik@ezekiel:~/hardware/Raspberry/boot/usbboot> sudo ./rpiboot -vv -d boot
Waiting for BCM2835/6/7
Found device 1 idVendor=0x1d6b idProduct=0x0003
Bus: 2, Device: 1 Path: 2
Found device 2 idVendor=0x138a idProduct=0x0097
Bus: 1, Device: 7 Path: 1-9
Found device 3 idVendor=0x04f2 idProduct=0xb5ab
Bus: 1, Device: 5 Path: 1-8
Found device 4 idVendor=0x8087 idProduct=0x0a2b
Bus: 1, Device: 3 Path: 1-7
Found device 5 idVendor=0x0a5c idProduct=0x2711
Bus: 1, Device: 112 Path: 1-2
Found candidate Compute Module...Device located successfully
Initialised device correctly
Found serial number 3
Sending bootcode.bin
libusb_bulk_transfer sent 24 bytes; returned 0
Writing 41504 bytes
libusb_bulk_transfer sent 41504 bytes; returned 0
Successful read 4 bytes 

No other indication of success, nothing on the serial port, although I set BOOT_UART=1 in bootconf.txt. I assumed that could be because the stock recovery.bin file contains a signature that is not valid for my Raspberry Pi. At least the stock pieeprom.bin file and the image I saved with flashrom differ in these 20 bytes.

As to the use case, this would be a great method of unbricking one's RPi4 if something goes wrong with the EEPROM… Of course, you can always flash the chip with an external programmer while 2711 is not running, but that's a lot more involved. Especially since the chip is located very close to the audio jack, so it's quite hard to put a clip on it without desoldering the connector first. 😏

[timg236]

The ROM will always load recovery.bin from the sd-card in preference to the EEPROM so it's not possible to brick a Pi with a corrupted EEPROM.

USB device boot is not yet supported by pieeprom.bin, so whilst it might be possible to run the code it won't be able to load files from start.elf. Other stuff might fail because other than validating that the ROM can load and run code it's essentially untested.

We do plan to add proper support for USB device boot but AFAIK it's unlikely to get any attention until after other bootmodes are implemented.

[ptesarik]

Thank you for all your answers. Much appreciated. I believe this issue can be closed for the time being.

[ptesarik]

For the record, I have just re-run USB boot now, and it works (EEPROM content was recovered). I'm not sure what went wrong before, but I'm no longer able to reproduce the failure.

With that, I'm going to update usbboot.

[Lucky7star]

I’ve encountered much of what you have here, just haven’t documented not nearly, but besides trying to boot straight from the usb, Ive ordered a PoE hat, and intend to upgrade the kernel to 5.0 ubuntu’s 18.04 flavor. So stay tuned