Booting OP-TEE in RPi3

[etraudonun]

Hello,
I'm new to OP-TEE and wanted to learn more about it by running it on a Raspberry Pi 3.
However, I'm having trouble setting it up.
Here's the list of commands/actions I just followed based on your build support page (https://github.com/OP-TEE/build):

• Installed the necessary dependencies;

• Installed repo;

• Got OP-TEE's code through repo;

• Got the toolchains;

• Ran the "make all run" command;

• Flashed an SD card with the following commands:
  $$ fdisk /dev/sdc
  > p
  > d
  > n
  > p
  > 1
  >
  > +32M
  > n
  > p
  > 2
  >
  >
  > t
  > 1
  > e
  > a
  > 1
  > p
  > w

  $$ mkfs.vfat -F16 -n BOOT /dev/sdc1"
  $$ mkdir -p /media/boot"
  $$ mount /dev/sdc1 /media/boot"
  $$ cd /media"
  $$ gunzip -cd ~/optee/gen_rootfs/filesystem.cpio.gz | sudo cpio -idmv "boot/*""
  $$ umount boot"

  $$ mkfs.ext4 -L rootfs /dev/sdc2"
  $$ mkdir -p /media/rootfs"
  $$ mount /dev/sdc2 /media/rootfs"
  $$ cd rootfs"
  $$ gunzip -cd ~/optee/gen_rootfs/filesystem.cpio.gz | sudo cpio -idmv"
  $$ rm -rf /media/rootfs/boot/*"
  $$ cd .. && umount rootfs"

Afterwards, I plugged the card in the Raspberry Pi but when I booted it, it couldn't read the card.
Could you please help me in understanding what I'm missing?

Thank you very much for time
Best Regards,
Nuno

[igoropaniuk]

Hi @etraudonun,

Could you please put back into PC your SD (should be run by root user) amd run these commands and copy here theirs output:

# fdisk /dev/sdc
> p 

# mount /dev/sdc1 /media/boot
# ls -lah /media/boot

# mount /dev/sdc2 /media/rootfs
# ls -lah /media/rootfs

...it couldn't read the card.

What actual output do you see when you are trying to boot RPi3 with inserted SD?

[etraudonun]

Hello,
Sorry for the late reply.
I failed to mention that I did "sudo -s" before actually writing to the card.
Regarding your commands, here is their output:

$ fdisk /dev/sdc/
Welcome to fdisk (util-linux 2.25.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): p
Disk /dev/sdc: 7,2 GiB, 7744782336 bytes, 15126528 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x466a911d

Device Boot Start End Sectors Size Id Type
/dev/sdc1 * 2048 67583 65536 32M e W95 FAT16 (LBA)
/dev/sdc2 67584 15126527 15058944 7,2G 83 Linux

$ mount /dev/sdc1 /media/boot
$ ls -lah /media/boot
total 20K
drwxr-xr-x 2 root root 16K Jan 1 1970 .
drwxr-xr-x 5 root root 4,0K Fev 22 14:43 ..

$ mount /dev/sdc2 /media/rootfs
$ ls -lah /media/rootfs
total 80K
drwxr-xr-x 17 root root 4,0K Fev 22 14:44 .
drwxr-xr-x 5 root root 4,0K Fev 22 14:43 ..
drwxr-xr-x 2 root root 4,0K Fev 22 14:44 bin
drwxr-xr-x 2 root root 4,0K Fev 22 14:44 boot
drwxr-xr-x 3 root root 4,0K Fev 22 14:44 data
drwxr-xr-x 2 root root 4,0K Fev 22 14:20 debug
drwxr-xr-x 3 root root 4,0K Fev 22 14:44 dev
drwxr-xr-x 6 root root 4,0K Fev 22 14:44 etc
lrwxrwxrwx 1 root root 11 Fev 22 14:44 init -> bin/busybox
drwxr-xr-x 5 root root 4,0K Fev 22 14:44 lib
drwx------ 2 root root 16K Fev 22 14:42 lost+found
drwxr-xr-x 2 root root 4,0K Fev 22 14:20 mnt
drwxr-xr-x 2 root root 4,0K Fev 22 14:20 proc
drwxr-xr-x 2 root root 4,0K Fev 22 14:44 sbin
drwxr-xr-x 2 root root 4,0K Fev 22 14:20 sys
drwxr-xr-x 2 root root 4,0K Fev 22 14:20 tmp
drwxr-xr-x 4 root root 4,0K Fev 22 14:44 usr
drwxr-xr-x 6 root root 4,0K Fev 22 14:44 var

When I boot the RPi3, it doesn't show any output and the ACT LED doesn't blink, which means the RPi3 can't actually boot from the card.
Does the output of the commands you asked seem ok?

[igoropaniuk]

Hi @etraudonun,
Seems that you've missed this step

gunzip -cd ~/optee/gen_rootfs/filesystem.cpio.gz | sudo cpio -idmv "boot/*""

because your boot partition is empty (based on your ls -lah /media/boot output).
Just mount it (mount /dev/sdc1 /media/boot) and run this gunzip command once more.

Best regards,
Igor

[etraudonun]

Thanks! It seems the double quotes were messing up the command, so it should be something like:

gunzip -cd ~/optee/gen_rootfs/filesystem.cpio.gz | sudo cpio -idmv boot/*

So at this point the RPi3 reads the card and displays the following information:

JTAG (gpio) enabled
NET:      NET Initialization Skipped
No ethernet found.
Hit any key to stop autoboot:    0
reading Image
10187264 bytes read in 875 ms (11.1 MiB/s)
reading bcm2710-rpi-3-b.dtb
13376 bytes read in 19 ms (687.5 KiB/s)
reading optee.bin
365560 bytes read in 47 ms (7.4 MiB/s)
##Flattened Device Tree blob at 01700000
     Booting using the fdt blob at 0x1700000
     reserving fdt memory region: addr=0 size=1000
     reserving fdt memory region: addr=8000000 size=2000000
     Loading Device Tree to 0000000039f2d000, end 0000000039f3343f  ... OK

Starting kernel  ...

## Transfering control to ARM TF (at address 8400000) (dtb at 39f2d000)...

After this point it jumps to a screen with the Raspberry Pi's logo with a blinking cursor and seems to get stuck as I cannot input any commands such as the "ps aux | grep tee-supplicant" to check if the tee-supplicant is running or not.
Similarly the RPi3 seems not to respond to the keyboard input when I try to stop the autoboot by hitting any key.
After the ARM TF takes control should I do something different?

Thank you very much for your patience

[igoropaniuk]

Hi @etraudonun,
Actually, you are doing everything right, and you don't see any output because UART console (/dev/ttyS0) is used by default. To get output on your HDMI connected display, change default console line in https://github.com/OP-TEE/build/blob/master/rpi3/firmware/uboot.env.txt#L13 to ttyconsole=tty0

Then rebuild u-boot.env

$ cd <your-rpi3-manifest>/build
$ make u-boot-env-clean
$ make u-boot-jtag-bin

and copy (with replacing old uboot.env) <your-rpi3-manifest>/out/uboot.env to your mounted SD BOOT partition (/dev/sdc1)

Then mount your SD ROOT FS partition (/dev/sdc2), and change in etc/inittab line ttyS0::askfirst:/bin/sh -sc ". /etc/profile" to tty0::askfirst:/bin/sh -sc ". /etc/profile", so it should look like:

#
# /etc/inittab
#
::sysinit:/etc/init.d/rc.init
tty0::askfirst:/bin/sh -sc ". /etc/profile"
::ctrlaltdel:/sbin/poweroff
::shutdown:/etc/init.d/rc.shutdown
::restart:/sbin/init

After all these steps you will receive output on your display

BTW, I'll create a PR with changes to support output to both UART and display

BR,
Igor

[etraudonun]

Hello @igoropaniuk,
Once again thank you very much for your help.
Your explanation solved the display issue, and I managed to create a dummy TA similar to hello_world, compile, deploy and run it successfully.

I read part of OP-TEE's documentation and looked at bits of code and several questions came to my mind. Although some are related with booting OP-TEE in RPi3, others are not, so let me know whether I should use the mailing list instead.

In order to ease testing and deployment, you recommend booting via NFS and TFTP. However, I ran into two problems. At the beginning of section 5 (where you describe how to boot through NFS), you have the following information:

In the description below we will use the following terminology:

HOST_IP=192.168.1.100   <--- This is your desktop computer
RPI_IP=192.168.1.200    <--- This is the Raspberry Pi

The problem here is that I cannot find my RPi3's ip, as ifconfig returns this information:

root@RPi3:/ ifconfig
lo        Link encap:Locao Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1%5174864/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

I had to do ifconfig eth0 up in order for the Ethernet connection to be set up. I don't know if this has any implication in running the command run nfsboot after halting in U-Boot.

The second issue is directly related with the first as I cannot halt U-Boot through the keyboard when the output is forwarded to HDMI. I believe in this case the issue could be solved by using the UART instead with ttyS0 in both uboot.env.txt and in etc/inittab as it was originally. Do you confirm this? And because I don't have an UART adapter, do you think a simple solution such as this one would be enough? Also, when editing uboot.env to account for the devices' ips, do you confirm that sections 5.4.1 are 5.4.2 are two different approaches to the same end, with one requiring handling the SD card, while the other allowing for modifications to be made through U-Boot, respectively? Again, I couldn't test the approach of 5.4.2 because of the UART issue. On a side note, in section 5.3.1 you mention a file called optee.bin whose name might have changed since then to head.bin. I'm not entirely sure about this, since I coudn't test NFS.

Another issue not directly related with these two is the root filesystem. In section 5.7 you describe how to setup an Ubuntu based filesystem, which I tried to use with the current setup, i.e., through SD card handling and not NFS. The problem however is that although Ubuntu boots into the login screen, I can't seem to login. I used the linaro/linaro combination but after typing linaro and pressing enter, the command prompt advances to the next line and no password is asked. Even after typing linaro a second time (as password) and pressing enter nothing happens, and after a while my input disappears and the login request resumes again. Am I doing something wrong?

Because I don't have a JTAG debug device, I would also like to know if the J-LINK EDU device would be a good option, since you mention you use J-LINK BASE devices, which according to Segger's website provides the same functionalities as the BASE, at a lower cost, and for non-commercial purposes only, which happens to be the case.

Again, thank you in advance for your help,
Best Regards,
Nuno

[igoropaniuk]

Hi @etraudonun ,
Sorry for the late answer.

I had to do ifconfig eth0 up in order for the Ethernet connection to be set up. I don't know if this has any implication in running the command run nfsboot after halting in U-Boot.

Linux kernel automatically brings up and configures your network interfaces, when you use ip command line param (particulary, ip=dhcp)

For details check https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt

The second issue is directly related with the first as I cannot halt U-Boot through the keyboard when the output is forwarded to HDMI.

Actually, I have the same issue. I suggest to check http://elinux.org/RPi_U-Boot, there is a section called "Oleksandr Tymoshenko's branch" where you can find information about building U-Boot fork with working USB support on RPi3. I haven't tried it yet, so if are going to check it - please notice me about your test results.

And because I don't have an UART adapter, do you think a simple solution such as this one would be enough?

I think this one is a bit overpriced (IMHO), I'm using a cheap CP2102 USB to UART (~ 1$) https://www.aliexpress.com/item/CP2102-module-USB-to-TTL-serial-UART-STC-download-cable-PL2303-Super-Brush-line-upgrade-for/32390863227.html?spm=2114.13010608.0.0.21GWwW. It works fine, believe me :)

Also, when editing uboot.env to account for the devices' ips, do you confirm that sections 5.4.1 are 5.4.2 are two different approaches to the same end, with one requiring handling the SD card, while the other allowing for modifications to be made through U-Boot, respectively?

you are right

On a side note, in section 5.3.1 you mention a file called optee.bin whose name might have changed since then to head.bin. I'm not entirely sure about this, since I coudn't test NFS.

Actually, these two binaries are not related to each other.
optee.bin is OP-TEE OS core
head.bin - generic rpi3 initialization stuff

Another issue not directly related with these two is the root filesystem. In section 5.7 you describe how to setup an Ubuntu based filesystem, which I tried to use with the current setup, i.e., through SD card handling and not NFS.

Unfortunately I didn't write this section of README, and I've never tried to use Linaro Ubuntu rootfs images on RPi3 setup. If you still encoutering this issue, I can check it and try to use it on my RPi3
@jbech-linaro , have you ever faced similar issue?

Because I don't have a JTAG debug device, I would also like to know if the J-LINK EDU device would be a good option, since you mention you use J-LINK BASE devices, which according to Segger's website provides the same functionalities as the BASE, at a lower cost, and for non-commercial purposes only, which happens to be the case.

As far as I know, @jbech-linaro has used J-LINK from Segger for RPi3. Joakim, do you have any particular suggestions about J-LINK jtags?

Best regards,
Igor

[utkarshagrawalwsu]

@etraudonun , I am trying to run Ubuntu as you did. Can you help me with some issue I am facing?
@igoropaniuk @jbech-linaro Your help would be appreciated too.

You can refer to my issue with the link below. I did not want to hijack your thread. :)
#1398