Core data-abort while reading from mapped virtual address in pTA

[maki-chan]

For a project I am working on, I need to map the normal world memory, so the secure world can read from it. To accomplish this, I am using a pTA, as, according to related issues #2933, #2150, #1706 and especially #1343, it is not possible to use core mapped memory via register_phys_mem in user TAs. According to my research of the memory in QEMU, I wanted to read from the address 0x00080830 (only 2 bytes for testing) and because of that, I mapped that small area in the plat-vexpress/main.c like that:

register_phys_mem(MEM_AREA_RAM_NSEC, 0x00080000, SMALL_PAGE_SIZE);

When I boot up in QEMU, this mapping seems to work. See the output on startup:

D/TC:0 0 add_phys_mem:576 TEE_SHMEM_START type NSEC_SHM 0x42000000 size 0x00200000
D/TC:0 0 add_phys_mem:576 TA_RAM_START type TA_RAM 0x0e300000 size 0x00d00000
D/TC:0 0 add_phys_mem:576 VCORE_UNPG_RW_PA type TEE_RAM_RW 0x0e151000 size 0x001af000
D/TC:0 0 add_phys_mem:576 VCORE_UNPG_RX_PA type TEE_RAM_RX 0x0e100000 size 0x00051000
D/TC:0 0 add_phys_mem:576 0x00080000 type RAM_NSEC 0x00080000 size 0x00001000
D/TC:0 0 add_phys_mem:576 ROUNDDOWN(0x09040000, CORE_MMU_PGDIR_SIZE) type IO_SEC 0x09000000 size 0x00200000
D/TC:0 0 verify_special_mem_areas:514 No NSEC DDR memory area defined
D/TC:0 0 add_va_space:615 type RES_VASPACE size 0x00a00000
D/TC:0 0 add_va_space:615 type SHM_VASPACE size 0x02000000
D/TC:0 0 dump_mmap_table:751 type TEE_RAM_RX   va 0x0e100000..0x0e150fff pa 0x0e100000..0x0e150fff size 0x00051000 (smallpg)
D/TC:0 0 dump_mmap_table:751 type TEE_RAM_RW   va 0x0e151000..0x0e2fffff pa 0x0e151000..0x0e2fffff size 0x001af000 (smallpg)
D/TC:0 0 dump_mmap_table:751 type NSEC_SHM     va 0x0e400000..0x0e5fffff pa 0x42000000..0x421fffff size 0x00200000 (pgdir)
D/TC:0 0 dump_mmap_table:751 type IO_SEC       va 0x0e600000..0x0e7fffff pa 0x09000000..0x091fffff size 0x00200000 (pgdir)
D/TC:0 0 dump_mmap_table:751 type RES_VASPACE  va 0x0e800000..0x0f1fffff pa 0x00000000..0x009fffff size 0x00a00000 (pgdir)
D/TC:0 0 dump_mmap_table:751 type SHM_VASPACE  va 0x0f200000..0x111fffff pa 0x00000000..0x01ffffff size 0x02000000 (pgdir)
D/TC:0 0 dump_mmap_table:751 type RAM_NSEC     va 0x11200000..0x11200fff pa 0x00080000..0x00080fff size 0x00001000 (smallpg)
D/TC:0 0 dump_mmap_table:751 type TA_RAM       va 0x11300000..0x11ffffff pa 0x0e300000..0x0effffff size 0x00d00000 (smallpg)
D/TC:0 0 core_mmu_entry_to_finer_grained:795 xlat tables used 1 / 5
D/TC:0 0 core_mmu_entry_to_finer_grained:795 xlat tables used 2 / 5
D/TC:0 0 core_mmu_entry_to_finer_grained:795 xlat tables used 3 / 5
D/TC:0 0 core_mmu_entry_to_finer_grained:795 xlat tables used 4 / 5
I/TC: 

My pTA application, called from the normal world, uses the physical address given from the normal world, translates it to the according virtual address and then tries to read from it. The full code of this part looks like this:

paddr_t paddr = *(paddr_t*)params[0].memref.buffer; // 0x00080830
uint64_t readlen = *(uint64_t*)params[1].memref.buffer; // 0x2
void *vaddr = phys_to_virt(paddr, MEM_AREA_RAM_NSEC); // 0x11200830

// Debug output of this below
DMSG("0x%08lx %p\n", paddr, vaddr);

memcpy(params[2].memref.buffer, vaddr, readlen);
params[2].memref.size = readlen;

The DMSG output shows:

D/TC:? 0 hash_address_range:36 0x00080830 0x11200830

So the physical address can be converted into its virtual address without any problems. But the following error arises as soon as the pTA reaches the memcpy() call:

D/TC:1 0 get_fault_type:504 [abort] Unhandled fault!
E/TC:1 0 
E/TC:1 0 Core data-abort at address 0x11200830
E/TC:1 0  esr 0x96000010  ttbr0 0x0e17a020   ttbr1 0x00000000   cidr 0x0
E/TC:1 0  cpu #1          cpsr 0x20000104
E/TC:1 0  x0  000000000f2036c0 x1  0000000011200830
E/TC:1 0  x2  0000000000000002 x3  0000000000000000
E/TC:1 0  x4  000000000e17ed75 x5  0000000000000000
E/TC:1 0  x6  0000000000000030 x7  0000000000000020
E/TC:1 0  x8  0000000000000008 x9  000000000e14e434
E/TC:1 0  x10 0000000000000000 x11 0000000000000000
E/TC:1 0  x12 0000000000000000 x13 0000000000000000
E/TC:1 0  x14 0000000000000000 x15 0000000000000000
E/TC:1 0  x16 0000000000000000 x17 0000000000000000
E/TC:1 0  x18 0000000000000000 x19 000000000e17f040
E/TC:1 0  x20 0000000000000002 x21 0000000011200830
E/TC:1 0  x22 0000000000080830 x23 000000000e17f130
E/TC:1 0  x24 0000000000000002 x25 000000000f200020
E/TC:1 0  x26 0000000000000000 x27 0000000000000000
E/TC:1 0  x28 0000000000000000 x29 000000000e17efc0
E/TC:1 0  x30 000000000e114098 elr 000000000e13e0f8
E/TC:1 0  sp_el0 000000000e17efc0

If I comment out the memcpy() call, everything runs fine. I even tried to only read from vaddr by outputting its contents directly to the console via DMSG, without copying it which results in the same core data-abort error as the memcpy() call, so it is not an error while writing the contents to the memref buffer.

Have I somehow overlooked something here? Is it not possible to read from a virtual address like that when mapped into the core? According to the documentation, pTAs should have the same access to memory as the rest of the OP-TEE core. Please help me figuring out, why reading from the virtual address is not possible at all. Thank you very much in advance!

[maki-chan]

Could the problem arise because, as far as I can see, the mapped memory is automatically used as dynamic shared memory?

When I map the area as MEM_AREA_RAM_SEC instead of MEM_AREA_RAM_NSEC, it seems that I can read from the virtual address (getting only bytes with value 0, though that could only show that I read from the wrong area and not from where I really want to read).

Is anybody able to help or explain, please?

[jenswi-linaro]

Physical address 0..0x08000000 is secure memory, BootROM. So the data you're reading depends on how large ROM you're feeding to QEMU.

[maki-chan]

In that case I am probably searching for the wrong offset. I want to read the linux kernel loaded in QEMU (using the qemu_v8 profile). After some more research into the physical memory of the QEMU guest, it seems that 0x40080000 could be the right offset for the kernel. Does this sound right to you?

[jenswi-linaro]

That address is more reasonable.

[maki-chan]

It does work now, thank you very much for your hint about the BootROM location. It helped finding the correct address.