cpu/esp32: add periph_flashpage support #19079

gschorcht · 2022-12-31T13:56:01Z

Contribution description

This PR provides the periph_flashpage support for ESP32x SoCs.

For byte-aligned read access to constant data in the flash, the MMU of all ESP32x SoCs allows to map a certain number of 64 kByte pages of the flash into the data address space of the CPU. This address space is called DROM. Normally the whole DROM address space is assigned to the section .rodata. The default flash layout used by all ESP32x SoCs is:

Address in Flash	Content
`0x0000` or `0x1000`	bootloader
`0x8000`	parition table
`0x9000`	`nvs` parition with WiFi data
`0xf000`	`phy_init` partition with RF data
`0x10000`	`factory` partition with the app image

The factory partition consists of a number of 64 kByte pages for the sections .text, .rodata, .bss and others. The .text and rodata sections are page-aligned and are simply mapped into the instruction address space (IROM) and the data address space (DROM), respectively. All other sections are loaded into RAM.

If the periph_flashpage module is used, the periph_flashpage driver

decreases the size of the .rodata section in DROM address space by CONFIG_ESP_FLASHPAGE_CAPACITY,
adds a section .flashpage.writable of size CONFIG_ESP_FLASHPAGE_CAPACITY at the end of DROM address space that is mapped into data address space of the CPU,
reserves a region of size CONFIG_ESP_FLASHPAGE_CAPACITY starting from 0x10000 in front of the image partition factory and
moves the image partition factory by CONFIG_ESP_FLASHPAGE_CAPACITY to address 0x10000 + CONFIG_ESP_FLASHPAGE_CAPACITY.

The new flash layout is then:

Address in Flash	Content
`0x0000` or `0x1000`	bootloader
`0x8000`	parition table
`0x9000`	`nvs` parition with WiFi data
`0xf000`	`phy_init` partition with RF data
`0x10000`	flashpage region
`0x10000 + CONFIG_ESP_FLASHPAGE_CAPACITY`	`factory` partition with the app image

This guarantees that the flash pages are not overwritten if a new app image with changed size is flashed. CONFIG_ESP_FLASHPAGE_CAPACITY has to be a multiple of 64 kBytes.

~~The PR includes PR #19077 and PR #19078 for the moment to be compilable.~~

Testing procedure

The following tests should pass.

USEMODULE='esp_log_startup ps shell_cmds_default' BOARD=esp32-wroom-32 make -j8 -C tests/periph_flashpage flash term

USEMODULE='esp_log_startup ps shell_cmds_default' BOARD=esp32-wroom-32 make -j8 -C tests/mtd_flashpage flash term

Issues/PRs references

Depends on PR #19077
Depends on PR #19078

riot-ci · 2022-12-31T13:58:07Z

Murdock results

✔️ PASSED

c21eb6b tests/periph_flashpage: use od_hex_dump for memdump

Success	Failures	Total	Runtime
6779	0	6779	15m:01s

Artifacts

Documentation preview

19077: cpu/esp32: configurable linker scripts r=benpicco a=gschorcht ### Contribution description This PR provides configurable linker scripts for ESP32x SoCs. Using the vendor `memory.ld.in` file and a `sections.ld.in` file instead of the static versions of these files, from which the actual used `memory.ld` and `sections.ld` are generated using the C preprocessor, allows to use the configuration in `sdkconfig.h` as well as Kconfig to define a custom memory layout. For example, it is no longer necessary to maintain different `memory.ld` files for the ESP32 BLE module, since the memory layout is now defined from the values of the configuration. Note for the review: The `memory.ld.in` files are now simply copies of the manufacturer's `memory.ld.in` files. However, it is not possible to use the vendor's `memory.ld.in` files directly, because they have to be extended further on, e.g. for the `periph_flashpage` implementation. This PR is prerequisite for the `periph_flashpage` support in PR #19079. ### Testing procedure Green CI. ### Issues/PRs references Prerequisite for PR #19079 Co-authored-by: Gunar Schorcht <gunar@schorcht.net>

19078: cpu/esp32: define FLASHFILE_POS r=benpicco a=gschorcht ### Contribution description Instead of using a fixed position of the image file in the flash, the variable `FLASHFILE_POS` is used which allows to override the default position of the image in the flash at 0x10000. This PR is a prerequisite for the `periph_flashpage` implementation PR #19079. ### Testing procedure Flashing a ESP32x SoC should work with `FLASHFILE_POS=0x20000`, for example: ``` USEMODULE=esp_log_startup FLASHFILE_POS=0x20000 BOARD=esp32-wroom-32 make -j8 -C tests/shell flash ``` The bootloader output should give `00020000` as offset for the `factory` partition ``` I (75) boot: Partition Table: I (78) boot: ## Label Usage Type ST Offset Length I (84) boot: 0 nvs WiFi data 01 02 00009000 00006000 I (91) boot: 1 phy_init RF data 01 01 0000f000 00001000 I (97) boot: 2 factory factory app 00 00 00020000 000199b0 I (104) boot: End of partition table ``` and ``` I (125) esp_image: segment 0: paddr=00020020 vaddr=3f400020 size=02140h ( 8512) map ``` during the load of the image. ### Issues/PRs references Prerequisite for PR #19079 Co-authored-by: Gunar Schorcht <gunar@schorcht.net>

gschorcht · 2023-01-03T14:48:15Z

Please note: There is an open problem with the ESP32-S2 implementation. The flashpage access requires any kind of additional mapping between the DCache (Data Bus Cache) and the MMU pages for ESP32-S2. Unfortunatly, til now this works only with ICache (Instruction Bus Cache) but not with DCache, no idea why yet. The problem of ICache is that it only allows word access.

gschorcht · 2023-01-07T12:15:40Z

periph_flashpage implementation has been finished now and is working also on ESP32-S2.

f

Replaces the custom `memdump` implementation with `od_hex_dump`, which provides a more common and especially more compact print format.

benpicco · 2023-01-14T22:51:55Z

bors try

bors · 2023-01-15T08:01:38Z

try

Build succeeded:

benpicco

Also works on ESP32-C3 and ESP32-S3

btw, what's needed for riotboot support?

bors merge

kaspar030 · 2023-01-18T14:32:41Z

bors merge

gschorcht · 2023-01-18T15:02:50Z

btw, what's needed for riotboot support?

I was already thinking about that. There will be no chance to replace the ESP-IDF bootloader by a bare metal implementation since the bootloader preconfigures a lot of the SoC. My approach would be to implement riotboot as an RIOT application which includes the CPU startup procedure and that is loaded by the ESP-IDF bootloader. But instead of starting RIOT kernel it loads the RIOT application without the CPU startup procedure which then starts the kernel. I have no idea whether this could be realized.

benpicco · 2023-01-18T15:22:47Z

My approach would be to implement riotboot as an RIOT application which includes the CPU startup procedure and that is loaded by the ESP-IDF bootloader.

That's already how riotboot works anyway.

But instead of starting RIOT kernel it loads the RIOT application without the CPU startup procedure which then starts the kernel.

That would make changes to the CPU startup routines impossible.

Is it a problem to call the CPU init twice?
Those ESP32s usually have plenty of flash.

gschorcht · 2023-01-18T15:58:33Z

That's already how riotboot works anyway.

I see, I didn't check it yet.

Is it a problem to call the CPU init twice?

I'm afraid that doing the complete SoC initialization a second time won't really work. We will have to give it a try. I have actually had it on my to-do list for quite some time.

benpicco · 2023-01-18T16:02:02Z

Well for the most simple riotboot_slot (needed for SUIT updates) you might not even need to init the CPU.

It just checks the riotboot header of the two slots to decide where to jump, so you don't need to initialize any peripherals, the boot config should be sufficient for that.

bors · 2023-01-18T20:56:52Z

Build succeeded:

gschorcht · 2023-01-19T05:18:35Z

@benpicco Thankd for reviewing and merging.

gschorcht · 2023-01-19T05:22:36Z

Well for the most simple riotboot_slot (needed for SUIT updates) you might not even need to init the CPU.

I will take a look into riotboot and check it when I have time.

gschorcht requested review from leandrolanzieri, jia200x and MrKevinWeiss as code owners December 31, 2022 13:56

github-actions bot added Area: build system Area: Build system Area: cpu Area: CPU/MCU ports Area: Kconfig Area: Kconfig integration Area: tools Area: Supplementary tools Platform: ESP Platform: This PR/issue effects ESP-based platforms labels Dec 31, 2022

gschorcht added State: waiting for other PR State: The PR requires another PR to be merged first CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Dec 31, 2022

This was referenced Dec 31, 2022

cpu/esp32: configurable linker scripts #19077

Merged

cpu/esp32: define FLASHFILE_POS #19078

Merged

gschorcht force-pushed the cpu/esp32/periph_flashpage branch 3 times, most recently from 6c0d578 to 4f0a28a Compare December 31, 2022 15:52

benpicco removed the State: waiting for other PR State: The PR requires another PR to be merged first label Jan 2, 2023

gschorcht force-pushed the cpu/esp32/periph_flashpage branch from 4f0a28a to 88ba041 Compare January 3, 2023 00:11

github-actions bot removed Area: tools Area: Supplementary tools Area: build system Area: Build system labels Jan 3, 2023

gschorcht added Type: new feature The issue requests / The PR implemements a new feature for RIOT State: WIP State: The PR is still work-in-progress and its code is not in its final presentable form yet labels Jan 3, 2023

gschorcht added 3 commits January 7, 2023 10:38

cpu/esp32: add flashpage definitions in CPU config

f4c0d7d

cpu/esp32: add flashpage support to Kconfig

6f9c64c

cpu/esp32: add flashpage support to linker scripts

80833a7

gschorcht force-pushed the cpu/esp32/periph_flashpage branch from 88ba041 to 65633a2 Compare January 7, 2023 12:14

gschorcht requested a review from smlng as a code owner January 7, 2023 12:14

gschorcht requested review from aabadie, MichelRottleuthner and fjmolinas as code owners January 7, 2023 12:14

github-actions bot added the Area: tests Area: tests and testing framework label Jan 7, 2023

gschorcht removed the State: WIP State: The PR is still work-in-progress and its code is not in its final presentable form yet label Jan 7, 2023

gschorcht added 2 commits January 7, 2023 14:49

cpu/esp32: add flashpage support

32168da

f

tests/periph_flashpage: use od_hex_dump for memdump

c21eb6b

Replaces the custom `memdump` implementation with `od_hex_dump`, which provides a more common and especially more compact print format.

gschorcht force-pushed the cpu/esp32/periph_flashpage branch from 65633a2 to c21eb6b Compare January 7, 2023 13:50

bors bot added a commit that referenced this pull request Jan 14, 2023

Try #19079:

7549f36

benpicco approved these changes Jan 18, 2023

View reviewed changes

bors bot merged commit 7855aad into RIOT-OS:master Jan 18, 2023

gschorcht deleted the cpu/esp32/periph_flashpage branch February 4, 2023 12:18

jia200x added this to the Release 2023.04 milestone Apr 25, 2023

Ollrogge mentioned this pull request Jan 29, 2024

cpu: Add start and end markers for flash_writable section #20308

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cpu/esp32: add periph_flashpage support #19079

cpu/esp32: add periph_flashpage support #19079

gschorcht commented Dec 31, 2022 •

edited

Loading

riot-ci commented Dec 31, 2022 •

edited

Loading

gschorcht commented Jan 3, 2023 •

edited

Loading

gschorcht commented Jan 7, 2023

benpicco commented Jan 14, 2023

bors bot commented Jan 15, 2023

benpicco left a comment •

edited

Loading

kaspar030 commented Jan 18, 2023

gschorcht commented Jan 18, 2023

benpicco commented Jan 18, 2023

gschorcht commented Jan 18, 2023

benpicco commented Jan 18, 2023 •

edited

Loading

bors bot commented Jan 18, 2023

gschorcht commented Jan 19, 2023

gschorcht commented Jan 19, 2023

cpu/esp32: add periph_flashpage support #19079

cpu/esp32: add periph_flashpage support #19079

Conversation

gschorcht commented Dec 31, 2022 • edited Loading

Contribution description

Testing procedure

Issues/PRs references

riot-ci commented Dec 31, 2022 • edited Loading

Murdock results

Artifacts

gschorcht commented Jan 3, 2023 • edited Loading

gschorcht commented Jan 7, 2023

benpicco commented Jan 14, 2023

bors bot commented Jan 15, 2023

try

benpicco left a comment • edited Loading

Choose a reason for hiding this comment

kaspar030 commented Jan 18, 2023

gschorcht commented Jan 18, 2023

benpicco commented Jan 18, 2023

gschorcht commented Jan 18, 2023

benpicco commented Jan 18, 2023 • edited Loading

bors bot commented Jan 18, 2023

gschorcht commented Jan 19, 2023

gschorcht commented Jan 19, 2023

gschorcht commented Dec 31, 2022 •

edited

Loading

riot-ci commented Dec 31, 2022 •

edited

Loading

gschorcht commented Jan 3, 2023 •

edited

Loading

benpicco left a comment •

edited

Loading

benpicco commented Jan 18, 2023 •

edited

Loading