esp_flash_write_encrypted is broken (IDFGH-4493)

[rojer]

Environment

• Development Kit: [ESP32-Wrover-Kit|ESP32-DevKitC|ESP32-PICO-Kit|ESP32-LyraT|ESP32-LyraTD-MSC|none]
• Kit version (for WroverKit/PicoKit/DevKitC): [v1|v2|v3|v4]
• Module or chip used: ESP32-WROOM-32
• IDF version 4.2
• Build System: Make
• Compiler version 8.4.0-2020r3
• Operating System: Linux
• Using an IDE?: No
• Power Supply: USB

Problem Description

esp_partition_write() does not work on encrypted partitions.

Expected Behavior

esp_partition_write() should be able to write data to encrypted partitions

Actual Behavior

new API falls back to ROM API (spi_flash_write_encrypted) that doesn't work.
gets stuck unable to set WEL bit properly (more likely, unable to read status register properly).

Steps to reproduce

ESP32D0WDQ6 with GigaDevice flash in 80 MHz QIO mode

[Dec 26 17:35:24.714] I (34) qio_mode: Enabling default flash chip QIO
[Dec 26 17:35:24.714] I (39) boot.esp32: SPI Speed      : 80MHz
[Dec 26 17:35:24.736] I (44) boot.esp32: SPI Mode       : QIO
[Dec 26 17:35:24.736] I (48) boot.esp32: SPI Flash Size : 4MB
...
[Dec 26 17:35:25.451] I (766) spi_flash: detected chip: gd
[Dec 26 17:35:25.451] I (767) spi_flash: flash io: qio

Code to reproduce this issue

enable flash encryption and try to write to an encrypted partition.

works fine if new API is disabled (CONFIG_SPI_FLASH_USE_LEGACY_IMPL=y)

[Alvin1Zhang]

Thanks for reporting, we will look into.

[igrr]

@rojer if you switch from QIO to DIO mode, does this issue still happen?

[rojer]

@igrr set CONFIG_ESPTOOLPY_FLASHMODE_DIO=y CONFIG_ESPTOOLPY_FLASHMODE_QIO=n - nope, doesn't seem to help.

from memory when i was debugging this - after new flash driver initializes, reading status register via rom function starts returning incorrect result. so yes, there is some configuration mismatch but it's not QIO / DIO, it seems.

[igrr]

Thanks for checking @rojer. We do have an automated test for flash encryption functionality (based on the examples/security/flash_encryption example), and that seems to be passing both on master and on release/v4.2 branch. So we need to find what is different between your setup and ours. Will look into it more.

If you get a chance, could you please try running examples/security/flash_encryption example with its default configuration on your hardware?

[rojer]

@igrr investigated further and found the root cause: frequency mismatch between rom settings and the driver.
if memspi_host_config_t.speed and the clock configuration in image parameters (set based on CONFIG_ESPTOOLPY_FLASHFREQ_* variable) get out of sync with reality (and in our case it does), things go awry.
the reason for things to go out of sync is that we sometimes rewrite the image header of the image we write based on command line settings.
i think flash driver should query current ROM settings when initializing to ensure actual config is in sync as opposed to compile-time.

[rojer]

alternatively (and preferably), esp_flash driver should sync ROM's settings to the one it uses.

[rojer]

ok, for now the workaround is pretty simple - set default to 80 mhz, to match what mos tool sets by default (cesanta/mongoose-os@23e8064) but proper fix wshould be made to ensure speed settings agree between ROM and modern APIs or that modern API does not require ROM API for encrypted writes anymore.

[rojer]

another twist on this issue is the SPIRAM clock - if it doesn't match the flash settings, ROM access to flash is also broken.

[mythbuster5]

Sorry for the long wait, I have fixed the issue(flash_encryption failed with 80M flash and 40M psram) with the patch below, could you please have a try? Thanks!

esp32_patch_for_flash_encryption_with_40Mram_80Mflash.txt

[rojer]

@mythbuster5 yes, that works, thank you!

[Alvin1Zhang]

Thanks for reporting and sharing updates, feel free to reopen.