Change flash size from 1MB to 4MB and recover EEPROM content

[SwenBischof]

Hi,

I just recently started to dig into a project of a former co-worker and wanted to update the arduinoCore to 2.7.4.
Module used is ESP12s with 4MB flash, but currently firmware is builded with 1MB flash (eagle.flash1mo.ld)
Important is that the devices need to be updateable via OTA.

Currently a very old core is used 2.4.1, thats also why I want to update. With the ArduinoCore 2.7.4 the acutal firmware size is 525kB-->OTA will not be possible anymore.

Because of this I need to change the flash Configuration to 4MB. As far as I have seen the location of the EEPROM will change with the new configuration.

1MB OLD Flash Sector : (0x402FB000 - 0x40200000) / SPI_FLASH_SEC_SIZE
4MB Flash Sector: (0x405FB000 - 0x40200000) / SPI_FLASH_SEC_SI

I would need to:

1. update via OTA
2. recover the data in the "EEPROM" (Flash Sector)

I tried to use what was described in "EEPROM location was changed #6531" to recover Data but it didn't work.

Anybody has an suggestion how to make it possible?

[d-a-v]

Currently a very old core is used 2.4.1, thats also why I want to update. With the ArduinoCore 2.7.4 the acutal firmware size is 525kB-->OTA will not be possible anymore.

1. FYI OTA is now compressed.

2. You can use esptool to dump your flash chip.

3. You can make a small sketch that will allow you to download EEPROM data through network
   or display it on the serial link ({0xb0, 0x0b, ...}).

For next times you can use FS and network filesystems to access to your data (ftp, dav, browse with http).

[SwenBischof]

Thanks for the fast reply..

Currently a very old core is used 2.4.1, thats also why I want to update. With the ArduinoCore 2.7.4 the acutal firmware size is 525kB-->OTA will not be possible anymore.

1. FYI OTA is now compressed.

A great... what would be the size of 525kB after the compression?

2. You can use esptool to dump your flash chip.

I would need to change it remotly on multiple devices.

3. You can make a small sketch that will allow you to download EEPROM data through network
   or display it on the serial link ({0xb0, 0x0b, ...}).

Idea was to make a new firmware where the full 4MB (1MB sketch, 2MB free, 1MB SPIFF) of the ESP12S are available and I can restore after the OTA update the old EEPROM data to have still a fully functional device

For next times you can use FS and network filesystems to access to your data (ftp, dav, browse with http).

Sorry I am not shure what you mean?

[d-a-v]

A great... what would be the size of 525kB after the compression?

Generally around 30% => 370KB
edit: (note that you have to reflash before being able to received compressed OTA)

Idea was to make a new firmware where the full 4MB (1MB sketch, 2MB free, 1MB SPIFF) of the ESP12S are available and I can restore after the OTA update the old EEPROM data to have still a fully functional device

There should be some solutions. One which is not better than another one:
Make an intermediary 1MB-flash small sketch with OTA ability and able to provide by network the old EEPROM content.
Once you have it, you can OTA overwrite anything (4MB flash) in a sketch able to receive by network and update EEPROM or FS with your backup data.

For next times you can use FS and network filesystems to access to your data (ftp, dav, browse with http).

If your internal data are valuable, maybe an easy-access-to-data network facility will be useful sometimes (= what you are lacking now).

BTW, SPIFFS is deprecated. Welcome LittleFS.

[SwenBischof]

Thanks for all this valuable information....
For the recovery of the EEPROM data, do you have any suggestion?

[SwenBischof]

The problem is that if I loose the EEPROM Data I loose the connection to the device from remote... I know it is a design flaw

[d-a-v]

The intermediary sketch can read EEPROM on your current location and copy it to the new location of which you already know the address, by using Esp.flashRead/Write().
I think there's no reason for the new/standard 4MB location to be later overwritten.
There's more risk for the old one to be overwritten if you compile the intermediary sketch for 4MB chips and not 1MB chips.

You can also allow to remotely back it up, using the ESP8266WebServer (there are examples).

Also, I am sorry to have to mention that I take no responsibility on whatever you do with these lines 😨 and advise you to extensively try with test boards before doing anything in the fields :)

[SwenBischof]

thanks I will try to do this!! thanks so much