spiffsimg and the firmware config out of sync

[TerryE]

Expected behaviour

spiffsimg should always be able to generate a SPIFFS image which can be loaded and read by the firmware.

Actual behaviour

As part of the SDK 3.0 changes, I tidied up the startup code which mounts the SPIFFS drops to a small block size if the file system is <32Kb, rather than <128Kb, but didn't carry this into spiffsimg. Oops.

Scope of work

• I need to change this small BS to large BS to be consistent in both. Any recommendations on which?
• The latest spiffs source has some minor tweaks, I can carry these across at the same time.
• There are deep sleep applications (LFS only) where it makes sense to not mount any SPIFFS, so the startup should now tolerate a zero length SPIFFS partition.
• Integration of spiffs remake into root Makefile. @HHHartmann, has the default behaviour of the make changed? I noticed that the root make no longer seems to update spiffsimg?

I just want a broad OK from the other committers before I raise this PR as the core element is a bugfix..

[pnkowl]

The latest spiffs source has some minor tweaks, I can carry these across at the same time.

I periodically run into SPIFFS "file" corruption issues. Is that something that might be applicable here? I am working on the "ticket" now. The beginning goes something like this

`problem

• files go missing
• files are truncated
• files are altered

Supplied cases show this occurs when all SPIFFs pages have been written once.
For a 4MB module, this usually takes awhile.
When the firmware is updated or SPIFFs is formatted, the risk clock is reset.

As I understand it, when no SPIFFs pages are free, but are either used or deleted, a garbage collection operation must be performed to free the deleted pages. Sometimes this results in

• truncated files (deleted pages)
• deleted index or object lookup pages (data is there but filename page is gone)
• corrupted pages (modified data file bytes at the beginning or end of a page boundary)
  `
  I have captured a sequence of module images (4MB each) illustrating this behavior. Demonstrated with a 32KB file either FTPed or serial uploaded. Corruption symptoms are slightly different, but corrupted all the same. Effect is repeatable across modules (e.g. ESP8266 Nodemcu V0.9 and Wemos D1 mini). I have seen this across firmware builds, LFS builds and time.

[TerryE]

This is really a duplication of #2943 raised by @albarrett Alex Barret, so let me close this and continue the discussion there.