Why does my filesystem get corrupted on resets? ESP32

[benj5378]

I regurlarly see my Arduino Nano ESP32 having it's filesystem corrupted. It always happens after a reset, either giving Unicode error when trying to read my main.py file or the whole OS corrupted as with

The filesystem appears to be corrupted. If you had important data there, you
may want to make a flash snapshot to try to recover it. Otherwise, perform
factory reprogramming of MicroPython firmware (completely erase flash, followed
by firmware programming).

Now, the question is - why do I so regularly have file corruptions? Is it expected to having flash corruptions every ~10 times the board is reset? What can I do to prevent these regular corruptions when it experiences power loss once in a while?

I am only writing error messages to the file system. My power supply is 5.1 V 3.1 A.

class logToFile(io.IOBase):
    def __init__(self):
        pass

    def write(self, data):
        with open("logfile.txt", mode="a") as f:
            f.write(data)
        return len(data)

os.dupterm(logToFile())

[davefes]

What type of "reset" are you talking about? Any software and reset on the dev board should not result in a "power loss". That is not to say there might be some drop-out in your 5V supply. Oscilloscope required and monitor 5V and 3V3 supplies during your "reset".

Having to save data before the unit shuts-down due to power failure or a reset is another challenge.

[benj5378]

Both power supply cut and hard reset by pressing the reset button triggers it. I don't remember if soft resets did, might have had

[bixb922]

What flash filesystem are you using? If you are using FAT, backup flash and reformat flash with os.VfsLfs2.mkfs(bdev) to the Littlefs2 filesystem. Littlefs2 is very stable and rather unlikely to get corrupted, even when resetting the ESP32 during a file write operation.

If you are using Littlefs2, possible causes of corruption could be for example problems with the power supply such as low voltage or high frequency noise. Or else a defective flash.

[scruss]

OP is asking about the ARDUINO_NANO_ESP32, which does seem to force a FAT32 filesystem partition on its esp32s3 core

[benj5378]

@bixb922 I tried formatting the filesystem, however getting very weird issues:

>>> bdev
<Partition type=1, subtype=129, address=6356992, size=9830400, label=ffat, encrypted=0>
>>> bdev2 = esp32.Partition.find(esp32.Partition.TYPE_DATA, label="ffat")[0]
>>> bdev2
<Partition type=1, subtype=129, address=6356992, size=9830400, label=ffat, encrypted=0>

>>> vfs.umount("/")
>>> vfs.VfsLfs2.mkfs(bdev)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 22] EINVAL
# Why doesn't this work as per the documentation, even though they are the same? Maybe differences in hidden attributes?

>>> vfs.VfsLfs2.mkfs(bdev2)
# Success
vfs.mount(bdev2, "/")
>>> os.statvfs("/")
(4096, 4096, 2400, 2398, 2398, 0, 0, 0, 0, 255)

I can write and read files and all that stuff, untill I do a reboot - whether it's soft or hard. After the reboot, it's all corrupted.

However, it seems to be a mounting issue. If I use

>>> bdev2 = esp32.Partition.find(esp32.Partition.TYPE_DATA, label="ffat")[0]
>>> vfs.mount(bdev2, "/")  # bdev2, not bdev!!!!!!
>>> os.statvfs("/")
(0, 0, 0, 0, 0, 0, 0, 0, 0, 255)

then I still have all my data? What's wrong with the mounting? Is it the ffat label?

[bixb922]

Could it be that the FAT filesystem is still mounted? I'd do os.umount first to dismount the FAT filesystem, then VfsLfs2.mkfs with bdev, then VfsLfs2.mount. I think it's not correct to create a second bdev2 instance, just use the bdev that is provided by MicroPython.

[benj5378]

@bixb922 No, the filesystem was already unmounted. The reason I use bdev2 is because bdev for some really strange reason yields an error - see me code example.

[bixb922]

Unfortunately I don't have a Arduino Nano, and can't reproduce what you are posting. The sequence of instructions you posted works well on my ESP32.
Also see #14452, mentioned in this below, because using the filesystem via USB from the PC simultaneously with using it from MicroPython is a very likely source of file system corruption. If those recommendations don't work, also ask on the Arduino forum. The Arduino Nano is using a special built MicroPython image by Arduino, not a stock MicroPython image.

[scruss]

This sounds very similar to my experience with the Nano RP2040 Connect, which was fixed using a manual format of the partitions

[peterhinch]

Beware that if you switch to LittleFS that you won't be able to use USB Mass Storage.

Which is a big point in its favour. See this note for the reason.

[steveh127]

Link doesn't go to a discussion of why not to use USB Mass Storage.

[scruss]

It does for me:

By default the Pyboard's /flash/boot.py enables MSC (mass storage) mode. This makes the Pyboard look like a USB stick, making its filesystem visible to the PC. This helpful feature ignores a fundamental flaw which leads to filesystem corruption. This is because the USB standard requires mass storage devices to behave like disks with static content. By contrast a Pyboard can independently modify the "disk" contents causing chaos.

[steveh127]

That's odd as I get to a page of peterhinch's on github with no mention of USB mass storage. Not now a problem as you have copied the relevant discussion here the conclusion of which I'm happy with.

[peterhinch]

I don't know the reason but links to GitHub content sometimes take you to the start of the doc rather than the relevant para. Later the same link will work fine. Go figure :)

[mokhtazerdali]

I have the same problem :

%Run -c $EDITOR_CONTENT

MPY: soft reboot
The filesystem appears to be corrupted. If you had important data there, you
may want to make a flash snapshot to try to recover it. Otherwise, perform
factory reprogramming of MicroPython firmware (completely erase flash, followed
by firmware programming).

[mokhtazerdali]

esp32 with thonny, under µpython

[scruss]

please ask in a new question: your hardware is very different from the OP's