memory problem

[kjm1102]

I've got the ulp counter in a 14k .mpy file as a def. If the def is not called everything runs OK so the 14k file size is not a problem. There is in fact plenty of ram because the test I'm running is to download a new version of the 14k .mpy file, so there is at least 14k spare.

However if I call the counter def, urequests crashes with "MemoryError: memory allocation failed, allocating 1792 bytes". This is a problem I've struck before on other boards with different defs. For example on ESP32camera boards I have to deinit the camera after taking a pic in the camera def before using urequests or else a similar memory error from urequests.

I'm a bit worried this time though because I can't just deinit the ulp counter, it needs to keep running during deepsleep. Apart from the slow mem it uses for the count & state registers does the counter.py progrm set aside any ram that wouldn't be clawed back automatically when the counter def returns the count to the .mpy?

PS: unlike my trials with with the ESP camera boards, where any attempt to use urequest would crash, my problem with the ulp counter def on this DEVKIT1 board only occurs when I try to download a 14k file from the server, the other urequest tasks (like uploading the count) are working OK.

[wnienhaus]

The assembler does use quite a bit of memory (it's actually mostly about memory fragmentation, rather than purely allocating it all).

But: the ULP requires "no memory" - at least not any memory that is available to MicroPython. MicroPython does not use the RTC_SLOW_MEM for any of its normal operation.

So as soon as you have loaded a program into the ULP, it will happily run irrespective of MicroPython's memory usage and vice versa.

I would see 3 ways of addressing your issue:

1. The first one is more of a "likely it wont help (enough)... but it's easy to try, so try it anyway": Simply try to call a garbage collect after assembling+loading your ULP. MicroPython should really do this by itself when it runs out of memory, but a forced garbage collect every so often can help reduce memory fragmentation.

2. Alternatively, after the assembly+loading of the ULP program, carefully deallocate everything again you put in memory and then force a garbage collect. You dont need to worry about variables that went out-of-scope (e.g. variables inside a method that ended), but any global variables of modules you imported can be removed. You can do that with del variablename. For modules, you need to free up both the local reference, plus it's entry in sys.modules[]. i.e. del modname; del sys.modules['modname']. Also note that modules might have loaded further modules and so on, so check the contents of sys.modules to del everything you no longer need (and all other reference to those things you have). After all this, call gc.collect() for the garbage collector to clean up everything no longer needed from RAM.

3. The above is a bit tedious, so what I do in my projects is to save the ULP binary to a file and on next boot, load that file instead of re-assembling the ULP source code (thus skipping the memory intensive and slow assembly process). It looks something like this:

## PART 1: save assembled output to file
from esp32_ulp import src_to_binary
source = """\
  ...assembly code
"""
binary = src_to_binary(source)
with open("counter.ulp", "w") as f:
    f.write(binary)

## PART 2: load from file
ulp = ULP()
with open("counter.ulp", "r") as f:
    binary = f.read()
    ulp.load_binary(0, binary)
    ulp.set_wakeup_period(0, 50000) #in usec
    ulp.run(entry_addr)

Then make sure you only do PART 1 when really needed (e.g. wrap the with open(..,"r") as f) in a try...except and in case of exception, do PART 1 and perhaps even force a restart (import machine; machine.reset()) afterwards to make sure all RAM is cleared). Then every time you change the source, make sure you delete the "counter.ulp" file to trigger assembly again.

For general memory management topics, it may also be useful to post on https://forum.micropython.org. Once the assembling step is done, there is no need for any (MicroPython) memory anymore for running / keeping-running the ULP. So then on memory management becomes a bit broader than this library.

(One more little tip: If you look at my edge counter (that I shared previously), you can also check if the magic token has been set already (i.e. ULP code is already loaded and running), and then even skip PART2 during a wakeup - i.e. do nothing. In my example I test for the magic token if if mem32[ULP_MEM_BASE + 4*4] & ULP_DATA_MASK == 0xcafe)

[kjm1102]

Limiting download of that 14k file to those cycles after the ulp has setup worked great. I should have worked that out for myself, thnx again.

Re #2 deleting modules. sys.modules shows 9 esp32 modules (& one flashdev I probaly should leave?) after ulp counter setup. Is there a way to see their size? Since each one needs to be done twice I'm hoping to go the low hanging fruit & just delete the big ones.

[RSC-Games]

Do you mean this file?
https://github.com/micropython/micropython/blob/master/ports/esp32/modules/flashbdev.py

I would not worry about keeping a reference to this file. That executes at boot to mount your flash VFS at root. However it's
only a couple hundred bytes long.
You could also make a function that contains a whitelist of modules and then it deletes all other modules from sys.modules.
Keep in mind that this would not delete your local references to the modules.

Also, as far as I know you cannot see the size of the module. The closest you can get to seeing how much RAM modules use is to enter micropython.mem_info(True) and see how much of the RAM is fragmented and how many consecutive blocks are marked as B or M. Link: https://www.adafruitdaily.com/2017/03/31/measure-your-memory-usage-with-mem_info/