Memory consumption

[RobRomijnders]

Hi all, thanks for implementing this API. It has enabled great research already.

I am running into memory problems. Running the simulator with 1 million users, it's intended use-case, my 60 gb memory run out in under 40 steps. Changing the 'days_of_interactions' parameter hasn't changed that.

Am I doing something wrong? What typical memory consumption do you observe?

Thanks in advance,
Rob

[p-robot]

Hi Rob,
Great to hear that OpenABM has been a useful tool!

Regarding your question, a simple run of the model with 1 million individuals should definitely not exhaust 60Gb of memory. The example in examples/example_101.py with 1M users runs without any hiccups on my 2016 MBP, which has 8Gb of physical memory:

import COVID19.model as abm
model = abm.Model( params = { "n_total" : 1000000, "end_time": 50 } )
model.run()

Would it be possible for you to please provide a minimal example of where you're coming up against problems with memory? It will help us diagnose the issue.

Best regards,
Will

[RobRomijnders]

Hi Will,

Thanks for the help. I digged a bit further and realized the memory leak was in the Python-bindings with SWIG. I'll make a pull-request soon to fix this.

Indeed, after the fix 100k users consumes less than 5gb and 1M users consumes less than 10gb.

[p-robot]

Hi Rob,
That's good to hear that you've found a remedy in the interim. That'd be fantastic if you had a PR for the issue you found - many thanks for contributing.
Will