Parallelization

[rlawson]

Topic

• Running an RMG job in parallel

Context

I'm trying to generate a model for a gasoline-air reactor system using the QM module in parallel.
In an instance I don't run in parallel, everything works fine. However, when I include the number of
threads flag in my job scheduler, the simulation tends to get 'stuck' at Writing QM files with 32 Processes
(where 32 is the number of threads for this job)

Is there a flag I need to include to run mopac in parallel as well.
I've attached my Script here.
script.txt

Installation Information

I've installed RMG using Anaconda on a Linux system.
RMG 3.0.0
Anaconda3-2019.10

Below is the error I'm getting.
Process ForkPoolWorker-1:
Traceback (most recent call last):
File "/home1/rlawson/.conda/envs/rmg_env3/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/home1/rlawson/.conda/envs/rmg_env3/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "/home1/rlawson/.conda/envs/rmg_env3/lib/python3.7/multiprocessing/pool.py", line 110, in worker
task = get()
File "/home1/rlawson/.conda/envs/rmg_env3/lib/python3.7/multiprocessing/queues.py", line 354, in get
return _ForkingPickler.loads(res)
TypeError: init() takes from 1 to 7 positional arguments but 8 were given
Process ForkPoolWorker-2:
Traceback (most recent call last):
File "/home1/rlawson/.conda/envs/rmg_env3/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/home1/rlawson/.conda/envs/rmg_env3/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "/home1/rlawson/.conda/envs/rmg_env3/lib/python3.7/multiprocessing/pool.py", line 110, in worker
task = get()
File "/home1/rlawson/.conda/envs/rmg_env3/lib/python3.7/multiprocessing/queues.py", line 354, in get
return _ForkingPickler.loads(res)
TypeError: init() takes from 1 to 7 positional arguments but 8 were given

[ajocher]

Hi, it might be possible that you are running out of memory when using 32 processes. You can try using a smaller number of processes.

[JacksonBurns]

This is likely a MOPAC licensing issue (although the traceback given by RMG is unhelpful) which has been resolved with #2417.

Closing.