[BUG] Building the reports crashes with multiple concurrent processes

[dweemx]

Describe the bug
The pipelines crash sometimes when multiple reports are building concurrently.

To Reproduce
Steps to reproduce the behavior:

1. Configure with these options:

nextflow config vib-singlecell-nf/vsn-pipelines -profile tenx,single_sample,singularity

This is not restricted to this pipeline. Any pipeline generating reports could potentially have this bug.

2. Run using this entry point:

nextflow -C nextflow.config run vib-singlecell-nf/vsn-pipelines -entry single_sample

3. See error:

Command output:
  (empty)

Command error:
    File "/opt/venv/lib/python3.7/site-packages/traitlets/config/application.py", line 663, in launch_instance
      app.initialize(argv)
    File "</opt/venv/lib/python3.7/site-packages/decorator.py:decorator-gen-124>", line 2, in initialize
    File "/opt/venv/lib/python3.7/site-packages/traitlets/config/application.py", line 87, in catch_config_error
      return method(app, *args, **kwargs)
    File "/opt/venv/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 527, in initialize
      self.init_sockets()
    File "/opt/venv/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 266, in init_sockets
      self.shell_port = self._bind_socket(self.shell_socket, self.shell_port)
    File "/opt/venv/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 213, in _bind_socket
      return self._try_bind_socket(s, port)
    File "/opt/venv/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 189, in _try_bind_socket
      s.bind("tcp://%s:%i" % (self.ip, port))
    File "zmq/backend/cython/socket.pyx", line 550, in zmq.backend.cython.socket.Socket.bind
    File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython.checkrc._check_rc
  zmq.error.ZMQError: Address already in use
  
  Executing:   0%|          | 0/41 [00:06<?, ?cell/s]
  Traceback (most recent call last):
    File "/opt/venv/bin/papermill", line 8, in <module>
      sys.exit(papermill())
    File "/opt/venv/lib/python3.7/site-packages/click/core.py", line 764, in __call__
      return self.main(*args, **kwargs)
    File "/opt/venv/lib/python3.7/site-packages/click/core.py", line 717, in main
      rv = self.invoke(ctx)
    File "/opt/venv/lib/python3.7/site-packages/click/core.py", line 956, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/opt/venv/lib/python3.7/site-packages/click/core.py", line 555, in invoke
      return callback(*args, **kwargs)
    File "/opt/venv/lib/python3.7/site-packages/papermill/cli.py", line 214, in papermill
      cwd=cwd,
    File "/opt/venv/lib/python3.7/site-packages/papermill/execute.py", line 104, in execute_notebook
      **engine_kwargs
    File "/opt/venv/lib/python3.7/site-packages/papermill/engines.py", line 49, in execute_notebook_with_engine
      return self.get_engine(engine_name).execute_notebook(nb, kernel_name, **kwargs)
    File "/opt/venv/lib/python3.7/site-packages/papermill/engines.py", line 304, in execute_notebook
      nb = cls.execute_managed_notebook(nb_man, kernel_name, log_output=log_output, **kwargs)
    File "/opt/venv/lib/python3.7/site-packages/papermill/engines.py", line 372, in execute_managed_notebook
      preprocessor.preprocess(nb_man, safe_kwargs)
    File "/opt/venv/lib/python3.7/site-packages/papermill/preprocess.py", line 20, in preprocess
      with self.setup_preprocessor(nb_man.nb, resources, km=km):
    File "/usr/local/lib/python3.7/contextlib.py", line 112, in __enter__
      return next(self.gen)
    File "/opt/venv/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 345, in setup_preprocessor
      self.km, self.kc = self.start_new_kernel(**kwargs)
    File "/opt/venv/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 296, in start_new_kernel
      kc.wait_for_ready(timeout=self.startup_timeout)
    File "/opt/venv/lib/python3.7/site-packages/jupyter_client/blocking/client.py", line 120, in wait_for_ready
      raise RuntimeError('Kernel died before replying to kernel_info')
  RuntimeError: Kernel died before replying to kernel_info

Expected behavior
Should not crash

Screenshots
/

Please complete the following information:

• OS: CentOS 7
• Nextflow Version: 19.12.0-edge
• vsn-pipelines Version: 0.18.0

Additional context
Add any other context about the problem here.

[dweemx]

Probably related to:

• zmq.error.ZMQError: Address already in use, when running multiprocessing with multiple notebooks using papermill nteract/papermill#511
• Spawning many kernels may result in ZMQError jupyter/jupyter_client#487