Bump minimum supported symengine version for built-in pickle support (#…

…7682)

* Bump minimum supported symengine version for built-in pickle support

The new symengine 0.9 release added native support in the package for
pickling symengine objects. Previously we had been converting symengine
objects to sympy objects so we could pickle them. With native support
for pickle in symengine now we no longer need this which besides
removing unnecessary should hopefully make pickling (which we do
internally as part of using multiprocessing) more reliable.

This also seems to fix the hanging we were seeing with multiprocessing
with Python 3.9 on Linux. While investigating that issue it points to
the underlying cause being a bug in cPython with the `fork()` based
start method, but we were only able to reliably trigger it after
switching to symengine in #6270 and having to rely on importing
symengine to pickle the symengine objects. Since we're no longer doing
that after bumping the minimum symengine version this removes the
default disabling of parallel dispatch with Python 3.9. While I'm not
100% confident this fixes the bug, in my testing locally I haven't been
able to reproduce the hang we were encountering (but this is ancedotal
at best). If we do encounter issues with multiprocess hanging in the
future we can look at rewriting the internals of `parallel_map()` or
switching it back to disabled by default.

Fixes #6188

* Fix typos in release notes

Co-authored-by: Jake Lishman <jake@binhbar.com>

Co-authored-by: Jake Lishman <jake@binhbar.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

qiskit/circuit/parameterexpression.py

@@ -506,34 +506,6 @@ def __eq__(self, other):
             return len(self.parameters) == 0 and complex(self._symbol_expr) == other
         return False
 
-    def __getstate__(self):
-        if _optionals.HAS_SYMENGINE:
-            from sympy import sympify
-
-            symbols = {k: sympify(v) for k, v in self._parameter_symbols.items()}
-            expr = sympify(self._symbol_expr)
-            return {"type": "symengine", "symbols": symbols, "expr": expr, "names": self._names}
-        else:
-            return {
-                "type": "sympy",
-                "symbols": self._parameter_symbols,
-                "expr": self._symbol_expr,
-                "names": self._names,
-            }
-
-    def __setstate__(self, state):
-        if state["type"] == "symengine":
-            import symengine
-
-            self._symbol_expr = symengine.sympify(state["expr"])
-            self._parameter_symbols = {k: symengine.sympify(v) for k, v in state["symbols"].items()}
-            self._parameters = set(self._parameter_symbols)
-        else:
-            self._symbol_expr = state["expr"]
-            self._parameter_symbols = state["symbols"]
-            self._parameters = set(self._parameter_symbols)
-        self._name_map = state["names"]
-
     def is_real(self):
         """Return whether the expression is real"""
 

qiskit/tools/parallel.py

@@ -65,9 +65,6 @@
     # Default False on Windows
     if sys.platform == "win32":
         PARALLEL_DEFAULT = False
-    # On python 3.9 default false to avoid deadlock issues
-    elif sys.version_info[0] == 3 and sys.version_info[1] == 9:
-        PARALLEL_DEFAULT = False
     # On macOS default false on Python >=3.8
     elif sys.platform == "darwin":
         if sys.version_info[0] == 3 and sys.version_info[1] >= 8:

releasenotes/notes/bump-symengine-8ca362f5b9fef199.yaml

@@ -0,0 +1,19 @@
+---
+upgrade:
+  - |
+    The minimum supported version of ``symengine`` is now 0.9.0. This was
+    necessary to improve compatibility with Python's ``pickle`` module which
+    is used internally as part of parallel dispatch with :func:`.parallel_map`.
+  - |
+    The default value of ``QISKIT_PARALLEL`` when running with Python 3.9 on
+    Linux is now set to ``TRUE``. This means when running :func:`.parallel_map`
+    or functions that call it internally such as :func:`.transpile` and
+    :func:`.assemble` the function will be executed in multiple processes and
+    should have better run time performance. This change was made because the
+    issues with reliability of parallel dispatch appear to have been resolved
+    (see `#6188 <https://github.com/Qiskit/qiskit-terra/issues/6188>`__ for
+    more details). If you still encounter issues because of this you can disable
+    multiprocessing and revert to the previous default behavior by setting the
+    ``QISKIT_PARALLEL`` environment variable to ``FALSE`` or setting the
+    ``parallel`` option to ``False`` in your user config file (also please file
+    an issue so we can track any issues related to multiprocessing).

requirements.txt

@@ -8,5 +8,5 @@ dill>=0.3
 python-constraint>=1.4
 python-dateutil>=2.8.0
 stevedore>=3.0.0
-symengine>=0.8 ; platform_machine == 'x86_64' or platform_machine == 'aarch64' or platform_machine == 'ppc64le' or platform_machine == 'amd64' or platform_machine == 'arm64'
+symengine>=0.9 ; platform_machine == 'x86_64' or platform_machine == 'aarch64' or platform_machine == 'ppc64le' or platform_machine == 'amd64' or platform_machine == 'arm64'
 tweedledum>=1.1,<2.0