Gracefully handle failure to parse historical symengine files

During the discovery on the fix in this PR we discovered that setting
the ``use_symengine`` flag from Qiskit 0.45.x and 0.46.x would result in
newer versions of Qiskit being unable to parse the QPY file for the same
issue as being addressed here. This commit expands the logic to account
for this and raise a useful warning. It also updates the release notes
and documentation to document these limitations.

qiskit/qpy/__init__.py

@@ -116,6 +116,13 @@
 
 .. autoexception:: QPYLoadingDeprecatedFeatureWarning
 
+.. note::
+
+    When loading a QPY file generated using Qiskit 0.45.x or 0.46.x where :func:`.qpy.dump`
+    was called with the ``use_symengine`` argument set ``True`` these files can not be
+    loaded by Qiskit releases >= 1.0.0 due to API differences in the ``symengine`` library
+    that is used when QPY serializes :class:`.ParameterExpression` objects.
+
 QPY format version history
 --------------------------
 

qiskit/qpy/common.py

@@ -322,6 +322,17 @@ def load_symengine_payload(payload: bytes) -> symengine.Expr:
     major = payload[2]
     minor = payload[3]
     if int(symengine_version[1]) != minor:
+        if major != "0":
+            raise exceptions.QpyError(
+                "Qiskit doesn't support loading a symengine payload generated with symengine >= 1.0"
+            )
+        if minor == 9:
+            raise exceptions.QpyError(
+                "Qiskit doesn't support loading a historical QPY file with `use_symengine=True` "
+                "generated in an environment using symengine 0.9.0. If you need to load this file "
+                "you can do so with Qiskit 0.45.x or 0.46.x and re-export the QPY file using "
+                "`use_symengine=False`."
+            )
         if minor not in (11, 13):
             raise exceptions.QpyError(
                 f"Incompatible symengine version {major}.{minor} used to generate the QPY "

qiskit/qpy/interface.py

@@ -144,6 +144,16 @@ def dump(
                 from the QPY format at that version will persist. This should only be used if
                 compatibility with loading the payload with an older version of Qiskit is necessary.
 
+            .. note::
+
+                If serializing a :class:`.QuantumCircuit` or :class:`.ScheduleBlock` that contain
+                :class:`.ParameterExpression` objects with version set to
+                :attr:`.qpy.QPY_COMPATIBILITY_VERSION` with the intent to load the payload using
+                a historical release of Qiskit, ensure you set the ``use_symengine`` flag to
+                ``False``. The symengine versions used between the Qiskit 1.0 major version boundary
+                are not compatible and you will be unable to load the QPY file in those cases.
+
+
     Raises:
         QpyError: When multiple data format is mixed in the output.
         TypeError: When invalid data type is input.

releasenotes/notes/fix-qpy-symengine-compat-858970a9a1d6bc14.yaml

@@ -11,3 +11,29 @@ fixes:
     mismatch with this version of Qiskit this might not work. You will need to
     install Qiskit >1.3.0 to fix this mismatch issue more broadly for any potential
     future version of symengine.
+issues:
+  - |
+    When dumping a QPY file using :func:`.qpy.dump` with the ``version=10`` flag and there are
+    unbound :class:`.ParameterExpression` in the circuit the output file
+    will not be loadable by Qiskit 0.45.x or 0.46.x due to incompatibilities
+    in the symengine serialization used between the symengine libraries used
+    in the different versions of Qiskit. Qiskit >= 1.0 requires symengine >=
+    0.11 and Qiskit < 1.0 required symengine >= 0.9.0 and < 0.10.0. Symengine
+    0.9.x and 0.11.0 (or 0.13.0) don't have a compatible serialization formats
+    which prevents the files from being loaded. In these cases the only option
+    available to generate a QPY file with unbound
+    :class:`.ParameterExpression` that can be loaded with Qiskit 0.45.x or
+    0.46.x is to set both ``version=10`` and ``use_symengine=False``.
+  - |
+    When loading a QPY file generated with Qiskit 0.45.x or 0.46.x and the
+    ``use_symengine`` flag set to ``True`` (an optional feature), these
+    payloads are not parsable with Qiskit >= 1.0 (including this release).
+    This is due to changes in the required symengine versions across the
+    Qiskit 1.0 major version boundary. Qiskit >= 1.0 requires symengine >=
+    0.11 and Qiskit < 1.0 required symengine >= 0.9.0 and < 0.10.0. Symengine
+    0.9.x and 0.11.0 (or 0.13.0) don't have a compatible serialization formats
+    which prevents newer version of Qiskit from loading those payloads. If you
+    have a QPY file generated with Qiskit 0.45.x or 0.46.x and the
+    ``use_symengine`` flag was set to ``True`` your only option is to load
+    the QPY file using Qiskit 0.46.3 and regenerate it using
+    ``use_symengine=False``.