QPY schedule serialization #7300

nkanazawa1989 · 2021-11-23T07:24:17Z

Summary

This PR adds qiskit.qpy that covers qpy serialization of both QuantumCircuit and ScheduleBlock which is requirement of Qiskit Experiments.

Details and comments

Currently only basic component for MVP

from qiskit import pulse, circuit
from qiskit.qpy.objects.schedules import _write_instruction, _read_instruction
import io

freq = circuit.Parameter("freq")
ch = circuit.Parameter("ch0")
inst = pulse.SetFrequency(freq, pulse.DriveChannel(ch))

container = io.BytesIO()
_write_instruction(container, inst)

container.seek(0)
container.read() # b'\x00\x0c\x00\x00\x00\x02SetFrequencyp\x00\x00\x00\x00\x00\x00\x00\x16\x00\x04\x84\xc4\x06\x13\x0e(G\x1d\x8b\xd0\xf3\xc9D\xd7\x1dXfreqd\x00\x00\x00\x00\x00\x00\x00&\x00\x0cp\x00\x15DriveChannel\x00\x03R\x89\xb3\xf3x2G\xbe\x86\xf8\xddX\xea\xcbe\x85ch0'

container.seek(0)
read_inst = _read_instruction(container); read_inst  # SetFrequency(freq, DriveChannel(ch0))

inst == read_inst  # True

Because in pulse schedule representation (in contrast to QuantumCircuit), parameterizable operand values exit across multiple layers from alignment context parameter to pulse values. To eliminate boilerplate code in each layer, parameter value serialization logic is isolated without changing data structure so that it can be reused.

TODO

ScheduleBlock serialization
Frame change instructions () serialization
Play serialization
~~[ ] Call serialization~~
Context serialization
~~[ ] Discriminator/Kernel serialization~~
Circuit QPY migration
Unittest
Documentation
release note

QuantumCircuit.calibrations field serialization will be done in follow-up.

coveralls · 2021-11-23T07:56:19Z

Pull Request Test Coverage Report for Build 2545753902

645 of 699 (92.27%) changed or added relevant lines in 11 files are covered.
6 unchanged lines in 2 files lost coverage.
Overall coverage increased (+0.08%) to 84.377%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
qiskit/qpy/binary_io/circuits.py	71	72	98.61%
qiskit/pulse/transforms/alignments.py	43	45	95.56%
qiskit/qpy/binary_io/schedules.py	162	164	98.78%
qiskit/qpy/common.py	50	54	92.59%
qiskit/qpy/interface.py	38	42	90.48%
qiskit/qpy/binary_io/value.py	53	65	81.54%
qiskit/qpy/type_keys.py	201	230	87.39%

Files with Coverage Reduction	New Missed Lines	%
qiskit/pulse/transforms/alignments.py	3	93.79%
qiskit/qpy/common.py	3	91.03%

Totals
Change from base Build 2545752111:	0.08%
Covered Lines:	55498
Relevant Lines:	65774

💛 - Coveralls

nkanazawa1989 · 2021-11-30T07:05:47Z

qiskit/qpy/objects/schedules.py

+
+def _write_measure_processor(file_obj, data):
+    name = data.name.encode("utf8")
+    params = json.dumps(data.params, separators=(",", ":")).encode("utf8")


I assume parameters of Kernel and Discriminator are a free-form but json serializable dictionary (i.e. no numpy, no parameter, etc...). However I've never seen actual spec of these objects. @taalexander do you have any information?

nkanazawa1989 · 2021-11-30T11:48:44Z

qiskit/qpy/objects/circuits.py

+
+def _parse_custom_instruction(custom_instructions, gate_name, params):
+    (type_str, num_qubits, num_clbits, definition) = custom_instructions[gate_name]
+    if type_str == "i":


TypeKey enum strategy doesn't work here because i is also used for Integer (because key is not scoped). This logic imperfection looks bit awkward (but removing enum makes pulse qpy code really hard to read because there are many kinds of operand classes).

I wonder if we want to have a custom instruction table type key enum defined in this module too? They're circuit specific but likely will grow more over time as we have special custom circuit operations with different representations that are needed. For example QPY v3 added a "p" here for PauliEvolutionGate

…hedule-serialization

mtreinish

This LGTM, thanks for the quick fixes. I left a few questions inline about the size of certain fields. I don't think it's a blocker at all (I'm going to tag as automerge) the choices you made are reasonable but I wanted to just raise the question so we have it in the review log, primarily because if we do need to bump any we'll have to bump the QPY version.

mtreinish · 2022-06-22T17:36:59Z

qiskit/qpy/formats.py

@@ -155,8 +154,49 @@
 CUSTOM_CIRCUIT_INST_DEF_PACK = "!H1cII?Q"
 CUSTOM_CIRCUIT_INST_DEF_SIZE = struct.calcsize(CUSTOM_CIRCUIT_INST_DEF_PACK)

+# CALIBRATION
+CALIBRATION = namedtuple("CALIBRATION", ["num_cals"])
+CALIBRATION_PACK = "!I"


I'm not sure how many calibrations are typical on a circuit, this sets the upper bound at 2**32 which is probably sufficient but I'm wondering if we want to allow more or less.

mtreinish · 2022-06-22T17:37:44Z

qiskit/qpy/formats.py

+
+# CALIBRATION_DEF
+CALIBRATION_DEF = namedtuple("CALIBRATION_DEF", ["name_size", "num_qubits", "num_params", "type"])
+CALIBRATION_DEF_PACK = "!III1c"


Same question here as above we're limiting the calibration name, number of qubits, and number of parameters to 2**32. I think that's reasonable but maybe too high.

mtreinish · 2022-06-22T17:39:11Z

qiskit/qpy/formats.py

+        "amp_limited",
+    ],
+)
+SYMBOLIC_PULSE_PACK = "!HHHH?"


This limits the number of bytes to the envelope, constraints, and valid amp conditions size to 2**16 bytes. I know they're zlib compressed but do you have as sense for how large these can get?

mtreinish · 2022-06-22T17:39:55Z

qiskit/qpy/formats.py

+
+# MAP_ITEM
+MAP_ITEM = namedtuple("MAP_ITEM", ["key_size", "type", "size"])
+MAP_ITEM_PACK = "!H1cH"


Do you think we can have keys more than 2*16 characters?

The support for serializing SymbolicPulse objects using QPY was added in PR Qiskit#7300 and has been released as part of 0.21.0. However, the documentation for the SymbolicPulse class had a note saying the QPY support was pending. This was necessary when the SymbolicPulse class was added in Qiskit#7821 because it was unclear when we'd be able to add the QPY support. But, Qiskit#7300 was updated and merged soon after Qiskit#7821, but we neglected to remove that note. This commit corrects the oversight and removes the stale note.

The support for serializing SymbolicPulse objects using QPY was added in PR #7300 and has been released as part of 0.21.0. However, the documentation for the SymbolicPulse class had a note saying the QPY support was pending. This was necessary when the SymbolicPulse class was added in #7821 because it was unclear when we'd be able to add the QPY support. But, #7300 was updated and merged soon after #7821, but we neglected to remove that note. This commit corrects the oversight and removes the stale note. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

The support for serializing SymbolicPulse objects using QPY was added in PR #7300 and has been released as part of 0.21.0. However, the documentation for the SymbolicPulse class had a note saying the QPY support was pending. This was necessary when the SymbolicPulse class was added in #7821 because it was unclear when we'd be able to add the QPY support. But, #7300 was updated and merged soon after #7821, but we neglected to remove that note. This commit corrects the oversight and removes the stale note. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> (cherry picked from commit 47c7c27)

…#8292) The support for serializing SymbolicPulse objects using QPY was added in PR #7300 and has been released as part of 0.21.0. However, the documentation for the SymbolicPulse class had a note saying the QPY support was pending. This was necessary when the SymbolicPulse class was added in #7821 because it was unclear when we'd be able to add the QPY support. But, #7300 was updated and merged soon after #7821, but we neglected to remove that note. This commit corrects the oversight and removes the stale note. Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> (cherry picked from commit 47c7c27) Co-authored-by: Matthew Treinish <mtreinish@kortar.org>

Co-authored-by: Matthew Treinish <mtreinish@kortar.org> Co-authored-by: Naoki Kanazawa <nkanazawa1989@gmail.com>

* Copy Qiskit/qiskit#8055 Co-authored-by: Matthew Treinish <mtreinish@kortar.org> * Copy qpy changes from Qiskit/qiskit#8093 Co-authored-by: Jake Lishman <jake@binhbar.com> Co-authored-by: Jake Lishman <jake.lishman@ibm.com> * Copy Qiskit/qiskit#8200 Co-authored-by: Matthew Treinish <mtreinish@kortar.org> * Copy of Qiskit/qiskit#7300 Co-authored-by: Matthew Treinish <mtreinish@kortar.org> Co-authored-by: Naoki Kanazawa <nkanazawa1989@gmail.com> * Copy of Qiskit/qiskit#8235 Co-authored-by: Matthew Treinish <mtreinish@kortar.org> Co-authored-by: Matthew Treinish <mtreinish@kortar.org> Co-authored-by: Jake Lishman <jake@binhbar.com> Co-authored-by: Jake Lishman <jake.lishman@ibm.com> Co-authored-by: Naoki Kanazawa <nkanazawa1989@gmail.com>

nkanazawa1989 assigned mtreinish Nov 23, 2021

nkanazawa1989 added mod: qpy Related to QPY serialization mod: pulse Related to the Pulse module labels Nov 23, 2021

Schedule serialization wip

f32989c

nkanazawa1989 force-pushed the feature/schedule-serialization branch from 2ed0a9a to f32989c Compare November 23, 2021 07:29

nkanazawa1989 added 7 commits November 24, 2021 16:48

pulse serialization

59cb2c4

move mapping io to separate file

dab2484

move dump numbers to parameter_values

81746b0

add alignment context serialization

3b01c81

block serialization

354e075

add interface

04cbf6d

bug fix

2033738

nkanazawa1989 force-pushed the feature/schedule-serialization branch from adc2e58 to 2033738 Compare November 25, 2021 13:58

nkanazawa1989 added 2 commits November 25, 2021 23:59

black / lint

242b0f3

documentation

8ba02f7

nkanazawa1989 mentioned this pull request Nov 30, 2021

PulseDefaults from_dict() method doesn't work with parameterized amplitudes. #7318

Closed

add serialization for kernel and discriminator

4b585d8

nkanazawa1989 commented Nov 30, 2021

View reviewed changes

migrate circuit qpy to qpy module

21bfe1b

nkanazawa1989 force-pushed the feature/schedule-serialization branch from c263532 to 21bfe1b Compare November 30, 2021 11:44

nkanazawa1989 commented Nov 30, 2021

View reviewed changes

nkanazawa1989 and others added 4 commits November 30, 2021 21:04

keep import path

20fd393

reno

64965b7

lint

9836cc6

Merge branch 'main' into feature/schedule-serialization

b8ee92b

nkanazawa1989 marked this pull request as ready for review November 30, 2021 13:28

nkanazawa1989 requested a review from a team as a code owner November 30, 2021 13:29

nkanazawa1989 assigned eggerdj Nov 30, 2021

fix import path

82520a8

nkanazawa1989 added 2 commits June 22, 2022 11:41

Merge branch 'main' of github.com:Qiskit/qiskit-terra into feature/sc…

6df43aa

…hedule-serialization

fix merge and cleanup

a148052

nkanazawa1989 force-pushed the feature/schedule-serialization branch 2 times, most recently from da0715b to 1f836c4 Compare June 22, 2022 05:37

add circuit calibrations serialization

7c9f829

nkanazawa1989 force-pushed the feature/schedule-serialization branch from 1f836c4 to 7c9f829 Compare June 22, 2022 05:46

fix qpy_compat test

8aee1d5

mtreinish linked an issue Jun 22, 2022 that may be closed by this pull request

qpy serialization for pulse schedule #6923

Closed

nkanazawa1989 added 3 commits June 23, 2022 01:18

remove compatibility test for parameterized program

8d666ab

Merge branch 'main' of github.com:Qiskit/qiskit-terra into feature/sc…

ced2f68

…hedule-serialization

update reno for calibrations

39b34b1

mtreinish previously approved these changes Jun 22, 2022

View reviewed changes

mtreinish added Changelog: Deprecation Include in "Deprecated" section of changelog Changelog: New Feature Include in the "Added" section of the changelog automerge labels Jun 22, 2022

fix data size

fe46dad

nkanazawa1989 dismissed mtreinish’s stale review via fe46dad June 22, 2022 17:53

mtreinish approved these changes Jun 22, 2022

View reviewed changes

Merge branch 'main' into feature/schedule-serialization

b9c4b54

mergify bot merged commit c650341 into Qiskit:main Jun 23, 2022

mtreinish mentioned this pull request Jul 3, 2022

Remove stale note about pending QPY support for SymbolicPulse #8289

Merged

rathishcholarajan added a commit to rathishcholarajan/qiskit-ibm-runtime that referenced this pull request Jul 5, 2022

Copy of Qiskit/qiskit#7300

bca1ba7

Co-authored-by: Matthew Treinish <mtreinish@kortar.org> Co-authored-by: Naoki Kanazawa <nkanazawa1989@gmail.com>

jyu00 mentioned this pull request Jul 6, 2022

Sampler should support pulse gates Qiskit/qiskit-ibm-runtime#413

Closed

garrison mentioned this pull request Oct 11, 2022

Fix deprecation warning for deprecated_functionality #8851

Merged

nkanazawa1989 deleted the feature/schedule-serialization branch November 25, 2022 02:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

QPY schedule serialization #7300

QPY schedule serialization #7300

nkanazawa1989 commented Nov 23, 2021 •

edited

Loading

coveralls commented Nov 23, 2021 •

edited

Loading

nkanazawa1989 Nov 30, 2021

nkanazawa1989 Nov 30, 2021

mtreinish Jan 24, 2022

mtreinish left a comment

mtreinish Jun 22, 2022

mtreinish Jun 22, 2022

mtreinish Jun 22, 2022

mtreinish Jun 22, 2022

QPY schedule serialization #7300

QPY schedule serialization #7300

Conversation

nkanazawa1989 commented Nov 23, 2021 • edited Loading

Summary

Details and comments

coveralls commented Nov 23, 2021 • edited Loading

Pull Request Test Coverage Report for Build 2545753902

💛 - Coveralls

nkanazawa1989 Nov 30, 2021

Choose a reason for hiding this comment

nkanazawa1989 Nov 30, 2021

Choose a reason for hiding this comment

mtreinish Jan 24, 2022

Choose a reason for hiding this comment

mtreinish left a comment

Choose a reason for hiding this comment

mtreinish Jun 22, 2022

Choose a reason for hiding this comment

mtreinish Jun 22, 2022

Choose a reason for hiding this comment

mtreinish Jun 22, 2022

Choose a reason for hiding this comment

mtreinish Jun 22, 2022

Choose a reason for hiding this comment

nkanazawa1989 commented Nov 23, 2021 •

edited

Loading

coveralls commented Nov 23, 2021 •

edited

Loading