Add StagedPassManager class for pass manager with defined stages

[mtreinish]

Summary

This commit adds a new PassManager subclass, StragedPassManager. This class
is used to have a PassManager with a defined structure and stages. For example,
when running the normal transpile() workflow. The preset pass managers are then updated
to be StragedPassManager objects they conform to the fixed structure. Having
a class with defined phases gives us flexibility in the future for making
the transpiler pluggable with external plugins (similar to what's done in
PR #6124) and also have backend hook points before or after different
phases of the transpile.

Details and comments

Fixes #5978

[mtreinish]

@ajavadia I've updated the preset passmanagers to abstract away the common code. There is still a bit of duplication in the layout and routing pass creation, but I opted to leave separate copies in each level mostly because the control flow logic (for layout) is different at each level and the parameters for some of the passes are different at each optimization level.

[nkanazawa1989]

Can you add a routine for scheduling? This is kind of post_optimization but seems like we are going to add many of them for pulse gate experiment (scheduled circuit, alignment, dynamical decoupling, pulse gate #6759)

[mtreinish]

Can you add a routine for scheduling? This is kind of post_optimization but seems like we are going to add many of them for pulse gate experiment (scheduled circuit, alignment, dynamical decoupling, pulse gate #6759)

I apologize for the delay I missed your comment here somehow. I've added a new scheduling stage to the end of the FullPassManager class in: f177d5f

[nkanazawa1989]

Thanks Matthew. This looks great. How easily we can update signature of generate_scheduling_post_opt method? Integration of circuit/schedule is somewhat of active development, and required argument set may change in future.

[kdk]

I like the added structure this brings (and the prospect of helping users and providers hook into the transpilation process). I have some mild reservations about the implementation as a PassManager subclass in that it seems less flexible than adding support for labeled stages in the PassManager (along the lines of #5069 ) both for future development of the presets and for other outside use cases. (It may be that a subclass is the interface we want to achieve this structure without needing #5069 .)

My only other concern is with regards to readability of the presets (similar to #3437 ). With the definition of stages (and corresponding functions to build them), it should be easier for newcomers to understand how the preset pass managers are built, and what will be run when, but we should collect some opinions here. (For me, consistency is key, but others may feel differently.)

There's also a failing QPY backwards compatibility test, but I'm not sure if it's related to these changes: https://dev.azure.com/qiskit-ci/qiskit-terra/_build/results?buildId=31290&view=logs&j=8fcf860c-8d5a-5d3a-546e-59924dc71ae1&t=b5052aa0-c7a8-5238-40d6-4558d53c7300&l=117

I like the direction this is heading, thanks for taking this on.

[kdk]

Is there a reason not to include these in generate_embed_passmanager?

[mtreinish]

IIRC it's because the condition differs between the optimization levels. See my comment below (which I meant to put here not there, but it kind of works for either). Basically I was trying to keep things in common isolated to just the pieces which are shared completely between each optimization level. That way you can got to a specific optimization level's code and see what is shared and what is unique.

[kdk]

I think this is a step in the right direction, but I think this leaves the preset pass managers in a state that's somewhat harder to read and reason about, because some of the stages are generated elsewhere and some (and their control flow) are generated here.

Could we push all the control flow in to the generate_* functions (so that every full pass manager has all the same stages, just some may be no-ops depending on the options or backend) with either different functions or arguments for the different optimization levels?

[mtreinish]

I think I did it this way because the control flow and logic around layout was different between the optimization levels. Like 0 is:

if initial_layout is not None:
    layout = initial_layout
else:
    layout = tivial_layout()

but level 1 is:

if initial_layout is not None:
    layout = initial_layout
elif trivial_layout().is_perfect():
    layout = trivial_layout()
else:
    layout = DenseLayout()

but 2 and 3 are:

if initial_layout is not None:
    layout = initial_layout
elif trivial_layout().is_perfect():
    layout = trivial_layout()
elif csplayout.converges():
   layout = csplayout()
else:
    layout = DenseLayout()

(where the parameters for csp differ)

I was trying to keep things in common isolated to just the pieces which are shared completely between each optimization level. That way you can got to a specific optimization level's code and see what is shared and what is unique.

Comments addressed

[1ucian0]

Let's get this merged! Thanks!

[pedrorrivero]

A few comments and bugs :)

[pedrorrivero]

Shouldn't it be this?

Suggested change

	or (attr.startswith("pre_") and attr[4:] not in self.stages)
	or (attr.startswith("post_") and attr[5:] not in self.stages)
	or (attr.startswith("pre_") and attr[4:] in self.stages)
	or (attr.startswith("post_") and attr[5:] in self.stages)

[pedrorrivero]

What would happen if I assign a StagedPassManager as a stage of itself (i.e. recursively)? Wouldn't _update_passmanager() become unstable?

[pedrorrivero]

Suggested change

	def __getitem(self, index):
	def __getitem__(self, index):

[pedrorrivero]

Wouldn't it make sense for this to be a @property?