Remove transpiler optimizations from the benchmarks

[atilag]

Summary

Transpilation process is not mandatory in simulation, and it can add different optimizations to the circuits over time that will interfere in the results of the benchmarks, so we are removing it from all the benchmarks.

Anyway, I want to make sure that this assumption is correct @chriseclectic :)

Details and comments

We may notice visible performance peak in the benchmarks right after this PR, that is expected.

[chriseclectic]

@atilag I disagree. Transpiling to u3, cx was intentional to force the arbitrary 2-qubit untiaries to be decomposed into CX and single-qubit gates, which make different implementations of quantum volume circuits equivalent. Otherwise you might have 2-qubit arbitrary unitaries that are simulated directly (if the circuit was generated that way) so you are not benchmarking core performance of single and two qubit gates, or optimizations like gate fusion.

The transpilation time shouldn't be included in the benchmark though, only the simulation time of the qobj after assembly.

If we are worried about other transpilation passes changing performance due to changing optimizations we should just make a custom passmanager for benchmarks that only uses the unroller and no other transpilation passes.

[atilag]

Ok, so let's keep transpilation but disable optimizations, the unrolling will still apply.
@kdk suggested to set a fixed value for seed_transpiler too.

[chriseclectic]

@atilag After thinking more on this I think what we really want two three tiers of benchmarks:

• Benchmarks for low-level functions (applying matrices, kraus, etc)
• Benchmarks for synthetic applications (Ideal ignis, aqua algorithms, etc)
• benchmarks for realistic applications (noisy ignis, aqua algorithms, etc)

All of these are necessary for assessing PRs for performance gains, however the second point could also be used for comparing Aer to other simulators (which we don't do yet but should), and the third is the most useful for "real world" uses of the simulator by researchers. In that last two cases we would want to report whatever gives to fastest overall execution time (likely doesn't involve unrolling).

Quantum volume is a bit odd as it could fit into all of those cases, and at the moment I was thinking of it more as the former case (benchmarking random u3 and cx gates), or the latter (circuit is designed to benchmark device quality in the presence of errors). In both those cases you would have to runroll to simulate the noise model at gate level.

However if we want to consider QV as a synthetic application, then the best overall execution time, which would be to build the circuit using random SU(4) unitary matrices (not cx,u3) and then then unrolling as you have done in this PR: This is because the unroller decomposition of SU(4) into CX,u3 gates is not optimal. One issue here is several different pieces of code used to generate quantum volume circuits and not all are equal (some use cx, u3 directly, others use SU(4) operators). Once quantum volume is in ignis we will have a "canonical" circuit that can be used for evaluation.

[atilag]

This makes a lot of sense.
I would merge this PR, which is an improvement, and address the changes you mention in a follow up.