Create VJP perf fix

[mlxd]

Before submitting

Please complete the following checklist when submitting a PR:

• All new features must include a unit test.
  If you've fixed a bug or added code that should be tested, add a test to the
  test directory!

• All new functions and code must be clearly commented and documented.
  If you do make documentation changes, make sure that the docs build and
  render correctly by running make docs.

• Ensure that the test suite passes, by running make test.

• Add a new entry to the doc/releases/changelog-dev.md file, summarizing the
  change, and including a link back to the PR.

• The PennyLane source code conforms to
  PEP8 standards.
  We check all of our code against Pylint.
  To lint modified files, simply pip install pylint, and then
  run pylint pennylane/path/to/file.py.

When all the above are checked, delete everything above the dashed
line and fill in the pull request template.

---

Context: v0.33.1 backport of PR #4792

Description of the Change: Updates VJP pipeline to favour direct matrix-vector products where possible.

Benefits: Improves performance for many parameter/many observable workloads

Possible Drawbacks:

Related GitHub Issues: #4792

[github-actions]

Hello. You may have forgotten to update the changelog!
Please edit doc/releases/changelog-dev.md with:

• A one-to-two sentence description of the change. You may include a small working example for new features.
• A link back to this PR.
• Your name (or GitHub username) in the contributors section.

[trbromley]

Thanks @mlxd @albi3ro! Nothing blocking from my side but I do have a few questions.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (1b20871) 99.64% compared to head (4136300) 99.64%.
Report is 2 commits behind head on v0.33.1-rc0.

Additional details and impacted files

@@             Coverage Diff              @@
##           v0.33.1-rc0    #4806   +/-   ##
============================================
  Coverage        99.64%   99.64%           
============================================
  Files              380      380           
  Lines            34266    34285   +19     
============================================
+ Hits             34145    34164   +19     
  Misses             121      121           

Files	Coverage Δ
pennylane/_version.py	100.00% <100.00%> (ø)
pennylane/devices/qubit/simulate.py	100.00% <100.00%> (ø)
pennylane/gradients/vjp.py	100.00% <100.00%> (ø)
pennylane/math/multi_dispatch.py	99.59% <100.00%> (+<0.01%)	⬆️
pennylane/ops/op_math/symbolicop.py	100.00% <100.00%> (ø)
pennylane/transforms/defer_measurements.py	100.00% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[rmoyard]

Thanks for the changes! It would be nice to have the list of cases where the it becomes slower because of the fallback. What I can identify is the following:

The performance is better:

1. Single measurement (not tensorflow) with parameters of the same shape.
2. Multiple measurements with same shapes and with parameters of the same shape.

The performance is worse:

1. Parameters of the different shapes.
2. Tensorflow in general takes a hit because it does not support @ for mulitplication
3. Mixing measurements with different shapes is slower because of the fallback in the compute_vjp_multi.

In the future we should come back to dispatch the cases properly.

[timmysilv]

try-except generally hurts performance. I made a table of when we need which try-excepts so we can improve this in the future:

	non-TF	TF
non-ragged	none	both
ragged	multi	both

[rmoyard]

Thanks! We made a small table of what cases need what try expect and it also show which one are better and which one become worse. It will be easier to come back to this issue.

[albi3ro]

Performance improvements for the vast majority of hpc cases we care about, at the cost of a penality for edge cases and tensorflow. I find that to be an acceptable tradeoff.

While this code could probably always use more fine-tuning, this makes accomplishes exactly what it needs to accomplish,